基于LoRA微调与RAG融合的煤矿专业大模型应用关键技术

Key technologies for application of coal mine domain large model based on LoRA fine-tuning and RAG fusion

  • 摘要: 目前煤矿行业大模型仅对用户的提问进行知识问答,未与现场实时数据相关联,无法对煤矿生产运行状况进行实时分析与指导。针对这些问题,提出一种基于大语言模型的低阶适应(LoRA)微调和检索增强生成(RAG)融合的煤矿专业大模型。该模型先运用LoRA技术从历史文本语料中抽取出知识实体并定义知识结构输入大模型进行微调,使大模型能够深入理解领域知识,再将实时产生的生产数据、实时更新的作业规程、法规条例等数据经过向量化清洗输入向量数据库,并与RAG的检索机制相结合,确保数据信息的实时性和准确性。实验结果表明:① 经 LoRA 微调后,模型回答可以精准契合某煤矿 “一通三防” 管理制度汇编,不仅详细阐述了控制瓦斯排放的增阻限风、分风限风、逐段排放等具体方法,还对排放时间计算、传感器设置、图纸绘制及断电撤人等操作细则进行说明,实现了从泛泛而谈到精准定位具体煤矿特定文件内容的跨越。② 选取现场143万条液压支架时序数据,分别存入Milvus向量数据库与MySQL关系型数据库,从写入效率与查询性能2个维度进行对比,结果表明:Milvus向量数据库写入速度为MySQL关系型数据库的2.4倍;在向量检索场景方面,Milvus的向量相似度检索延迟稳定在20 ms,在混合查询场景效率方面,MySQL需全表扫描后排序,143万条数据延迟超100 ms,而Milvus将设备ID过滤后的子集输入分层可导航小世界图(HNSW)层,仅读取查询涉及的向量字段,避免了全表扫描。③ 将本地基于LoRA微调与RAG融合的煤矿专业大模型与本地离线deepseekR1−7b模型进行部署,对多项指标进行测试,结果表明:基于LoRA微调与RAG融合的煤矿专业大模型在煤矿专业领域知识学习性、知识动态化更新时效性、模型泛化与回答精确度方面具有显著优势,为工业级AI落地提供了可行路径。

     

    Abstract: At present, large models in the coal mine industry only perform knowledge question answering for users’ queries, without being linked to real-time on-site data, and therefore cannot conduct real-time analysis and guidance on coal mine production and operation conditions. To address these problems, a coal mine domain large model based on Low-Rank Adaptation (LoRA) fine-tuning and Retrieval-Augmented Generation (RAG) fusion was proposed. The model first used LoRA technology to extract knowledge entities from historical text corpora and define knowledge structures, which were then input into the large model for fine-tuning, enabling the fine-tuned large model to deeply understand domain knowledge. Then, real-time production data, updated operating procedures, and regulations were vectorized, cleaned, and input into a vector database, and combined with the retrieval mechanism of RAG to ensure the timeliness and accuracy of the information. Experimental results showed that: ① after LoRA fine-tuning, the model’s answers precisely matched a certain coal mine's "One Ventilation and Three Prevention" management regulations compilation, not only elaborating specific methods such as increasing resistance to limit airflow, branch airflow limiting, and section-by-section discharge for controlling gas emissions, but also explaining operational details such as discharge time calculation, sensor setting, drawing preparation, and power-cut evacuation, thus achieving a leap from general discussion to precisely locating the content of specific coal mine documents. ② A total of 1.43 million items of hydraulic support time-series data from the site were stored separately in the Milvus vector database and the MySQL relational database. A comparison was made in two dimensions: write efficiency and query performance. The results showed that the write speed of the Milvus vector database was 2.4 times that of MySQL. In vector retrieval scenarios, the vector similarity retrieval latency of Milvus was stable at the 20 ms level; in hybrid query scenarios, MySQL needed to perform a full table scan followed by sorting, with a latency exceeding 100 ms for 1.43 million data entries, whereas Milvus filtered the subset by equipment ID and then input it into the Hierarchical Navigable Small World (HNSW) graph, reading only the vector fields involved in the query, thereby avoiding a full table scan. ③ The locally deployed coal mine domain large model based on LoRA fine-tuning and RAG fusion, along with and the offline DeepSeekR1-7b model, were tested on multiple indicators. The results show that the coal mine domain large model based on LoRA fine-tuning and RAG fusion has significant advantages in domain knowledge learning, timeliness of dynamic knowledge updates, model generalization, and answer accuracy, providing a feasible path for the industrial application of AI.

     

/

返回文章
返回