UeDiff-GAN-based target detection and twin synchronization mapping for fully mechanized mining faces
-
摘要:
矿井综采工作面数字孪生模型的构建过程需要手动构建实体的数字孪生3D模型，然后对实体进行目标检测，并根据实时检测结果控制3D模型，以确保孪生体与实体之间的同步映射关系。因此，对井下目标的实时、准确检测是实现虚实同步映射控制的关键。目前主流的目标检测方法需要在传统模型中引入或改进模块，使得模型网络结构复杂、训练周期较长，降低了目标检测的实时性；同时，对于一些含高强度噪声的图像难以精确检测。针对上述问题，提出了一种基于UeDiff−GAN的综采工作面目标检测与孪生体同步映射方法。通过扩散模型对高质量样本进行加噪扩散，得到不同程度的加噪样本，然后使用生成对抗网络（GAN）模型进行训练；设计了平滑扩散算法，以控制扩散步长，加入不均衡扩散模块，以得到与预识别样本匹配的检测算法模型。使用Unity3D构建综采工作面3D模型并进行渲染，实现井下物理实体的对象孪生，据此构建综采工作面实体与其孪生模型的映射关系，根据井下不同位置的检测结果控制对应机器运动状态及姿态，实现孪生模型协同控制，从而实现过程孪生。在自制数据集上的实验结果表明：UeDiff−GAN模型对井下移动目标的平均检测精度较SSD，R−CNN，YOLOv7和Diff−GAN模型分别提升了19.4%，14.3%，9.1%，24.3%；检测速度较SSD，R−CNN分别提升了13.86，42.73 帧/s；孪生模型与实体的实时性延迟至多为0.873 s。
Abstract:The construction of a digital twin model for fully mechanized mining faces requires manually creating a digital twin 3D model of a physical entity, performing target detection on the entity, and adjusting the 3D model based on real-time detection results to ensure synchronization mapping between the twin and the physical entity. Therefore, real-time and accurate detection of underground targets is crucial for achieving virtual-physical synchronization mapping control. Current mainstream target detection methods require incorporating or modifying modules within traditional models, resulting in complex network structures and prolonged training cycles, which reduce the real-time performance of target detection. Moreover, these methods struggle with precisely detecting targets in images with high-intensity noise. To address these issues, this study proposed a UeDiff-GAN-based target detection and twin synchronization mapping method for fully mechanized mining faces. The diffusion model was used to add noise into high-quality samples to generate samples of varying levels, which were then used to train a generative adversarial network (GAN) model. A smooth diffusion algorithm was designed to regulate the diffusion step sizes, while an imbalanced diffusion module was incorporated to obtain a detection algorithm model that matches pre-identified samples. A 3D model of the fully mechanized mining face was constructed and rendered using Unity3D, achieving a digital twin of underground physical entities. Based on this model, a mapping relationship between the physical entity and its twin model was established. The corresponding machine's motion state and posture are controlled according to the detection results at different underground locations. This approach enabled twin model coordinated control, thereby achieving process-level twinning. Experimental results on a self-developed dataset demonstrated that the UeDiff-GAN model improved the average detection accuracy of underground moving targets by 19.4%, 14.3%, 9.1%, and 24.3% compared to SSD, R-CNN, YOLOv7, and Diff-GAN models, respectively. The detection speed improved by 13.86, 42.73 frames per second (fps) compared to SSD and R-CNN models, respectively. The real-time delay between the twin model and the physical entity was at a maximum of 0.873 seconds.
-
-
![]()
图 4 综采工作面3D模型构建流程
Figure 4. Construction process of 3D model for fully mechanized mining face
![]()
图 5 综采工作面“三机”实体与孪生体协同控制关系
Figure 5. Collaborative control relationship of physical entity and twin entity of "three machines" in fully mechanized mining face
表 1 各模型对比结果
Table 1 Comparison of different models
模型 检测精度% 平均检测
精度%检测速度/
（帧·s−1）采煤机 人 车辆 SSD 72.3 87.1 71.9 77.1 45.60 R−CNN 77.4 88.2 75.8 80.5 16.73 YOLOv7 82.6 90.9 81.0 84.8 70.56 Diff−GAN 67.4 74.9 65.1 69.1 59.71 UeDiff−GAN 91.7 92.4 91.0 91.7 59.46 
下载: 导出CSV
表 2 消融实验结果
Table 2 Results of ablation experiment
模型 检测精度% 平均检测
精度%检测速度/
（帧·s−1）采煤机 人 车辆 Diff−GAN 67.4 74.9 65.1 69.1 59.71 模型1 88.7 89.3 88.1 88.7 59.50 模型2 74.3 83.0 73.1 76.8 59.66 UeDiff−GAN 91.7 92.4 91.0 91.7 59.46
下载: 导出CSV
-
[1] 邢震. 面向智能矿山的数字孪生技术研究进展[J]. 工矿自动化,2024,50(3):22-34,41. XING Zhen. Research progress on digital twin technology for intelligent mines[J]. Journal of Mine Automation,2024,50(3):22-34,41.
[2] 张旭辉. 打造数字孪生行业生态,突破采掘装备远程控制[J]. 西安科技大学学报,2023,43(1):2. ZHANG Xuhui. Creating digital twin industry ecology,advancing remote control of mining equipment[J]. Journal of Xi'an University of Science and Technology,2023,43(1):2.
[3] 鲍久圣,张可琨,王茂森,等. 矿山数字孪生MiDT:模型架构、关键技术及研究展望[J]. 绿色矿山,2023(1):166-177. BAO Jiusheng,ZHANG Kekun,WANG Maosen,et al. Mine digital twin:model architecture,key technologies and research prospects[J]. Journal of Green Mine,2023(1):166-177.
[4] 张帆,葛世荣,李闯. 智慧矿山数字孪生技术研究综述[J]. 煤炭科学技术,2020,48(7):168-176. ZHANG Fan,GE Shirong,LI Chuang. Research summary on digital twin technology for smart mines[J]. Coal Science and Technology,2020,48(7):168-176.
[5] 张帆,葛世荣. 矿山数字孪生构建方法与演化机理[J]. 煤炭学报,2023,48(1):510-522. ZHANG Fan,GE Shirong. Construction method and evolution mechanism of mine digital twins[J]. Journal of China Coal Society,2023,48(1):510-522.
[6] 熊思博,王琦,刘光洁. 三维重建技术的发展与现状研究综述[J]. 电脑知识与技术,2022,18(36):114-117. XIONG Sibo,WANG Qi,LIU Guangjie. A review on the development and current status of 3D reconstruction technology[J]. Computer Knowledge and Technology,2022,18(36):114-117.
[7] 齐海明,李俊林,罗广利. 基于三维激光扫描技术的地下矿山三维模型重建方法研究[J]. 经纬天地,2023(1):79-81. DOI: 10.3969/j.issn.1673-7563.2023.01.023 QI Haiming,LI Junlin,LUO Guangli. Research on 3D model reconstruction method of underground mine based on 3D laser scanning technology[J]. Survey World,2023(1):79-81. DOI: 10.3969/j.issn.1673-7563.2023.01.023
[8] 苗丙,葛世荣,郭一楠,等. 煤矿数字孪生智采工作面系统构建[J]. 矿业科学学报,2022,7(2):143-153. MIAO Bing,GE Shirong,GUO Yinan,et al. Construction of digital twin system for intelligent mining in coal mines[J]. Journal of Mining Science and Technology,2022,7(2):143-153.
[9] 郭虎升. 目标检测综述:从传统方法到深度学习[J]. 新兴科学和技术趋势,2024(2):128-145. DOI: 10.12405/j.issn.2097-1486.2024.02.002 GUO Husheng. Object detection:from traditional methods to deep learning[J]. Emerging Science and Technology,2024(2):128-145. DOI: 10.12405/j.issn.2097-1486.2024.02.002
[10] 高涵,赵培培,于正,等. 基于特征增强与Transformer的煤矿输送带异物检测[J]. 煤炭科学技术,2024,52(7):199-208. GAO Han,ZHAO Peipei,YU Zheng,et al. Coal mine conveyor belt foreign object detection based on feature enhancement and Transformer[J]. Coal Science and Technology,2024,52(7):199-208.
[11] 王宇,于春华,陈晓青,等. 基于多模态特征融合的井下人员不安全行为识别[J]. 工矿自动化,2023,49(11):138-144. WANG Yu,YU Chunhua,CHEN Xiaoqing,et al. Recognition of unsafe behaviors of underground personnel based on multi modal feature fusion[J]. Journal of Mine Automation,2023,49(11):138-144.
[12] 陈伟,江志成,田子建,等. 基于YOLOv8的煤矿井下人员不安全动作检测算法 [J/OL]. 煤炭科学技术:1-19[2024-08-18]. http://kns.cnki.net/kcms/detail/11.2402.td.20240322.1343.003.html. CHEN Wei,JIANG Zhicheng,TIAN Zijian,et al. YOLOv8-based algorithm for detecting unsafe movements of underground personnel in coal mines[J/OL]. Coal Science and Technology:1-19[2024-08-18]. http://kns.cnki.net/kcms/detail/11.2402.td.20240322.1343.003.html.
[13] 杨伟康,吕文生,杨鹏,等. 基于倒置残差的井下无人车目标检测研究[J]. 矿业研究与开发,2024,44(4):222-227. YANG Weikang,LYU Wensheng,YANG Peng,et al. Research on target detection of underground unmanned vehicle based on inverted residual[J]. Mining Research and Development,2024,44(4):222-227.
[14] 周浩,叶夏磊,包志舟,等. 数字孪生机器人虚实双向控制方法研究[J]. 机电工程技术,2024,53(12):135-140. ZHOU Hao,YE Xialei,BAO Zhizhou,et al. Research on virtual real bidirectional control method for digital twin robots[J]. Mechanical & Electrical Engineering Technology,2024,53(12):135-140.
[15] 牛进鑫,孙文磊,刘国良,等. 基于数字孪生驱动的机器人运动仿真与状态虚实同步映射研究[J]. 现代制造工程,2024(5):48-55. NIU Jinxin,SUN Wenlei,LIU Guoliang,et al. Research on robot motion simulation and virtual-real synchronized mapping driven by digital twins[J]. Modern Manufacturing Engineering,2024(5):48-55.
[16] SA Guodong,SUN Jiacheng,HOU Mingjie,et al. A digital twin synchronous evolution method of CNC machine tools associated with dynamic and static errors[J]. The International Journal of Advanced Manufacturing Technology,2024,134(5):2753-2763.
[17] WANG Zhendong,ZHENG Huangjie,HE Pengcheng,et al. Diffusion-GAN:training GANs with diffusion[EB/OL]. [2024-08-16]. https://arxiv.org/abs/2206.02262.
[18] HO J,JAIN A,ABBEEL P. Denoising diffusion probabilistic models[J]. Advances in Neural Information Processing Systems,2020,33:6840-6851.
[19] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al. Generative adversarial nets[C]. 28th Annual Conference on Neural Information Processing Systems,Montreal,2014:2672-2680.
[20] 王鑫,吴士良. 智能综采工作面系统设计及关键技术研究[J]. 中国煤炭,2024,50(9):73-79. WANG Xin,WU Shiliang. Research on system design and key technology of intelligent fully mechanized mining face[J]. China Coal,2024,50(9):73-79.
[21] 张捷. UG软件在汽车车身CAD建模中的使用[J]. 内燃机与配件,2021(19):206-207. DOI: 10.3969/j.issn.1674-957X.2021.19.097 ZHANG Jie. Application of UG software in auto body CAD modeling[J]. Internal Combustion Engine & Parts,2021(19):206-207. DOI: 10.3969/j.issn.1674-957X.2021.19.097
[22] 王进峰,问丛川,花广如,等. 基于Unity3D的数控机床数字孪生系统设计与实现[J]. 中国工程机械学报,2023,21(5):443-448. WANG Jinfeng,WEN Congchuan,HUA Guangru,et al. Design and implementation of digital twin system for CNC machine tools based on Unity3D[J]. Chinese Journal of Construction Machinery,2023,21(5):443-448.
[23] 陈述,鲁世立,王建平,等. 融合Unity3D的缆索起重机安全运行数字孪生模型构建方法[J]. 中国安全生产科学技术,2024,20(1):154-159. CHEN Shu,LU Shili,WANG Jianping,et al. Construction method of digital twin model for safe operation of cable crane by integrating Unity3D[J]. Journal of Safety Science and Technology,2024,20(1):154-159.
计量
- 文章访问数: 56
- HTML全文浏览量: 12
- PDF下载量: 16
