Foreign object detection method for belt conveyor based on generative adversarial nets
-
摘要: 煤矿井下胶带运输图像具有照度低、细节不清晰、背景干扰等特点，现有的带式输送机异物检测模型存在精度低、灵活性差、计算量大、优化空间存在差异等问题。针对上述问题，提出了一种基于生成对抗网络（GAN）的带式输送机异物检测方法。对胶带运输过程视频文件进行预处理，分类得到正常图像、异常图像，制作实验数据集对改进GANomaly模型进行训练，再通过训练好的模型进行带式输送机异物检测。在训练阶段，将不含异物的带式输送机图像作为输入；在测试阶段，将含有异物的带式输送机图像作为输入，得到的重构图像与输入网络的原图像作差，即可得到异物的具体位置。GANomaly模型轻量化改进方法：在GANomaly基础网络模型中加入深度可分离卷积残差模块，采用深度可分离卷积代替原有主干网络中的卷积操作，大幅降低了模型计算量，同时减少了参数的冗余计算，能够明显提高异物检测速度；通过合并多个批量归一化（BN）层，加快模型的收敛迭代速度，提高模型的泛化收敛能力，有效避免梯度消失。实验结果表明，改进GANomaly模型相较于传统GANomaly模型，在运行速度上提升了6.27%，评价指标F1分数、AUC、召回率（Recall）和平均精度均值（mAP）分别提升了19.05%，22.22%，15.00%，17.14%。Abstract: The images of coal mine underground belt transportation have the features of low illumination, unclear details, and background interference. The existing foreign object detection models for belt conveyors have problems such as low precision, poor flexibility, large computational complexity, and differences in optimization space. In order to solve the above problems, a foreign object detection method for belt conveyors based on generative adversarial nets (GAN) is proposed. The method preprocesses the video files of the tape transportation process, classifies them into normal and abnormal images. The method creats an experimental dataset to train the improved GANomaly model, and then uses the trained model to detect foreign objects in the belt conveyor. During the training phase, the image of the belt conveyor without foreign objects is used as input. In the testing phase, the image of the belt conveyor containing foreign objects is used as input. The reconstructed image obtained is subtracted from the original image of the input network to obtain the specific position of the foreign object. The lightweight improvement method of GANomaly model adds a depthwise separable convolution residual module to the GANomaly basic network model, and uses depthwise separable convolution to replace the convolution operation in the original backbone network. It greatly reduces the computational complexity of the model and reduces the redundant calculation of parameters, which can significantly improve the speed of foreign object detection. By merging multiple batch normalization (BN) layers, the convergence iteration speed of the model is accelerated, the generalization convergence capability of the model is improved, and gradient vanishing is effectively avoided. The experimental results show that the improved GANomaly model has improved the running speed by 6.27% compared to the traditional GANomaly model. The evaluation indicators F1 score, AUC, Recall, and mean average precision (mAP) have increased by 19.05%, 22.22%, 15.00%, and 17.14%, respectively.
-
图 1 基于GAN的带式输送机异物检测模型结构
Figure 1. Structure of foreign object detection model for belt conveyor based on generative adversarial nets
图 3 深度可分离卷积生成冗余特征图过程
Figure 3. The process of generating redundant feature maps through deep separable convolution
表 1 合并前后性能比较
Table 1. Performance comparison before and after consolidation
合并前/后 CPU前向时间/ms GPU前向时间/ms 合并前 175.18 11.02 合并后 161.70 7.20 性能提升/% 7.69 34.66 
下载: 导出CSV
-
[1] 王国法,赵国瑞,任怀伟. 智慧煤矿与智能化开采关键核心技术分析[J]. 煤炭学报,2019,44(1):34-41. doi: 10.13225/j.cnki.jccs.2018.5034WANG Guofa,ZHAO Guorui,REN Huaiwei. Analysis on key technologies of intelligent coal mine and intelligent mining[J]. Journal of China Coal Society,2019,44(1):34-41. doi: 10.13225/j.cnki.jccs.2018.5034 [2] 鲍久圣,章全利,葛世荣,等. 煤矿井下无人化辅助运输系统关键基础研究及应用实践[J]. 煤炭学报,2023,48(2):1085-1098.BAO Jiusheng,ZHANG Quanli,GE Shirong,et al. Basic research and application practice of unmanned auxiliary transportation system in coal mine[J]. Journal of China Coal Society,2023,48(2):1085-1098. [3] 杨清翔,向秀华,孟斌,等. 一种煤矿带式输送机故障诊断方法[J]. 工矿自动化,2017,43(12):48-52. doi: 10.13272/j.issn.1671-251x.2017.12.010YANG Qingxiang,XIANG Xiuhua,MENG Bin,et al. A fault diagnosis method of coal mine belt conveyor[J]. Industry and Mine Automation,2017,43(12):48-52. doi: 10.13272/j.issn.1671-251x.2017.12.010 [4] 孙继平,钱晓红. 煤矿事故与应急救援技术装备[J]. 工矿自动化,2016,42(10):1-5. doi: 10.13272/j.issn.1671-251x.2016.10.001SUN Jiping,QIAN Xiaohong. Coal mine accident and emergency rescue technology and equipment[J]. Industry and Mine Automation,2016,42(10):1-5. doi: 10.13272/j.issn.1671-251x.2016.10.001 [5] 吕志强. 复杂环境下煤矿皮带运输异物图像识别研究[D]. 徐州:中国矿业大学,2020.LYU Zhiqiang. Research on image recognition of foreign bodies in the process of coal mine belt transportation in complex environment[D]. Xuzhou:China University of Mining and Technology,2020. [6] 程德强,徐进洋,寇旗旗,等. 融合残差信息轻量级网络的运煤皮带异物分类[J]. 煤炭学报,2022,47(3):1361-1369.CHENG Deqiang,XU Jinyang,KOU Qiqi,et al. Lightweight network based on residual information for foreign body classification on coal conveyor belt[J]. Journal of China Coal Society,2022,47(3):1361-1369. [7] 刘硕. 基于生成式对抗网络的异常检测研究[D]. 北京:北方工业大学,2022.LIU Shuo. Research on anomaly detection based on generative adversarial network[D]. Beijing:North China University of Technology,2022. [8] 任国强,韩洪勇,李成江,等. 基于Fast_YOLOv3算法的煤矿胶带运输异物检测[J]. 工矿自动化,2021,47(12):128-133.REN Guoqiang,HAN Hongyong,LI Chengjiang,et al. Foreign object detection in coal mine belt transportation based on Fast_YOLOv3 algorithm[J]. Industry and Mine Automation,2021,47(12):128-133. [9] 杜京义,陈瑞,郝乐,等. 煤矿带式输送机异物检测[J]. 工矿自动化,2021,47(8):77-83. doi: 10.13272/j.issn.1671-251x.2021040026DU Jingyi,CHEN Rui,HAO Le,et al. Coal mine belt conveyor foreign object detection[J]. Industry and Mine Automation,2021,47(8):77-83. doi: 10.13272/j.issn.1671-251x.2021040026 [10] 吴守鹏. 基于机器视觉的运煤皮带异物识别方法研究[D]. 徐州:中国矿业大学,2019.WU Shoupeng. Research on detection method of foreign object on coal conveyor belt based on computer vision[D]. Xuzhou:China University of Mining and Technology,2019. [11] 卢学明,于在川,许升起. 基于深度生成模型的煤矿运输皮带异物检测[J]. 计算机系统应用,2022,31(5):358-363.LU Xueming,YU Zaichuan,XU Shengqi. Foreign object detection of coal mine belt conveyor based on deep generative model[J]. Computer Systems and Applications,2022,31(5):358-363. [12] 郜振国. 煤矿井下运输异物检测关键技术研究[D]. 徐州:中国矿业大学,2018.GAO Zhenguo. Study on key technologies of foreign object detection in the coal mine transportation[D]. Xuzhou:China University of Mining and Technology,2018. [13] 张海明. 基于深度学习的皮带机上的异物识别方法研究[D]. 马鞍山:安徽工业大学,2020.ZHANG Haiming. Research on method of identifying foreign object on belt conveyor based on deep learning[D]. Ma'anshan:Anhui University of Technology,2020. [14] 孔二伟,张亚邦,李佳悦,等. 面向煤矿井下低光照图像的增强方法[J]. 工矿自动化,2023,49(4):62-69,85.KONG Erwei,ZHANG Yabang,LI Jiayue,et al. An enhancement method for low light images in coal mines[J]. Journal of Mine Automation,2023,49(4):62-69,85. [15] 吴开兴,张琳,李丽宏. 煤矿井下雾尘图像清晰化算法[J]. 工矿自动化,2018,44(3):70-75. doi: 10.13272/j.issn.1671-251x.2017100078WU Kaixing,ZHANG Lin,LI Lihong. Sharpening algorithm for underground images with fog and dust[J]. Industry and Mine Automation,2018,44(3):70-75. doi: 10.13272/j.issn.1671-251x.2017100078 [16] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al. Generative adversarial nets[EB/OL]. [2023-07-10]. https://arxiv.org/abs/1406.2661. [17] RADFORD A,METZ L,CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. [2023-07-10]. https://arxiv.org/abs/1511.06434v1. [18] AKCAY S,ATAPOUR-ABARGHOUEI A,BRECKON T P. GANomaly:semi-supervised anomaly detection via adversarial training[EB/OL]. [2023-07-10]. https://arxiv.org/abs/1805.06725. [19] 郑学召,童鑫,郭军,等. 煤矿智能监测与预警技术研究现状与发展趋势[J]. 工矿自动化,2020,46(6):35-40. doi: 10.13272/j.issn.1671-251x.17530ZHENG Xuezhao,TONG Xin,GUO Jun,et al. Research status and development trend of intelligent monitoring and early warning technology in coal mine[J]. Industry and Mine Automation,2020,46(6):35-40. doi: 10.13272/j.issn.1671-251x.17530 [20] 李敬兆,秦晓伟,汪磊. 基于边云协同框架的煤矿井下实时视频处理系统[J]. 工矿自动化,2021,47(12):1-7. doi: 10.13272/j.issn.1671-251x.2021070023LI Jingzhao,QIN Xiaowei,WANG Lei. Real-time video processing system in coal mine based on edge-cloud collaborative framework[J]. Industry and Mine Automation,2021,47(12):1-7. doi: 10.13272/j.issn.1671-251x.2021070023 [21] SIFRE L,MALLAT S. Rigid-motion scattering for texture classification[EB/OL]. [2023-07-10]. https://arxiv.org/abs/1403.1687. -
点击查看大图
图(10) / 表(1)
计量
- 文章访问数: 1234
- HTML全文浏览量: 102
- PDF下载量: 68
- 被引次数: 0
