Detection of surface defects on conveyor belts based on adversarial repair networks
-
摘要: 针对输送带缺陷数据获取和标注困难、输送带工作场景中的不稳定因素和数据波动导致基于深度学习的输送带缺陷检测方法精度低的问题，提出了一种基于对抗修复网络的输送带表面缺陷检测模型。该模型主要由自编码器结构的生成器和马尔可夫判别器组成。在训练阶段，将模拟的输送带表面缺陷图像输入生成器，得到无模拟缺陷的重构图像，提升模型对未知缺陷的泛化能力；将原始无损输送带图像、重构图像和模拟的输送带表面缺陷图像输入马尔可夫判别器，通过残差块获得特征图，提高模型对于微小缺陷的检测能力。在检测阶段，将待测图像输入训练完的生成器得到重构图像，再通过训练完的马尔可夫判别器提取待测图像与重构图像的特征图，根据待测图像与重构图像特征图之间的均方误差和待测图像特征图最大值，计算异常分数并与设定的阈值进行比较，从而判断待测图像是否存在缺陷。实验结果表明，该模型的接收操作特征曲线下面积（ROC−AUC）达0.999，精确率−召回率曲线下面积（PR−AUC）达0.997，单张图像检测时间为13.51 ms，能准确定位不同类型缺陷位置。Abstract: In response to the challenges of acquiring and labeling defect data on conveyor belts, as well as the low accuracy of deep learning-based conveyor belt defect detection methods due to unstable factors and data fluctuations in working environments, this study proposed a surface defect detection model based on adversarial repair networks. The model primarily consisted of a generator with an autoencoder structure and a Markov discriminator. During the training phase, simulated surface defect images of the conveyor belt were input into the generator to obtain reconstructed images without simulated defects, enhancing the model's ability to generalize to unknown defects. The original undamaged conveyor belt images, reconstructed images, and simulated surface defect images were input into the Markov discriminator, and feature maps were obtained through a residual network, improving the model's detection capability for subtle defects. In the detection phase, the test image was input into the trained generator to obtain the reconstructed image, and the trained Markov discriminator was used to extract feature maps from both the test image and the reconstructed image. The anomaly score was calculated based on the mean squared error between the feature maps of the test image and the reconstructed image, as well as the maximum value of the feature map of the test image, and compared with a set threshold to determine whether the test image contained defects. Experimental results showed that the area under the receiver operating characteristic curve (ROC-AUC) of this model reached 0.999, the area under the precision-recall curve (PR-AUC) reached 0.997, and the detection time for a single image was 13.51 ms, which could accurately locate the positions of different types of defects.
-
-
![]()
图 1 基于对抗修复网络的输送带表面缺陷检测模型结构
Figure 1. Structure of conveyor belt surface defect detection model based on adversarial repair network
![]()
图 2 模拟的输送带表面缺陷图像生成流程
Figure 2. Simulated conveyor belt surface defect image generation process
![]()
图 6 各模型ROC曲线
Figure 6. Receiver operating characteristic（ROC） curves of various models
表 1 各模型性能对比
Table 1 Performance comparison of various models
模型 ROC−AUC PR−AUC 单张图像检测时间/ms SSIM−AE 0.914 0.904 28.11 OCR−GAN 0.815 0.637 23.43 GANomaly 0.944 0.898 18.01 F−AnoGAN 0.944 0.917 11.72 本文模型 0.999 0.997 13.51 
下载: 导出CSV
表 2 消融实验结果
Table 2 Ablation experiment results
模型 模拟的输
送带表面
缺陷图像自编码器
结构的
生成器马尔可夫
判别器生成器
复合损失
函数异常分
数计算ROC−AUC PR−AUC 1 √ √ √ √ √ 0.999 0.997 2 × √ √ √ √ 0.704 0.753 3 √ × √ √ √ 0.974 0.939 4 √ √ × √ √ 0.779 0.554 5 √ √ √ × √ 0.824 0.690 6 √ √ √ √ × 0.986 0.975
下载: 导出CSV
-
[1] 王海军,王洪磊. 带式输送机智能化关键技术现状与展望[J]. 煤炭科学技术,2022,50(12):225-239. WANG Haijun,WANG Honglei. Status and prospect of intelligent key technologies of belt conveyor[J]. Coal Science and Technology,2022,50(12):225-239.
[2] 曹虎奇. 煤矿带式输送机撕带断带研究分析[J]. 煤炭科学技术,2015,43(增刊2):130-134. CAO Huqi. Study and analysis on tear belt and break belt of belt conveyor in coal mine[J]. Coal Science and Technology,2015,43(S2):130-134.
[3] GUO Xiaoqiang,LIU Xinhua,ZHOU Hao,et al. Belt tear detection for coal mining conveyors[J]. Micromachines,2022,13(3):449. DOI: 10.3390/mi13030449
[4] 蹇华,向何,孙万权,等. 带式输送机胶带纵向撕裂问题分析与对策[J]. 中国设备工程,2022(增刊2):152-154. JIAN Hua,XIANG He,SUN Wanquan,et al. Analysis and countermeasures of longitudinal tearing of conveyor belt[J]. China Plant Engineering,2022(S2):152-154.
[5] 程月,尚学文,王福平,等. 皮带撕裂的视觉检测[J]. 机械工程与自动化,2018(3):132-134,137. DOI: 10.3969/j.issn.1672-6413.2018.03.054 CHENG Yue,SHANG Xuewen,WANG Fuping,et al. Visual inspection of belt tearing[J]. Mechanical Engineering & Automation,2018(3):132-134,137. DOI: 10.3969/j.issn.1672-6413.2018.03.054
[6] 王以娜. 基于视觉检测的皮带纵向撕裂检测关键技术研究[D]. 鞍山:辽宁科技大学,2020. WANG Yi'na. Research on key technology of belt longitudinal tear detection based on visual inspection[D]. Anshan:University of Science and Technology Liaoning,2020.
[7] 周宇杰,徐善永,黄友锐,等. 基于改进YOLOv4的输送带损伤检测方法[J]. 工矿自动化,2021,47(11):61-65. ZHOU Yujie,XU Shanyong,HUANG Yourui,et al. Conveyor belt damage detection method based on improved YOLOv4[J]. Industry and Mine Automation,2021,47(11):61-65.
[8] 张梦超,周满山,张媛,等. 基于深度学习的矿用输送带损伤检测方法[J]. 工矿自动化,2021,47(6):51-56. ZHANG Mengchao,ZHOU Manshan,ZHANG Yuan,et al. Damage detection method for mine conveyor belt based on deep learning[J]. Industry and Mine Automation,2021,47(6):51-56.
[9] GUO Xiaoqiang,LIU Xinhua,KRÓLCZYK G,et al. Damage detection for conveyor belt surface based on conditional cycle generative adversarial network[J]. Sensors,2022,22(9). DOI: 10.3390/s22093485.
[10] YANG Qi,LI Fang,TIAN Hong,et al. A new knowledge-distillation-based method for detecting conveyor belt defects[J]. Applied Sciences,2022,12(19). DOI: 10.3390/app121910051.
[11] 罗东亮,蔡雨萱,杨子豪,等. 工业缺陷检测深度学习方法综述[J]. 中国科学:信息科学,2022,52(6):1002-1039. DOI: 10.1360/SSI-2021-0336 LUO Dongliang,CAI Yuxuan,YANG Zihao,et al. Survey on industrial defect detection with deep learning[J]. Scientia Sinica (Informationis),2022,52(6):1002-1039. DOI: 10.1360/SSI-2021-0336
[12] TENG Yapeng,LI Haoyang,CAI Fuzhen,et al. Unsupervised visual defect detection with score-based generative model[EB/OL]. [2023-11-25]. https://arxiv.org/abs/2211.16092v1.
[13] LYU Chengkan,ZHANG Zhengtao,SHEN Fei,et al. Unsupervised automatic defect inspection based on image matching and local one-class classification[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,Vancouver,2023:4435-4444.
[14] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al. Generative adversarial nets[J]. Advances in Neural Information Processing Systems,2014,27:2672-2680.
[15] ZAVRTANIK V,KRISTAN M,SKOČAJ D. DRÆM-a discriminatively trained reconstruction embedding for surface anomaly detection[C]. IEEE/CVF International Conference on Computer Vision,Montreal,2021:8310-8319.
[16] YANG Minghui,WU Peng,FENG Hui. MemSeg:a semi-supervised method for image surface defect detection using differences and commonalities[J]. Engineering Applications of Artificial Intelligence,2023,119. DOI: 10.1016/j.engappai.2023.105835.
[17] CIMPOI M,MAJI S,KOKKINOS I,et al. Describing textures in the wild[C]. IEEE Conference on Computer Vision and Pattern Recognition,Columbus,2014:3606-3613.
[18] WANG Zhou,BOVIK A C,SHEIKH H R,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing,2004,13(4):600-612. DOI: 10.1109/TIP.2003.819861
[19] JOHNSON J,ALAHI A,LI Feifei. Perceptual losses for real-time style transfer and super-resolution[C]. 14th European Conference on Computer Vision,Amsterdam,2016:694-711.
[20] BERGMANN P,LÖWE S,FAUSER M,et al. Improving unsupervised defect segmentation by applying structural similarity to autoencoders[EB/OL]. [2023-11-25]. https://arxiv.org/abs/1807.02011v3.
[21] LIANG Yufei,ZHANG Jiangning,ZHAO Shiwei,et al. Omni-frequency channel-selection representations for unsupervised anomaly detection[J]. IEEE Transactions on Image Processing,2023,32:4327-4340. DOI: 10.1109/TIP.2023.3293772
[22] AKCAY S,ATAPOUR-ABARGHOUEI A,RECKON T P. GANomaly:semi-supervised anomaly detection via adversarial training[C]. 14th Asian Conference on Computer Vision,Perth,2019:622-637.
[23] SCHLEGL T,SEEBÖCK P,WALDSTEIN S M,et al. F-AnoGAN:fast unsupervised anomaly detection with generative adversarial networks[J]. Medical Image Analysis,2019,54:30-44. DOI: 10.1016/j.media.2019.01.010
-
期刊类型引用(35)
1. 张文科，郭瑜，赵辉. 基于图像识别的煤矿带式输送机自适应调速系统设计. 煤炭工程. 2024(01): 220-224 . 
百度学术
2. 刘锋，白金牛. 基于视觉技术的胶带输送机煤量检测方法. 陕西煤炭. 2024(01): 52-57+64 . 百度学术
3. 刘文梅. 基于煤流量监控的带式输送机监测方案设计. 机械管理开发. 2024(02): 243-244+247 . 百度学术
4. 岑梁，尹小明，黄海棠，胡文博，张世荣. 带式输送系统节能优化调度研究. 制造业自动化. 2024(05): 114-119 . 百度学术
5. 尚伟栋，杨大山，张坤. 基于深度神经网络的带式输送机煤量检测方法. 工矿自动化. 2024(S1): 139-145 . 本站查看
6. 张克亮. 基于MT-CNN的矿井带式输送机输煤量检测技术. 中国矿业. 2024(06): 137-142 . 百度学术
7. 王涛. 基于改进MobileNet的带式输送机煤量检测研究. 能源与环保. 2024(07): 198-202 . 百度学术
8. 尹瑞，张冬雪，倪强. 基于数组的刮板输送机运载模型及煤量计算算法研究. 工矿自动化. 2024(08): 84-90 . 本站查看
9. 段树深. 基于ROI边缘图像直线特征的井下带式输送机跑偏故障检测方法. 中国矿业. 2024(10): 162-167 . 百度学术
10. 刘西尧，纪妙，陈恩博，余宝意，曾庆杰，李海生. 基于机器视觉的烟草输送机皮带跑偏识别方法研究. 设备管理与维修. 2024(19): 165-169 . 百度学术
11. 左明明，张习，杨子豪，孙其飞，张梦超，张媛，李虎. 基于深度学习的输送带跑偏状态智能监测方法. 工矿自动化. 2024(12): 166-172+182 . 本站查看
12. 尚伟栋. 矿用带式输送机过载传感检测中的模糊阈值判断. 煤矿机械. 2023(04): 197-200 . 百度学术
13. 秦亚敏，张雷，赵东哲，张金辉，贾英新，靳晔. 基于机器视觉的带式输送机跑偏监测预警方法. 煤矿机械. 2023(05): 165-167 . 百度学术
14. 曾飞，胡文祥，高彦鑫，宋杰杰. 基于激光扫描的输送带横向跑偏检测系统. 制造业自动化. 2023(05): 21-24 . 百度学术
15. 吕晨辉，李新，刘新龙，赵安新，张晨阳. 基于煤量检测与变频一体机的煤流自适应智能调速. 煤矿机械. 2023(08): 213-216 . 百度学术
16. 李阿红. 基于图像处理的矿用带式输送机系统设计及其能效分析. 能源与环保. 2023(07): 228-232+238 . 百度学术
17. 曾飞，陶玉衡，苏俊彬，李翔. 融合ResNet18和Deconvolution的输送带横向跑偏检测方法. 现代制造工程. 2023(08): 121-126 . 百度学术
18. 毛清华，郭文瑾，翟姣，王荣泉，尚新芒，李世坤，薛旭升. 煤矿带式输送机异常状态视频AI识别技术研究. 工矿自动化. 2023(09): 36-46 . 本站查看
19. 高京，董华丛. 基于目标检测的铁路货车承载鞍错位故障识别. 自动化技术与应用. 2023(10): 63-67 . 百度学术
20. 周坪，马国庆，周公博，马天兵，李远博. 智能化带式输送机健康监测技术研究综述. 仪器仪表学报. 2023(12): 1-21 . 百度学术
21. 杨志方，张立亚，郝博南，刘渊，赵青. 基于双流融合网络的输送带跑偏检测方法. 煤炭科学技术. 2023(S2): 259-267 . 百度学术
22. 王桂梅，李学晖，杨立洁，刘杰辉. 基于深度学习的永磁直驱带式输送机煤量检测方法研究. 煤炭技术. 2022(01): 188-190 . 百度学术
23. 邢建旭，岑梁，卢峰，黄益军，胡文博，邱泽晶，张世荣. 基于带式输送机的仓储转运系统优化调度研究. 机电工程. 2022(03): 402-410 . 百度学术
24. 屠鑫. 基于图像识别的中长跑运动员摆臂高度校准方法研究. 河北北方学院学报(自然科学版). 2022(07): 7-12+18 . 百度学术
25. 白光星，陈炜乐，孙勇，宋双林，李国芳，王彩萍，王伟峰，康付如. 煤矿带式输送机运输火灾风险智能监测与早期预警技术研究进展. 煤矿安全. 2022(09): 47-54 . 百度学术
26. 王海军，王洪磊. 带式输送机智能化关键技术现状与展望. 煤炭科学技术. 2022(12): 225-239 . 百度学术
27. 刘孝军，王飞. 基于AI的煤矿视频智能分析技术. 煤炭科学技术. 2022(S2): 260-264 . 百度学术
28. 姜阔胜，李良和，王开松，胡坤. 煤矿输送带修复点定位及监测系统设计. 煤矿机械. 2021(02): 1-4 . 百度学术
29. 王平. 基于数字图像处理的输送带跑偏状态实时监测技术. 煤矿机械. 2021(02): 168-170 . 百度学术
30. 云艳. 矿用带式输送机跑偏机理及纠偏系统设计. 能源与环保. 2021(12): 264-268 . 百度学术
31. 李林. 矿用皮带输送机纠偏装置的分析研究. 机械管理开发. 2020(08): 74-75+78 . 百度学术
32. 高云飞. 带式输送机托辊结构受力分析. 中国石油和化工标准与质量. 2020(15): 157-158 . 百度学术
33. 张凯，田原，贾曲. 机器视觉在煤机装备中的应用现状与趋势. 煤矿机械. 2020(12): 123-125 . 百度学术
34. 陈小霞，徐伟同，侯飞，高奎，刘兆雪，温玺杰，王兵，王延勇，王晓倩. 基于图像处理的煤炭量AI识别系统的设计. 选煤技术. 2020(06): 75-80 . 百度学术
35. 成新文. 带式输送机防跑偏技术分析研究. 内蒙古煤炭经济. 2020(15): 47-48 . 百度学术
其他类型引用(14)
计量
- 文章访问数: 60
- HTML全文浏览量: 25
- PDF下载量: 8
- 被引次数: 49
