Multi-object detection of iron foreign bodies in scraper conveyor based on improved Mask R-CNN
-
摘要: 刮板输送机是煤矿井下的关键运输设备，铁质异物进入刮板输送机会引发磨损、断链等，甚至会造成停产、伤人等严重事故。现有刮板输送机异物识别方法存在对井下图像的适应性较差、无法区分异物类别与数量等问题。针对上述问题，提出了一种基于改进掩码区域卷积神经网络（Mask R−CNN）的刮板输送机铁质异物多目标检测方法。采用基于Laplace算子的图像增强算法对井下低照度、高粉尘环境下采集的图像进行预处理，对增强后的图像进行标注，制作数据集。采用Mask R−CNN 模型的ResNet−50特征提取器获取铁质异物图像特征；采用特征金字塔网络进行特征融合，保证同时拥有高层的语义特征（如类别、属性等）和低层的轮廓特征（如颜色、轮廓、纹理等），以提高小尺度铁质异物识别精度；针对Mask R−CNN模型生成的锚点与待检测的铁质异物尺寸不对应的问题，对Mask R−CNN模型进行改进，采用k−meansⅡ聚类算法代替原来的锚点生成方案，通过遍历数据集中标注框的长宽信息得到聚类中心点，实现刮板输送机铁质异物多目标检测。实验结果表明，改进Mask R−CNN模型对单张图像的平均检测时间为0.732 s，与Mask R−CNN，YOLOv5相比，分别缩短0.093，0.002 s；平均精度为91.7%，与Mask R−CNN，YOLOv5相比，分别提高11.4%，2.9%。
-
关键词:
- 刮板输送机 /
- 铁质异物 /
- 多目标检测 /
- 深度学习 /
- Mask R−CNN /
- k−meansⅡ聚类算法
Abstract: The scraper conveyor is the key transportation equipment in the coal mine. The iron foreign body entering the scraper conveyor will lead to wear and tear, chain breakage, and even cause serious accidents such as production stoppage and personal injury. The existing scraper conveyor foreign bodies identification method has the problems of poor adaptability to underground images and the incapability of distinguishing the types and quantities of foreign bodies. To solve the above problems, a multi-object detection method for iron foreign bodies in scraper conveyor based on improved mask region-convolutional neural network (Mask R-CNN) is proposed. The image enhancement algorithm based on the Laplace operator is used to preprocess the images collected under the environment of low illumination and high dust. The enhanced images are marked to make a data set. The ResNet-50 feature extractor of the Mask R-CNN model is used to obtain the image features of iron foreign bodies. The feature pyramid network is used for feature fusion to ensure both high-level semantic features (such as category, attribute, etc.) and low-level contour features (such as color, contour, texture, etc.), so as to improve the accuracy of small-scale iron foreign body identification. To solve the problem that the anchor point generated by the Mask R-CNN model does not correspond to the size of the iron foreign body to be detected, the Mask R-CNN model is improved. K-means Ⅱ clustering algorithm is used to replace the original anchor point generation scheme. The cluster center point is obtained by traversing the length and width information of the tag box in the data set, so as to achieve the multi-object detection of iron foreign bodies in the scraper conveyor. The experimental results show that the average detection time of the improved Mask R-CNN model is 0.732 s, which is shortened by 0.093 s and 0.002 s compared with Mask R-CNN and YOLOv5 respectively. The average precision is 91.7%, which is 11.4% and 2.9% higher than that of Mask R-CNN and YOLOv5 respectively. -
图 1 刮板输送机铁质异物多目标检测流程
Figure 1. Multi-object detection process of iron foreign bodies in scraper conveyor
图 2 Mask R−CNN模型结构
Figure 2. Structure of mask region-convolutional neural network model
图 4 刮板输送机铁质异物智能识别实验台
Figure 4. Test bed for intelligent identification of iron foreign bodies in scraper conveyor
图 8 图像增强前后模型损失值对比
Figure 8. Comparison of model loss values before and after image enhancement
图 9 模型改进前后训练结果对比
Figure 9. Comparison of training results before and after model improvement
图 10 刮板输送机5种常见铁质异物多目标检测效果
Figure 10. Multi-object detection effect of five common iron foreign bodies in scraper conveyor
表 1 实验环境配置
Table 1. Experimental environment configuration
实验环境 配置 操作系统 Windows 10 专业版 显卡 NVIDIA Quadro P620 处理器 Intel(R)Core(TM)i7−10875H CPU 学习框架 Tensorflow 
下载: 导出CSV
表 2 不同模型检测效果对比
Table 2. Comparison of detection effects of different models
模型 检出
张数未检出
张数单张图像平均
检测时间/s平均
精度/%Mask R−CNN 642 32 0.825 80.3 YOLOv5 658 16 0.734 88.8 改进Mask R−CNN 667 9 0.732 91.7
下载: 导出CSV
-
[1] 任国强,韩洪勇,李成江,等. 基于Fast_YOLOv3算法的煤矿胶带运输异物检测[J]. 工矿自动化,2021,47(12):128-133.REN Guoqiang,HAN Hongyong,LI Chengjiang,et al. Foreign object detection in coal mine belt transportation based on Fast_YOLOv3 algorithm[J]. Industry and Mine Automation,2021,47(12):128-133. [2] 杜京义,陈瑞,郝乐,等. 煤矿带式输送机异物检测[J]. 工矿自动化,2021,47(8):77-83.DU Jingyi,CHEN Rui,HAO Le,et al. Coal mine belt conveyor foreign object detection[J]. Industry and Mine Automation,2021,47(8):77-83. [3] 吴守鹏,丁恩杰,俞啸. 基于改进FPN的输送带异物识别方法[J]. 煤矿安全,2019,50(12):127-130. doi: 10.13347/j.cnki.mkaq.2019.12.029WU Shoupeng,DING Enjie,YU Xiao. Foreign body identification of belt based on improved FPN[J]. Safety in Coal Mines,2019,50(12):127-130. doi: 10.13347/j.cnki.mkaq.2019.12.029 [4] 王卫东,张康辉,吕子奇,等. 基于深度学习的煤中异物机器视觉检测[J]. 矿业科学学报,2021,6(1):115-123.WANG Weidong,ZHANG Kanghui,LYU Ziqi,et al. Machine vision detection of foreign objects in coal using deep learning[J]. Journal of Mining Science and Technology,2021,6(1):115-123. [5] 王燕,郭潇樯,刘新华. 带式输送机大块异物视觉检测系统设计[J]. 机械科学与技术,2021,40(12):1939-1943. doi: 10.13433/j.cnki.1003-8728.20200284WANG Yan,GUO Xiaoqiang,LIU Xinhua. Design of visual detection system for large foreign body in belt conveyor[J]. Mechanical Science and Technology for Aerospace Engineering,2021,40(12):1939-1943. doi: 10.13433/j.cnki.1003-8728.20200284 [6] HE K,GKIOXARI G,DOLLÁR P,et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(2):386-397. doi: 10.1109/TPAMI.2018.2844175 [7] 翁玉尚,肖金球,夏禹. 改进Mask R−CNN算法的带钢表面缺陷检测[J]. 计算机工程与应用,2021,57(19):235-242. doi: 10.3778/j.issn.1002-8331.2010-0446WENG Yushang,XIAO Jinqiu,XIA Yu. Strip surface defect detection based on improved Mask R-CNN algorithm[J]. Computer Engineering and Applications,2021,57(19):235-242. doi: 10.3778/j.issn.1002-8331.2010-0446 [8] 徐慧芳,黄冬梅,贺琪,等. 改进Mask R−CNN模型的海洋锋检测[J]. 中国图象图形学报,2021,26(12):2981-2990.XU Huifang,HUANG Dongmei,HE Qi,et al. Ocean front detection method based on improved Mask R-CNN[J]. Journal of Image and Graphics,2021,26(12):2981-2990. [9] 朱繁,王洪元,张继. 基于改进的Mask R−CNN的行人细粒度检测算法[J]. 计算机应用,2019,39(11):3210-3215.ZHU Fan,WANG Hongyuan,ZHANG Ji. Fine-grained pedestrian detection algorithm based on improved Mask R-CNN[J]. Journal of Computer Applications,2019,39(11):3210-3215. [10] 储珺,束雯,周子博,等. 结合语义和多层特征融合的行人检测[J]. 自动化学报,2022,48(1):282-291.CHU Jun,SHU Wen,ZHOU Zibo,et al. Combining semantics with multi-level feature fusion for pedestrian detection[J]. Acta Automatica Sinica,2022,48(1):282-291. [11] 张晓雪. 基于Mask R−CNN的自动驾驶目标检测分析[J]. 科学与信息化,2019(11):115-117,120.ZHANG Xiaoxue. Automatic driving target detection based on Mask R-CNN[J]. Science and Informatization,2019(11):115-117,120. [12] 杨俊闯,赵超. K−Means聚类算法研究综述[J]. 计算机工程与应用,2019,55(23):7-14,63.YANG Junchuang,ZHAO Chao. Survey on K-Means clustering algorithm[J]. Computer Engineering and Applications,2019,55(23):7-14,63. [13] 王希. 煤矿井下运输异物检测关键技术研究[J]. 智能建筑与工程机械,2021,3(9):119-121.WANG Xi. Research on key technology of detecting foreign body in coal mine underground transportation[J]. Intelligent Building and Construction Machinery,2021,3(9):119-121. [14] 程德强,徐进洋,寇旗旗,等. 融合残差信息轻量级网络的运煤皮带异物分类[J]. 煤炭学报,2022,47(3):1361-1369. doi: 10.13225/j.cnki.jccs.xr21.1736CHENG Deqiang,XU Jinyang,KOU Qiqi,et al. Lightweight network based on residual information for foreign body classification on coal conveyor belt[J]. Journal of China Coal Society,2022,47(3):1361-1369. doi: 10.13225/j.cnki.jccs.xr21.1736 -
点击查看大图
计量
- 文章访问数: 300
- HTML全文浏览量: 54
- PDF下载量: 54
- 被引次数: 0
