基于FBEC−YOLOv5s的采掘工作面多目标检测研究

张辉; 苏国用; 赵东洋

doi:10.13272/j.issn.1671-251x.2023060063

基于FBEC−YOLOv5s的采掘工作面多目标检测研究

doi: 10.13272/j.issn.1671-251x.2023060063

张辉^{1, 2,},
苏国用^{1, 2, 3, ,},
赵东洋^{1, 2}

1.
安徽理工大学深部煤矿采动响应与灾害防控国家重点实验室，安徽淮南　232001
2.
安徽理工大学机械工程学院，安徽淮南　232001
3.
矿山智能技术与装备省部共建协同创新中心，安徽淮南　232001

基金项目: 安徽省高等学校科学研究项目(2022AH050834)；安徽理工大学引进人才科研启动基金项目（2022yjrc61）；安徽理工大学矿山智能技术与装备省部共建协同创新中心开放基金项目（CICJMITE202206）；Open Fund of State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines（SKLMRDPC22KF24）。

详细信息

作者简介:
张辉（2000—），男，安徽亳州人，硕士研究生，研究方向为矿山机械，E-mail：zh18326741542@126.com

通讯作者:
苏国用（1990—），男，安徽淮南人，讲师，研究方向为矿山机械，E-mail：guoyongs005@sina.cn。

中图分类号: TD67
计量
- 文章访问数: 239
- HTML全文浏览量: 57
- PDF下载量: 77
- 被引次数: 0
出版历程
- 收稿日期: 2023-06-20
- 修回日期: 2023-11-14
- 网络出版日期: 2023-11-23

Research on multi object detection in mining face based on FBEC-YOLOv5s

ZHANG Hui^{1, 2
,},
SU Guoyong^{1, 2, 3
, ,},
ZHAO Dongyang^{1, 2}

1.
State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mine, Anhui University of Science and Technology, Huainan 232001, China
2.
School of Mechanical Engineering, Anhui University of Science and Technology, Huainan 232001, China
3.
Collaborative Innovation Center for Mining Intelligent Technology and Equipment, Huainan 232001, China

摘要

摘要: 针对采掘工作面目标尺度跨度大、多目标间相互遮挡严重及恶劣环境导致的检测精度降低等问题，提出了一种基于FBEC−YOLOv5s的采掘工作面多目标检测算法。首先，在主干网络引入FasterNet网络，以凭借其残差连接与批标准化模块，增强模型的特征提取和语义信息捕捉能力；其次，在YOLOv5s模型颈部融合BiFPN网络，以通过其双向跨尺度连接和快速归一化融合操作，实现多尺度特征的快速捕捉与融合；最后，采用ECIoU损失函数代替CIoU损失函数，以提升检测框定位精度和模型收敛速度。实验结果表明：① 在满足煤矿井下实时检测要求的同时，FBEC−YOLOv5s模型的准确率较YOLOv5s模型的准确率提升了3.6%。② 与YOLOv5s模型相比，FBEC−YOLOv5s模型的平均检测精度均值上升了2.8%，平均检测精度均值为92.4%，能够满足实时检测要求。③ FBEC−YOLOv5s模型的综合检测性能好，能够在恶劣环境、多目标间相互遮挡严重及目标尺度跨度大导致检测精度降低的情况下表现出良好的实时检测能力且具有较好的鲁棒性。
- 采掘工作面 /
- 多目标检测 /
- FasterNet网络 /
- 双向特征金字塔网络 /
- YOLOv5s /
- BiFPN /
- ECIoU损失函数
Abstract: A multi object detection algorithm based on FBEC-YOLOv5s is proposed to address the issues of reduced detection precision caused by large object scale spans, severe obstruction between multiple objects, and harsh environments in mining faces. Firstly, the FasterNet network is introduced into the backbone network to enhance the model's feature extraction and semantic information capture capabilities through its residual connection and batch standardization module. Secondly, the BiFPN network is fused in the neck of the YOLOv5s model to achieve rapid capture and fusion of multi-scale features through its bidirectional cross scale connection and fast normalization fusion operation. Finally, the ECIoU loss function is used instead of the CIoU loss function to improve the positioning precision of the detection frame and the convergence speed of the model. The experimental results show the following points. ① While meeting the real-time detection requirements of coal mines, the precision of the FBEC-YOLOv5s model has increased by 3.6% compared to YOLOv5s model. ② Compared with the YOLOv5s model, the average detection precision of the FBEC-YOLOv5s model has increased by 2.8%, with an average detection precision of 92.4%, which can meet real-time detection requirements. ③ The FBEC-YOLOv5s model has good comprehensive detection performance, demonstrating good real-time detection capability and robustness in condition that detection accuracy is reduced caused by harsh environments, severe mutual obstruction between multiple objects, and large object scale spans.
- mining face /
- multi object detection /
- FasterNet network /
- bidirectional feature pyramid network /
- YOLOv5s /
- BiFPN /
- ECIoU loss function

HTML全文

图 1 FBEC−YOLOv5s的网络结构

Figure 1. Network structure of FBEC-YOLOv5s

下载: 全尺寸图片幻灯片

图 2 FasterNet网络架构

Figure 2. Architecture of FasterNet

下载: 全尺寸图片幻灯片

图 3 不同特征金字塔网络结构对比

Figure 3. Comparison of different features pyramid network structures

下载: 全尺寸图片幻灯片

图 4 数据增强部分图像

Figure 4. Data enhanced partial images

下载: 全尺寸图片幻灯片

图 5 图像标注部分图像

Figure 5. Image annotated partial images

下载: 全尺寸图片幻灯片

图 6 不同算法部分检测结果

Figure 6. Partial detection results of different algorithms

下载: 全尺寸图片幻灯片

表 1 网络训练环境

Table 1. Network training environment

环境	配置参数
CPU	15 vCPU Intel（R） Xeon（R） Platinum 8358P CPU
GPU	RTX 3090（24 GB）
加速环境	Python3.8，Cuda11.3
语言环境	Python3.8

下载: 导出CSV

表 2 消融实验结果

Table 2. Results of ablation experiments

模型	ECIoU	BiFPN	FasterNet	准确率/%	平均检测精度均值/%	参数量/MiB	FPS/（帧·s⁻¹）
YOLOv5s	×	×	×	93.8	89.6	7.03	138.9
优化模型1	√	×	×	94.5	90.4	7.03	139.2
优化模型2	×	√	×	95.2	91.2	8.09	133.3
优化模型3	×	×	√	94.7	90.6	8.06	135.1
优化模型4	√	√	×	95.1	91.5	8.09	133.3
优化模型5	√	×	√	94.7	91.3	8.06	128.2
优化模型6	×	√	√	96.9	92.1	8.15	125.4
优化模型7	√	√	√	97.4	92.4	8.15	128.8

下载: 导出CSV

表 3 对比实验结果

Table 3. Comparison of experimental results

模型	平均检测精度均值/%	参数量/MiB	FPS/（帧·s⁻¹）
YOLOv5s	89.6	7.03	138.9
YOLOv3−tiny	84.2	8.68	156.3
YOLOv7	90.8	36.51	84.7
YOLOv7−tiny	80.9	6.02	180.5
FBEC−YOLOv5s	92.4	8.15	128.8

下载: 导出CSV

参考文献(29)

[1]	王国法,张建中,薛国华,等. 煤矿回采工作面智能地质保障技术进展与思考[J]. 煤田地质与勘探,2023,51(2):12-26. doi: 10.12363/issn.1001-1986.23.02.0062 WANG Guofa,ZHANG Jianzhong,XUE Guohua,et al. Progress and reflection of intelligent geological guarantee technology in coal mining face[J]. Coal Geology & Exploration,2023,51(2):12-26. doi: 10.12363/issn.1001-1986.23.02.0062
[2]	谢和平,任世华,谢亚辰,等. 碳中和目标下煤炭行业发展机遇[J]. 煤炭学报,2021,46(7):2197-2211. XIE Heping,REN Shihua,XIE Yachen,et al. Development opportunities of the coal industry towards the goal of carbon neutrality[J]. Journal of China Coal Society,2021,46(7):2197-2211.
[3]	魏文艳. 综采工作面智能化开采技术发展现状及展望[J]. 煤炭科学技术,2022,50(增刊2):244-253. WEI Wenyan. Development status and prospect of intelligent mining technology of longwall mining[J]. Coal Science and Technology,2022,50(S2):244-253.
[4]	王国法,杜毅博,徐亚军,等. 中国煤炭开采技术及装备50年发展与创新实践——纪念《煤炭科学技术》创刊50周年[J]. 煤炭科学技术,2023,51(1):1-18. WANG Guofa,DU Yibo,XU Yajun,et al. Development and innovation practice of China coal mining technology and equipment for 50 years:commemorate the 50th anniversary of the publication of Coal Science and Technology[J]. Coal Science and Technology,2023,51(1):1-18.
[5]	ZHANG Kexue,KANG Lei,CHEN Xuexi,et al. A review of intelligent unmanned mining current situation and development trend[J]. Energies,2022,15(2):513. doi: 10.3390/en15020513
[6]	李伟. 深部煤炭资源智能化开采技术现状与发展方向[J]. 煤炭科学技术,2021,49(1):139-145. LI Wei. Current status and development direction of intelligent mining technology for deep coal resources[J]. Coal Science and Technology,2021,49(1):139-1455.
[7]	程德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349-365. CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349-365.
[8]	李章维,胡安顺,王晓飞. 基于视觉的目标检测方法综述[J]. 计算机工程与应用,2020,56(8):1-9. doi: 10.3778/j.issn.1002-8331.2001-0163 LI Zhangwei,HU Anshun,WANG Xiaofei. Survey of vision based object detection methods[J]. Computer Engineering and Applications,2020,56(8):1-9. doi: 10.3778/j.issn.1002-8331.2001-0163
[9]	李程,车文刚,高盛祥. 一种用于航拍图像的目标检测算法[J]. 山东大学学报(理学版),2023,58(9):59-70. LI Cheng,CHE Wengang,GAO Shengxiang. A object detection algorithm for aerial images[J]. Journal of Shandong University(Natural Science),2023,58(9):59-70.
[10]	GIRSHICK R. Fast R-CNN[C]. Proceedings of the IEEE International Conference on Computer Vision,Santiago,2015:1440-1448.
[11]	REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[12]	HE Kaiming,GKIOXARI G,DOLLAR P,et al. Mask r-cnn[C]. IEEE International Conference on Computer Vision,Venice,2017:2980-2988.
[13]	杨文斌. 基于Faster−RCNN算法的刮板输送机异物识别技术研究[J]. 煤矿机械,2022,43(11):54-56. YANG Wenbin. Research on foreign matter recognition technology of scraper conveyor based on Faster-RCNN algorithm[J]. Coal Mine Machinery,2022,43(11):54-56.
[14]	郭永存,童佳乐,王爽. 井下无人驾驶电机车行驶场景中多目标检测研究[J]. 工矿自动化,2022,48(6):56-63. GUO Yongcun,TONG Jiale,WANG Shuang. Research on multi-object detection in driving scene of underground unmanned electric locomotive[J]. Journal of Mine Automation,2022,48(6):56-633.
[15]	史凌凯,耿毅德,王宏伟,等. 基于改进Mask R−CNN的刮板输送机铁质异物多目标检测[J]. 工矿自动化,2022,48(10):55-61. SHI Lingkai,GENG Yide,WANG Hongwei,et al. Multi-object detection of iron foreign bodies in scraper conveyor based on improved Mask R-CNN[J]. Journal of Mine Automation,2022,48(10):55-61.
[16]	王彦雅. 基于Two−Stage的目标检测算法综述[J]. 河北省科学院学报,2022,39(2):14-22. WANG Yanya. Overview of target detection algorithms based on two stage[J]. Journal of the Hebei Academy of Sciences,2022,39(2):14-22.
[17]	唐聪,凌永顺,郑科栋,等. 基于深度学习的多视窗SSD目标检测方法[J]. 红外与激光工程,2018,47(1):302-310. TANG Cong,LING Yongshun,ZHENG Kedong,et al. Object detection method of multi-view SSD based on deep learning[J]. Infrared and Laser Engineering,2018,47(1):302-310.
[18]	LAW H,DENG Jia. CornerNet:detecting objects aspaired keypoints[J]. International Journal of Computer Vision,2020,128(2):642-656.
[19]	王琳毅,白静,李文静,等. YOLO系列目标检测算法研究进展[J]. 计算机工程与应用,2023,59(14):15-29. WANG Linyi,BAI Jing,LI Wenjing,et al. Research progress of YOLO series target detection algorithms[J]. Computer Engineering and Applications,2023,59(14):15-29.
[20]	王科平,连凯海,杨艺,等. 基于改进YOLOv4的综采工作面目标检测[J]. 工矿自动化,2023,49(2):70-76. WANG Keping,LIAN Kaihai,YANG Yi,et al. Target detection of the fully mechanized working face based on improved YOLOv4[J]. Journal of Mine Automation,2023,49(2):70-76.
[21]	杨艺,付泽峰,高有进,等. 基于深度神经网络的综采工作面视频目标检测[J]. 工矿自动化,2022,48(8):33-42. YANG Yi,FU Zefeng,GAO Youjin,et al. Video object detection of the fully mechanized working face based on deep neural network[J]. Journal of Mine Automation,2022,48(8):33-42.
[22]	郭永存,杨豚,王爽. 基于改进YOLOv4–Tiny的矿井电机车多目标实时检测[J]. 工程科学与技术,2023,55(5):232-241. GUO Yongcun,YANG Tun,WANG Shuang. Multi-object real-time detection of mine electric locomotive based on improved YOLOv4-Tiny[J]. Advanced Engineering Sciences,2023,55(5):232-241.
[23]	樊红卫,刘金鹏,曹现刚,等. 低照度尘雾下煤、异物及输送带早期损伤多尺度目标智能检测方法[J/OL]. 煤炭学报:1-12[2023-08-19]. https://doi.org/10.13225/j.cnki.jccs.2023.0707. FAN Hongwei,LIU Jinpeng,CAO Xiangang,et al. Multi-scale target intelligent detection method for coal,foreign object and early damage of conveyor belt surface under low illumination and dust fog[J/OL]. Journal of China Coal Society:1-12[2023-08-19]. https://doi.org/10.13225/j.cnki.jccs.2023.0707.
[24]	CHEN Jierun,KAO S,HE Hao,et al. Run,don't walk:chasing higher FLOPS for faster neural networks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:12021-12031.
[25]	TAN Mingxing,PANG Ruoming,LE Q V. Efficientdet:scalable and efficient object detection[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,2020:10781-10790.
[26]	LIU Shu,QI Lu,QIN Haifang,et al. Path aggregation network for instance segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition,Salt Lake City,2018:8759-8768.
[27]	ZHENG Zhaohui,WANG Ping,LIU Wei,et al. Distance-IoU loss:faster and better learning for bounding box regression[C]. AAAI Conference on Artificial Intelligence,New York,2020:12993-13000.
[28]	CHEN Xinlin,LIAN Qingwang,CHEN Xuanlai,et al. Surface crack detection method for coal rock based on improved YOLOv5[J]. Applied Sciences,2022,12(19):9695. doi: 10.3390/app12199695
[29]	YU Jimin,WU Tao,ZHANG Xin,et al. An efficient lightweight SAR ship target detection network with improved regression loss function and enhanced feature information expression[J]. Sensors,2022,22(9):3447. doi: 10.3390/s22093447