矿用无人驾驶车辆行人检测技术研究

周李兵; 于政乾; 卫健健; 蒋雪利; 叶柏松; 赵叶鑫; 杨斯亮

doi:10.13272/j.issn.1671-251x.2024050058

矿用无人驾驶车辆行人检测技术研究

doi: 10.13272/j.issn.1671-251x.2024050058

周李兵^{1, 2, 3,},
于政乾^{1, 2},
卫健健^{1, 2},
蒋雪利^{1, 2},
叶柏松^{1, 2},
赵叶鑫^{1, 2},
杨斯亮^{1, 2}

1.
天地（常州）自动化股份有限公司，江苏常州　213015
2.
中煤科工集团常州研究院有限公司，江苏常州　213015
3.
南京航空航天大学机电学院，江苏南京　210016

基金项目: 江苏省科技成果转化专项项目（BA2022040）；天地科技股份有限公司科技创新创业资金专项项目（2023-TD-ZD005-003）；天地(常州)自动化股份有限公司科研项目(2022TY1003)。

详细信息

作者简介:
周李兵（1984—），男，湖北黄梅人，高级工程师，研究方向为矿山机电系统智能化、智能检测与控制等，E-mail：yjj20002022@163.com

中图分类号: TD67
计量
- 文章访问数: 93
- HTML全文浏览量: 15
- PDF下载量: 21
- 被引次数: 0
出版历程
- 收稿日期: 2024-05-18
- 修回日期: 2024-10-20
- 网络出版日期: 2024-09-29

Research on pedestrian detection technology for mining unmanned vehicles

ZHOU Libing^{1, 2, 3
,},
YU Zhengqian^{1, 2},
WEI Jianjian^{1, 2},
JIANG Xueli^{1, 2},
YE Baisong^{1, 2},
ZHAO Yexin^{1, 2},
YANG Siliang^{1, 2}

1.
Tiandi(Changzhou) Automation Co., Ltd., Changzhou 213015, China
2.
CCTEG Changzhou Research Institute, Changzhou 213015, China
3.
School of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

摘要

摘要: 矿用无人驾驶车辆的工作环境光照条件复杂，行人检测经常出现漏检现象，导致矿用无人驾驶车辆可靠性及安全性不足。针对巷道光照条件复杂的问题，提出了一种弱光图像增强算法：将弱光图像由RGB图像空间分解为HSV图像空间，通过Logarithm函数对亮度分量先进行光照，再通过双边滤波器去除噪声；采用形态学对饱和度分量进行闭操作，再通过高斯滤波器滤除噪声；将图像转换回RGB图像空间，通过半隐式ROF去噪模型对图像再次进行去噪，得到增强图像。针对行人检测存在漏检、精度低的问题，提出了一种基于改进YOLOv3的矿用无人驾驶车辆行人检测算法：采用密集连接块取代YOLOv3中的Residual连接，提高特征图利用率；采用Slim−neck结构优化YOLOv3的特征融合结构，使得特征图之间能够进行高效的信息融合，进一步提高对小目标行人的检测精度，并利用其内部特殊的轻量化卷积结构，提高检测速度；加入轻量级的卷积注意力模块（CBAM）增强算法对目标类别和位置的注意程度，提高行人检测精度。实验结果表明：① 提出的弱光图像增强算法能够有效提高图像可见度，图像中行人的纹理更加清晰，并具有更好的噪声抑制效果。② 基于增强后图像的矿用无人驾驶车辆行人检测算法的平均精度达95.68%，相较于基于改进YOLOv7和ByteTrack的煤矿关键岗位人员不安全行为识别算法、YOLOv5、YOLOv3算法分别提高了2.53%，6.42%，11.77%，且运行时间为29.31 ms。③ 基于增强后图像，YOLOv3和基于改进YOLOv7和ByteTrack的煤矿关键岗位人员不安全行为识别算法出现了漏检和误检的问题，而矿用无人驾驶车辆行人检测算法有效改善了该问题。
- 矿用无人驾驶车辆 /
- 井下行人检测 /
- YOLOv3 /
- 弱光图像增强 /
- 半隐式ROF去噪 /
- 密集连接模块 /
- Slim−neck /
- 卷积注意力模块
Abstract: The working environment of mining unmanned vehicles features complex lighting conditions, leading to frequent occurrences of missed detections in pedestrian detection, which undermines the reliability and safety of these vehicles. To address the challenges posed by intricate tunnel lighting conditions, a low-light image enhancement algorithm was proposed. This algorithm decomposed low-light images from the RGB color space into the HSV color space, applied a Logarithm function to enhance the V component, and employed a bilateral filter to reduce noise. Morphological operations were applied to the S component for closing, followed by Gaussian filtering to further eliminate noise. The enhanced image was then transformed back into the RGB color space and subjected to a semi-implicit ROF denoising model for additional noise reduction, resulting in an enhanced image. To tackle issues of missed detections and low accuracy in pedestrian detection, an improved YOLOv3-based pedestrian detection algorithm for mining unmanned vehicles was introduced. This approach replaced the Residual connections in YOLOv3 with densely connected modules to enhance feature map utilization. Additionally, a Slim-neck structure optimized the feature fusion architecture of YOLOv3, facilitating efficient information fusion between feature maps and further improving the detection accuracy for small-target pedestrians, while its unique lightweight convolutional structure enhanced detection speed. Finally, a lightweight convolutional block attention module (CBAM) was integrated to improve attention to object categories and locations, thereby enhancing pedestrian detection accuracy. Experimental results demonstrated that the proposed low-light image enhancement algorithm effectively improved image visibility, making pedestrian textures clearer and achieving better noise suppression. The average precision of the pedestrian detection algorithm for mining unmanned vehicles based on enhanced images reached 95.68%, representing improvements of 2.53%, 6.42%, and 11.77% over YOLOv5, YOLOv3, and a coal mine key position personnel unsafe behavior recognition method based on improved YOLOv7 and ByteTrack, respectively, with a runtime of 29.31 ms. YOLOv3 and a coal mine key position personnel unsafe behavior recognition method based on improved YOLOv7 and ByteTrack experienced missed detections and false positives based on enhanced images, while the proposed pedestrian detection algorithm effectively mitigated these issues.
- mining unmanned vehicles /
- underground pedestrian detection /
- YOLOv3 /
- low-light image enhancement /
- semi-implicit ROF denoising /
- densely connected modules /
- Slim-neck /
- convolutional attention module

HTML全文

图 1 弱光图像增强算法原理

Figure 1. Principle of low-light image enhancement method

下载: 全尺寸图片幻灯片

图 2 弱光图像HSV图像空间分量

Figure 2. HSV component of low-light images

下载: 全尺寸图片幻灯片

图 3 增强前的井下弱光图像

Figure 3. Underground low-light images before enhancement

下载: 全尺寸图片幻灯片

图 4 增强后的井下弱光图像

Figure 4. Enhanced underground low-light images

下载: 全尺寸图片幻灯片

图 5 基于改进YOLOv3的矿用无人驾驶车辆行人检测算法的网络结构

Figure 5. Network structure of pedestrian detection algorithm for mining unmanned vehicles based on improved YOLOv3

下载: 全尺寸图片幻灯片

图 6 密集连接块结构

Figure 6. Densely connected block structure

下载: 全尺寸图片幻灯片

图 7 改进YOLOv3网络中Slim−neck结构

Figure 7. Slim-neck structure in improved YOLOv3 network

下载: 全尺寸图片幻灯片

图 8 CBAM结构

Figure 8. Convolutional block attention module (CBAM) structure

下载: 全尺寸图片幻灯片

图 9 CAM结构

Figure 9. Channel attention model (CAM) structure

下载: 全尺寸图片幻灯片

图 10 SAM结构

Figure 10. Spatial attention model（SAM） structure

下载: 全尺寸图片幻灯片

图 11 矿用无轨胶轮车行人检测平台

Figure 11. Pedestrian detection platform for trackless rubber-wheeled vehicle in mining applications

下载: 全尺寸图片幻灯片

图 12 实验硬件计算平台

Figure 12. Experimental hardware computing platform

下载: 全尺寸图片幻灯片

图 13 不同算法在煤矿巷道弱光图像下的行人检测结果

Figure 13. Pedestrian detection results of different algorithms on low-light images of coal mine roadway

下载: 全尺寸图片幻灯片

图 14 不同算法在煤矿巷道增强图像下的行人检测结果

Figure 14. Pedestrian detection results of different algorithms on enhanced images of coal mine roadway

下载: 全尺寸图片幻灯片

表 1 弱光图像增强算法定量分析结果

Table 1. Quantitative results of low-light images enhancement algorithm

算法	PSNR	SSIM
RetinexNet	16.51	0.646 1
LLFlow	25.27	0.924 9
本文增强算法	26.48	0.996 7

下载: 导出CSV

表 2 各行人检测算法性能比较

Table 2. Comparison of the performance of various pedestrian detection algorithms

输入	算法	平均精度/%	运行时间/ms
弱光图像	SSD	35.22	57.66
	RetinaNet	37.73	46.45
	Faster R−CNN	44.22	84.41
	YOLOv3	72.23	33.56
	YOLOv5	79.37	32.98
	文献[26]	81.81	31.52
	本文算法	83.67	31.28
增强图像	SSD	36.51	52.88
	RetinaNet	51.63	46.31
	Faster R−CNN	45.72	83.27
	YOLOv3	83.91	31.46
	YOLOv5	89.26	30.47
	文献[26]	93.15	29.59
	本文算法	95.68	29.31

下载: 导出CSV

表 3 消融实验结果

Table 3. Results of ablation experiments

输入	算法	密集连接块	Slim−neck	CBAM	平均精度/%	运行时间/ms
弱光图像	YOLOv3	×	×	×	72.23	33.56
	A	√	×	×	77.71	34.97
	B	√	√	×	81.16	29.35
	本文算法	√	√	√	83.67	31.28
增强图像	YOLOv3	×	×	×	83.91	31.46
	A	√	×	×	88.69	32.72
	B	√	√	×	93.53	26.91
	本文算法	√	√	√	95.68	29.31

下载: 导出CSV

参考文献(26)

[1]	林燕霞,苏丹. 基于SLAM技术的矿区巷道巡检机器人路径规划优化[J]. 金属矿山,2024(4):209-214. LIN Yanxia,SU Dan. Path planning optimization of mine roadway inspection robot based on SLAM technique[J]. Metal Mine,2024(4):209-214.
[2]	韩江洪,卫星,陆阳,等. 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报,2020,45(6):2104-2115. HAN Jianghong,WEI Xing,LU Yang,et al. Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society,2020,45(6):2104-2115.
[3]	杨伟康,吕文生,杨鹏,等. 基于倒置残差的井下无人车目标检测研究[J]. 矿业研究与开发,2024,44(4):222-227. YANG Weikang,LYU Wensheng,YANG Peng,et al. Research on target detection of underground unmanned vehicle based on inverted residual[J]. Mining Research and Development,2024,44(4):222-227.
[4]	董观利,宋春林. 基于视频的矿井行人越界检测系统[J]. 工矿自动化,2017,43(2):29-34. DONG Guanli,SONG Chunlin. Underground pedestrian crossing detection system based on video[J]. Industry and Mine Automation,2017,43(2):29-34.
[5]	刘备战,赵洪辉,周李兵. 面向无人驾驶的井下行人检测方法[J]. 工矿自动化,2021,47(9):113-117. LIU Beizhan,ZHAO Honghui,ZHOU Libing. Unmanned driving-oriented underground mine pedestrian detection method[J]. Industry and Mine Automation,2021,47(9):113-117.
[6]	李伟山,卫晨,王琳. 改进的Faster RCNN煤矿井下行人检测算法[J]. 计算机工程与应用,2019,55(4):200-207. LI Weishan,WEI Chen,WANG Lin. Improved Faster RCNN approach for pedestrian detection in underground coal mine[J]. Computer Engineering and Applications,2019,55(4):200-207.
[7]	罗坤鑫. 矿用车辆多信息融合行人检测技术研究[D]. 西安:西安科技大学,2021. LUO Kunxin. Research on pedestrian detection technology of multi-information fusion for mining vehicles[D]. Xi'an:Xi'an University of Science and Technology,2021.
[8]	张应团,李涛,郑嘉祺. 基于DCNN的井下行人监测方法研究[J]. 计算机与数字工程,2019,47(8):2027-2032. doi: 10.3969/j.issn.1672-9722.2019.08.039 ZHANG Yingtuan,LI Tao,ZHENG Jiaqi. Research of underground pedestrian monitoring method based on DCNN[J]. Computer & Digital Engineering,2019,47(8):2027-2032. doi: 10.3969/j.issn.1672-9722.2019.08.039
[9]	谭显静. 图像去噪的ROF模型的理论分析与算法研究[D]. 重庆:重庆大学,2019. TAN Xianjing. Theoretical analysis and algorithm of ROF model for image denoising[D]. Chongqing:Chongqing University,2019.
[10]	刘寿鑫,龙伟,李炎炎,等. 基于HSV色彩空间的低照度图像增强[J]. 计算机工程与设计,2021,42(9):2552-2560. LIU Shouxin,LONG Wei,LI Yanyan,et al. Low-light image enhancement based on HSV color space[J]. Computer Engineering and Design,2021,42(9):2552-2560.
[11]	余化鹏,李舟,杨新瑞,等. 基于目标检测结果的轮廓及颜色识别研究[J]. 成都大学学报(自然科学版),2019,38(3):276-280. doi: 10.3969/j.issn.1004-5422.2019.03.011 YU Huapeng,LI Zhou,YANG Xinrui,et al. Research on object contours extraction and color recognition based on object detection result[J]. Journal of Chengdu University (Natural Science Edition),2019,38(3):276-280. doi: 10.3969/j.issn.1004-5422.2019.03.011
[12]	董红召,赵龙钢,赵晨馨,等. OBD支持下公交车到达时间的回归预测方法[J]. 高技术通讯,2021,31(4):425-434. doi: 10.3772/j.issn.1002-0470.2021.04.010 DONG Hongzhao,ZHAO Longgang,ZHAO Chenxin,et al. Regression prediction method of bus arrival time supported by OBD[J]. Chinese High Technology Letters,2021,31(4):425-434. doi: 10.3772/j.issn.1002-0470.2021.04.010
[13]	RUDIN L I,OSHER S,FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D:Nonlinear Phenomena,1992,60(1/2/3/4):259-268.
[14]	WEI Chen,WANG Wenjing,YANG Wenhan,et al. Deep Retinex decomposition for low-light enhancement[EB/OL]. (2022-08-21)[2024-04-22]. https://arxiv.org/abs/1808.04560v1.
[15]	WANG Yufei,WAN Renjie,YANG Wenhan,et al. Low-light image enhancement with normalizing flow[J]. Proceedings of the AAAI Conference on Artificial Intelligence,2022,36(3):2604-2612. doi: 10.1609/aaai.v36i3.20162
[16]	WANG Zhou,BOVIK A C,SHEIKH H R,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing,2004,13(4):600-612. doi: 10.1109/TIP.2003.819861
[17]	舒军,蒋明威,杨莉,等. DenseNet模型轻量化改进研究[J]. 华中师范大学学报(自然科学版),2020,54(2):187-193. SHU Jun,JIANG Mingwei,YANG Li,et al. Lightweight improvement research of DenseNet model[J]. Journal of Central China Normal University(Natural Sciences),2020,54(2):187-193.
[18]	LI Hulin,LI Jun,WEI Hanbing,et al. Slim-neck by GSConv:a lightweight-design for real-time detector architectures[J]. Journal of Real-Time Image Processing,2024,21(3). DOI: 10.1007/s11554-024-01436-6.
[19]	WOO S,PARK J,LEE J Y,et al. CBAM:convolutional block attention module[M]. Cham:Springer International Publishing,2018:3-19.
[20]	吴永俊,汪泓,杨晨. 基于改进DeepLabV3+的石漠化地区裸岩信息提取[J]. 航天返回与遥感,2024,45(1):123-135. WU Yongjun,WANG Hong,YANG Chen. Extraction of bare rock information in rocky desertification area based on improved DeepLabV3+[J]. Spacecraft Recovery & Remote Sensing,2024,45(1):123-135.
[21]	吕璐璐,陈树越,王利平,等. 深度特征融合与重构的微纤维识别算法[J]. 现代电子技术,2022,45(1):83-88. LYU Lulu,CHEN Shuyue,WANG Liping,et al. Microfiber recognition algorithm based on deep feature fusion and reconstruction[J]. Modern Electronics Technique,2022,45(1):83-88.
[22]	樊嵘,马小陆. 面向带钢表面小目标缺陷检测的改进YOLOv7算法[J]. 合肥工业大学学报(自然科学版),2024,47(3):303-308,316. FAN Rong,MA Xiaolu. Improved YOLOv7 algorithm for small target defect detection on strip steel surface[J]. Journal of Hefei University of Technology(Natural Science),2024,47(3):303-308,316.
[23]	韩崇,樊卫北,郭澳. 基于特征融合的毫米波雷达行为识别算法[J/OL]. 计算机科学:1-10[2024-03-12]. http://kns.cnki.net/kcms/detail/50.1075.TP.20240513.1347.011.html. HAN Chong,FAN Weibei,GUO Ao. Millimeter wave radar human activity recognition algorithm based on feature fusion [J/OL]. Computer Science:1-10[2024-03-12]. http://kns.cnki.net/kcms/detail/50.1075.TP.20240513.1347.011.html.
[24]	彭垚潘,张荣芬,刘宇红,等. 融入特征交互与注意力的轻量化混凝土裂缝分割算法[J/OL]. 光电子·激光:1-11[2024-03-12]. http://kns.cnki.net/kcms/detail/12.1182.O4.20240428.1852.016.html. PENG Yaopan,ZHANG Rongfen,LIU Yuhong,et al. Lightweight concrete crack segmentation algorithm integrating feature interaction and attention[J/OL]. Journal of Optoelectronics·Laser:1-11[2024-03-12]. http://kns.cnki.net/kcms/detail/12.1182.O4.20240428.1852.016.html.
[25]	韩康,战洪飞,余军合,等. 基于空洞卷积和增强型多尺度特征自适应融合的滚动轴承故障诊断[J]. 浙江大学学报(工学版),2024,58(6):1285-1295. HAN Kang,ZHAN Hongfei,YU Junhe,et al. Rolling bearing fault diagnosis based on dilated convolution and enhanced multi-scale feature adaptive fusion[J]. Journal of Zhejiang University(Engineering Science),2024,58(6):1285-1295.
[26]	韩康,李敬兆,陶荣颖. 基于改进YOLOv7和ByteTrack的煤矿关键岗位人员不安全行为识别[J]. 工矿自动化,2024,50(3):82-91. HAN Kang,LI Jingzhao,TAO Rongying. Recognition of unsafe behaviors of key position personnel in coal mines based on improved YOLOv7 and ByteTrack[J]. Journal of Mine Automation,2024,50(3):82-91.