Visual automatic detection method for hydraulic support loss state at the working face
-
摘要:
受采场地质条件变化、泵站压力波动及自动跟机系统误差等因素影响，液压支架在自动跟机移架过程中存在丢架情况，人工丢架监测严重影响工作面自动跟机效率。而基于传感器和感知信息的液压支架实时丢架状态监测方法的稳定性和可靠性较差。针对上述问题，提出一种液压支架丢架状态视觉自动检测方法。首先采用YOLOv8对实时获取的工作面监控视频图像进行工作面目标区域划分，通过充分学习工作面图像内部特征准确获取液压支架底座及推杆的轮廓信息与位置信息，分析不同液压支架底座及推杆的位置信息，确定监控视频图像中的支架号；然后提取相邻液压支架最小底座区域局部图像，利用融合多尺度特征信息的ResNet50卷积网络对底座局部图像进行特征提取，获取图像多尺度融合特征信息，再将特征信息映射到类别空间，获取不同液压支架状态的概率分布，根据概率判断液压支架正常移架或丢架状态，结合支架号信息确定处于丢架状态的液压支架。实验结果表明：基于监控视频的工作面目标区域平均分割精度为0.98，准确实现目标区域结构化提取；支架号自动识别准确率为98.78%，为液压支架丢架状态检测提供准确的支架号信息；工作面液压支架丢架状态视觉自动检测的平均准确率达99.17%，单帧图像处理时间为36 ms，满足采煤工作面AI视频监控系统检测丢架状态的实时性与可靠性需求。
Abstract:Due to changes in mining field geological conditions, fluctuations in pump station pressure, and errors in automated following system, hydraulic supports may experience a loss state during the automatic frame movement process. Manual monitoring of support loss significantly affects the efficiency of automated following at the working face. However, real-time loss state monitoring methods for hydraulic supports based on sensors and perception information often lack stability and reliability. To address these issues, a visual automatic detection method for hydraulic support loss state was proposed. First, YOLOv8 was used to segment key areas of the working face from real-time monitoring video images. By thoroughly analyzing the internal features of the working face images, the contours and location information of hydraulic support bases and push rods were accurately obtained. The location information of different hydraulic support bases and push rods was analyzed to determine the number of each hydraulic support in the monitoring video. Then, the local image of the smallest base region between adjacent hydraulic supports was extracted. A ResNet50 convolutional network, incorporating multi-scale feature fusion, was employed to extract features from the local image of the base, obtaining multi-scale fusion feature information of the image. This feature information was then mapped to a classification space to derive the probability distribution of different hydraulic support states. The support's normal movement or loss state was determined based on these probabilities, and the support number information was combined to identify the hydraulic support experiencing a loss state. Experimental results showed that the average segmentation accuracy of key areas at the working face using monitoring video was 0.98, achieving structured extraction of target regions. The automatic identification accuracy for support numbers was 98.78%, providing precise support number information for hydraulic support loss state detection. The average detection accuracy of the visual automatic detection method for hydraulic support loss state at the working face was 99.17%, and the single frame image processing time is 36 ms, meeting the real-time and reliability requirements for detecting support loss states in coal mining face AI video monitoring systems.
-
-
![]()
图 1 工作面液压支架丢架状态视觉自动检测系统框架
Figure 1. Framework of visual automatic detection method for hydraulic support loss state at the working face
![]()
图 4 基于改进ResNet50的液压支架丢架状态识别
Figure 4. Recognition of hydraulic support loss state based on improved ResNet50
![]()
图 5 语义分割网络训练过程中学习曲线
Figure 5. Learning curves during semantic segmentation network training
![]()
图 8 改进前后ResNet50液压支架丢架状态识别模型学习曲线变化
Figure 8. Changes in learning curve of hydraulic support loss state recognition model using ResNet50 before and after improvement
![]()
图 9 改进前后ResNet50丢架状态识别网络混淆矩阵
Figure 9. Confusion matrix of loss state recognition network using ResNet50 before and after improvement
![]()
图 10 工作面液压支架丢架状态视觉自动检测结果
Figure 10. Visual automatic detection results of hydraulic support loss state at the working face
表 1 工作面目标区域语义分割网络实验结果
Table 1 Experimental results of semantic segmentation network for key areas at the working face
目标区域 底座 推杆 线缆槽 煤壁 浮煤 刮板输送机 mAP50/% 97.9 98.3 99.5 98.7 98.0 95.4 
下载: 导出CSV
表 2 语义分割网络对比实验结果
Table 2 Comparison of experimental results for the semantic segmentation network
模型 Mask2former Mask R−CNN YOLOv8 平均精度/% 82.4 94.7 98.0
下载: 导出CSV
表 3 改进前后ResNet50丢架状态识别精度
Table 3 Accuracy of loss state recognition using ResNet50 before and after improvement
模型 准确率/% 召回率/% 单帧图像处理时间/ms 改进ResNet50
分类模型99.17 99.43 36 ResNet50分类模型 92.63 92.86 36
下载: 导出CSV
-
[1] 王国法,任怀伟,赵国瑞,等. 智能化煤矿数据模型及复杂巨系统耦合技术体系[J]. 煤炭学报,2022,47(1):61-74. WANG Guofa,REN Huaiwei,ZHAO Guorui,et al. Digital model and giant system coupling technology system of smart coal mine[J]. Journal of China Coal Society,2022,47(1):61-74.
[2] 任怀伟,巩师鑫,刘新华,等. 煤矿千米深井智能开采关键技术研究与应用[J]. 煤炭科学技术,2021,49(4):149-158. REN Huaiwei,GONG Shixin,LIU Xinhua,et al. Research and application on key techniques of intelligent mining for kilo-meter deep coal mine[J]. Coal Science and Technology,2021,49(4):149-158.
[3] 张超,张旭辉,毛清华,等. 煤矿智能掘进机器人数字孪生系统研究及应用[J]. 西安科技大学学报,2020,40(5):813-822. ZHANG Chao,ZHANG Xuhui,MAO Qinghua,et al. Research on key technology of intelligent tunneling robotic system in coal mine[J]. Journal of Xi'an University of Science and Technology,2020,40(5):813-822.
[4] 原铖. 液压支架乳化液泵站智能控制系统的应用研究[J]. 山东煤炭科技,2022,40(5):18-20. DOI: 10.3969/j.issn.1005-2801.2022.05.007 YUAN Cheng. Study and application of intelligent control system of hydraulic support emulsion pumping station[J]. Shandong Coal Science and Technology,2022,40(5):18-20. DOI: 10.3969/j.issn.1005-2801.2022.05.007
[5] REN Huaiwei,ZHANG Desheng,GONG Shixin,et al. Dynamic impact experiment and response characteristics analysis for 1∶2 reduced-scale model of hydraulic support[J]. International Journal of Mining Science and Technology,2021,31(3):347-356. DOI: 10.1016/j.ijmst.2021.03.004
[6] 张帅,任怀伟,韩安,等. 复杂条件工作面智能化开采关键技术及发展趋势[J]. 工矿自动化,2022,48(3):16-25. ZHANG Shuai,REN Huaiwei,HAN An,et al. Key technology and development trend of intelligent mining in complex condition working face[J]. Journal of Mine Automation,2022,48(3):16-25.
[7] 徐刚. 综采工作面配套技术研究[J]. 煤炭学报,2010,35(11):1921-1924. XU Gang. Study on match technology for fully mechanized longwall coal mining face[J]. Journal of China Coal Society,2010,35(11):1921-1924.
[8] 李俊士. 一种基于多种传感器的工作面找直方法[J]. 煤矿机电,2014(6):11-12,15. DOI: 10.3969/j.issn.1001-0874.2014.06.004 LI Junshi. A straightness control technology based on multi-sensor of fully mechanized coal face[J]. Colliery Mechanical & Electrical Technology,2014(6):11-12,15. DOI: 10.3969/j.issn.1001-0874.2014.06.004
[9] 谭超,刘婷,王忠宾,等. 一种根据液压支架有效推移行程调直刮板输送机的方法:CN108518237A[P]. 2018-09-11. TAN Chao,LIU Ting,WANG Zhongbin,et al. A method of straightening the scraper conveyor according to the hydraulic support:CN108518237A[P]. 2018-09-11.
[10] 王鹏星. 综采工作面液压支架工作状态监测技术的应用[J]. 机械管理开发,2021,36(11):163-164. WANG Pengxing. Working condition monitoring technology of hydraulic support at header working face[J]. Mechanical Management and Development,2021,36(11):163-164.
[11] 王宝录. 推移偏差补偿下的煤层倾斜综采面液压支架控制方法设计[J]. 液压气动与密封,2024,44(6):85-90. DOI: 10.3969/j.issn.1008-0813.2024.06.013 WANG Baolu. Design of hydraulic support control method for coal seam inclined fully mechanized mining face under displacement deviation compensation[J]. Hydraulics Pneumatics & Seals,2024,44(6):85-90. DOI: 10.3969/j.issn.1008-0813.2024.06.013
[12] 李伟,张行,朱真才,等. 综采工作面刮板输送机机身自动调直装置及方法:CN105000328A[P]. 2015-10-28. LI Wei,ZHANG Xing,ZHU Zhencai,et al. Automatic straightening device and method of fully mechanized mining face scraper conveyor:CN105000328A[P]. 2015-10-28.
[13] 杨学军,王然风,王怀法,等. 基于机器人的工作面液压支架姿态和直线度检测装置及方法:CN108663032A[P]. 2018-10-16. YANG Xuejun,WANG Ranfeng,WANG Huaifa,et al. The device and method of hydraulic support in working face:CN108663032A[P]. 2018-10-16.
[14] 刘旭. 综采工作面模拟装置与模拟试验研究[D]. 徐州:中国矿业大学,2019. LIU Xu. Research on simulation device and simulation test of longwall mining[D]. Xuzhou:China University of Mining and Technology,2019.
[15] 夏婷. 综采工作面刮板输送机直线度检测方法研究[D]. 徐州:中国矿业大学,2019. XIA Ting. Research on straightness detection method of scraper conveyor in fully mechanized mining face[D]. Xuzhou:China University of Mining and Technology,2019.
[16] HE Kaiming,GKIOXARI G,DOLLAR P,et al. Mask R-CNN[C]. IEEE International Conference on Computer Vision,Venice,2017. DOI: 10.1109/ICCV.2017.322.
[17] RONNEBERGER O,FISCHER P,BROX T. U-net:convolutional networks for biomedical image segmentation[M]. Cham:Springer International Publishing,2015:234-241.
[18] BADRINARAYANAN V,KENDALL A,CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495. DOI: 10.1109/TPAMI.2016.2644615
[19] CHEN L C,PAPANDREOU G,KOKKINOS I,et al. DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848. DOI: 10.1109/TPAMI.2017.2699184
[20] CHENG Bowen,MISRA I,SCHWING A G,et al. Masked-attention mask transformer for universal image segmentation[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,New Orleans,2022:1290-1299.
[21] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:unified,real-time object detection[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016:779-788.
[22] KRIZHEVSKY A,SUTSKEVER I,HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM,2017,60(6):84-90. DOI: 10.1145/3065386
[23] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2024-05-20]. https://arxiv.org/abs/1409.1556v6.
[24] SZEGEDY C,LIU Wei,JIA Yangqing,et al. Going deeper with convolutions[C]. IEEE Conference on Computer Vision and Pattern Recognition,Boston,2015. DOI: 10.1109/CVPR.2015.7298594.
[25] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016. DOI: 10.1109/CVPR.2016.90.
-
期刊类型引用(7)
1. 姚钰鹏，商楚浩，刘清. 基于工艺引擎的规划放煤控制系统. 工矿自动化. 2024(09): 41-46+107 . 
本站查看
2. 罗开成，高阳，杨艺，常亚军，袁瑞甫. 基于均值偏差奖赏函数的放煤口控制策略研究. 煤炭工程. 2022(09): 105-111 . 百度学术
3. 牛磊，张学亮，刘清，霍栋，郝跃. 综放开采瓦斯安全联动控制技术研究. 煤炭科学技术. 2022(10): 86-92 . 百度学术
4. 杨艺，李庆元，李化敏，李东印，杨延麟，费树岷. 基于批量式强化学习的群组放煤智能决策研究. 煤炭科学技术. 2022(10): 188-197 . 百度学术
5. 吴桐，尉瑞，刘清，魏文艳. 综放工作面智能放煤工艺研究及应用. 工矿自动化. 2021(03): 105-111 . 本站查看
6. 李庆元，杨艺，李化敏，费树岷. 基于Q-learning模型的智能化放顶煤控制策略. 工矿自动化. 2020(01): 72-79 . 本站查看
7. 张学亮，刘清，郎瑞峰，邵斌，吴少伟. 厚煤层智能放煤工艺及精准控制关键技术研究. 煤炭工程. 2020(09): 1-6 . 百度学术
其他类型引用(4)
计量
- 文章访问数: 395
- HTML全文浏览量: 14
- PDF下载量: 22
- 被引次数: 11
