基于边缘智能的煤矿外因火灾感知方法

赵端; 李涛; 董彦强; 王志强; 刘春

doi:10.13272/j.issn.1671-251x.2022080046

基于边缘智能的煤矿外因火灾感知方法

doi: 10.13272/j.issn.1671-251x.2022080046

赵端^{1, 2,},
李涛²,
董彦强³,
王志强³,
刘春¹

1.
中国矿业大学矿山互联网应用技术国家地方联合工程实验室, 江苏徐州　221116
2.
中国矿业大学信息与控制工程学院, 江苏徐州　221116
3.
国家电投集团内蒙古白音华煤电有限公司, 内蒙古锡林郭勒　026200

基金项目: 中央高校基本科研业务费专项资金项目（XJ2021005101）。

详细信息

作者简介:
赵端（1983—），男，河北承德人，教授，博士，硕士研究生导师，主要从事无线传感器网络、煤矿能量采集、机器学习方面的研究工作，E-mail：duan.zhao@cumt.edu.cn

中图分类号: TD752.3
计量
- 文章访问数: 306
- HTML全文浏览量: 53
- PDF下载量: 33
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-16
- 修回日期: 2022-12-10
- 网络出版日期: 2022-12-05

Coal mine external fire detection method based on edge intelligence

1.
National and Local Joint Engineering Laboratory of Mining Internet Application Technology, China University of Mining and Technology, Xuzhou 221116, China
2.
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China
3.
Inner Mongolia Baiyinhua Coal and Electricity Co., Ltd., State Power Investment Group, Xilin Gol 026200, China

摘要

摘要: 对煤矿外因火灾隐患进行检测，实现对初期火灾的可靠判识，对于提升煤矿火灾检测水平有重要意义，也是未来智能矿山建设的重要方向。为了提高煤矿外因火灾检测速度、精度和实时性，提出一种基于边缘智能的煤矿外因火灾感知方法。对YOLOv5s模型主干网络特征尺度进行改进，使模型能够充分学习浅层特征，改善小目标检测性能，同时在原有的特征金字塔网络（FPN）基础上添加自适应注意模块，提高模型检测精度。为解决井下光线条件差、粉尘多及摄像机拍摄角度引起的图像检测误差和漏检问题，采用多传感器辅助检测，通过动态加权算法对视频检测信息和多传感器检测信息进行加权融合判识，构建了YOLOv5s−as模型。将YOLOv5s−as模型移植到智能边缘处理器上，并进行轻量化处理，实现边缘智能设备部署。实验结果表明：与未加入传感器信息融合推理的YOLOv5s−a模型相比，YOLOv5s−as模型推理时间略长，但交并比为0.5时的平均精度均值(mAP@0.5)提高了7.24%；与移植前的YOLOv5s模型相比，移植到智能边缘处理器上并进行轻量化处理的YOLOv5s−as模型mAP@0.5提高15.04%；SSD 300，SSD 512及YOLOv5s模型无法识别小目标火源，YOLOv5s−a，YOLOv5s−as模型能够检测出小目标火源，适应性较好；使用边缘处理方式时，YOLOv5s−as模型的响应周期为238 ms，比集中式处理方法缩短了38.66%。
- 外因火灾检测 /
- YOLOv5s /
- 深度学习 /
- 自适应注意机制 /
- 多源信息融合 /
- 边缘计算
Abstract: The detection of external fire in coal mines and the reliable identification of initial fire are of great significance for improving the level of coal mine fire detection. It is also an important direction of intelligent mine construction in the future. In order to improve the speed, precision and real-time of coal mine external fire detection, a coal mine external fire detection method based on edge intelligence is proposed. The feature scale of the backbone network of the YOLOv5s model is improved. The model can fully learn the shallow features and improve the small target detection performance. At the same time, an adaptive attention module is added on the basis of the original feature pyramid network (FPN) to improve the detection precision of the model. There are problems of image detection error and missed detection caused by poor light conditions, more dust and camera shooting angle in the underground mine. In order to solve the above problems, the YOLOv5s-as model is constructed by using multi-sensor auxiliary detection and weighting fusion identification of video detection information and multi-sensor detection information through dynamic weighting algorithm. The YOLOv5s-as model is transplanted to the intelligent edge processor, and lightweight processing is carried out to realize the deployment of edge intelligent devices. The experimental results show that the reasoning time of the YOLOv5s-as model is slightly longer than that of the YOLOv5s-a model without sensor information fusion reasoning, but mean value of average precision when the intersection over union is 0.5 (mAP@0.5) is increased by 7.24%. Compared with the YOLOv5s model before transplantation, the mAP@0.5 of the YOLOv5s-as model transplanted to the intelligent edge processor and subjected to lightweight processing increased by 15.04%. For small target fire sources, SSD 300, SSD 512 and YOLOv5s models cannot identify them. The YOLOv5s-a and YOLOv5s-as models can detect small target fire sources with good adaptability. When using the edge processing method, the response period of YOLOv5s as model is 238 ms, which is 38.66% shorter than the centralized processing method.
- external fire detection /
- YOLOv5s /
- deep learning /
- adaptive attention mechanism /
- multi source information fusion /
- edge computing

HTML全文

图 1 基于边缘智能的煤矿外因火灾检测模型

Figure 1. Detection model of coal mine external fire based on edge intelligence

下载: 全尺寸图片幻灯片

图 2 YOLOv5s模型结构

Figure 2. Structure of YOLOv5s model

下载: 全尺寸图片幻灯片

图 3 改进后的4尺度检测网络

Figure 3. Improved 4-scale detection network

下载: 全尺寸图片幻灯片

图 4 自适应注意模块结构

Figure 4. Structure of adaptive attention module

下载: 全尺寸图片幻灯片

图 5 图像和传感信息加权融合判识流程

Figure 5. Weighted fusion identification process of image and sensor information

下载: 全尺寸图片幻灯片

图 6 实验巷道

Figure 6. Experimental roadway

下载: 全尺寸图片幻灯片

图 7 部分现场拍摄样本

Figure 7. Some samples taken on site

下载: 全尺寸图片幻灯片

图 8 火灾检测实验结果

Figure 8. Fire detection test results

下载: 全尺寸图片幻灯片

表 1 各权重对比分析结果

Table 1. Comparative analysis results of each weight

$ {\beta }_{1} $	$ {\beta }_{2} $	$ {\beta }_{3} $	$ {\beta }_{4} $	阈值	准确率
0.70	0.1	0.1	0.1	0.81	0.83
0.75	0.12	0.07	0.06	0.85	0.88
0.81	0.08	0.06	0.05	0.88	0.90
0.90	0.05	0.03	0.02	0.92	0.98
0	0.65	0.25	0.1	0.7	0.75
0	0.73	0.22	0.07	0.8	0.80
0	0.78	0.14	0.08	0.83	0.87
0	0.79	0.11	0.10	0.89	0.90

下载: 导出CSV

表 2 图像拍摄参数

Table 2. Image capture parameters

相机编号	拍摄角度	安装高度/cm	图像/张	光线	粉尘
1号	正水平	0	1 000	正常	少
2号	正水平	0	2 000	较暗	多
3号	正45°	210	1 000	正常	多
4号	正45°	210	2 000	较暗	少
5号	后水平	0	1 000	正常	少
6号	后水平	0	2 000	较暗	多
7号	后45°	210	1 000	正常	多
8号	后45°	210	2 000	较暗	多

下载: 导出CSV

表 3 各模型移植前检测结果对比

Table 3. Comparison of detection results of each algorithm before transplantation

模型	召回率	mAP@0.5	每帧推理时间/ms
SSD 300（VGG16）	0.764	0.732	26
SSD 521（VGG16）	0.779	0.751	62
YOLOv5s	0.825	0.811	24
YOLOv5s−a	0.915	0.907	18
YOLOv5s−as	0.967	0.941	20

下载: 导出CSV

表 4 各模型移植后检测结果对比

Table 4. Comparison of detection results of each algorithm after transplantation

模型	每帧推理时间/ms	mAP@0.5
SSD 300（VGG16）	19	0.710
SSD 521（VGG16）	53	0.742
YOLOv5s	18	0.803
YOLOv5s−a	11	0.870
YOLOv5s−as	12	0.933

下载: 导出CSV

表 5 边缘计算性能测试结果

Table 5. Edge computing performance test results

步骤	边缘处理	集中式处理
步骤1	捕获图像用时39 ms	捕获图像用时39 ms
步骤2	边缘计算用时157 ms	上传图像用时154 ms
步骤3	响应警报用时42 ms	算法计算用时49 ms
步骤4	—	反馈检测结果用时104 ms
步骤5	—	响应警报用时42 ms
响应周期/ms	238	388

下载: 导出CSV

参考文献(22)

[1]	孙继平,钱晓红. 2004—2015年全国煤矿事故分析[J]. 工矿自动化,2016,42(11):1-5. doi: 10.13272/j.issn.1671-251x.2016.11.001 SUN Jiping,QIAN Xiaohong. Analysis of coal mine accidents in China during 2004-2015[J]. Industry and Mine Automation,2016,42(11):1-5. doi: 10.13272/j.issn.1671-251x.2016.11.001
[2]	宁小亮. 2013—2018年全国煤矿事故规律分析及对策研究[J]. 工矿自动化,2020,46(7):34-41. doi: 10.13272/j.issn.1671-251x.17610 NING Xiaoliang. Law analysis and counter measures research of coal mine accidents in China from 2013 to 2018[J]. Industry and Mine Automation,2020,46(7):34-41. doi: 10.13272/j.issn.1671-251x.17610
[3]	孟远,谢东海,苏波,等. 2010年—2019年全国煤矿生产安全事故统计与现状分析[J]. 矿业工程研究,2020,35(4):27-33. doi: 10.13582/j.cnki.1674-5876.2020.04.005 MENG Yuan,XIE Donghai,SU Bo,et al. Statistics and analysis of coal mine production safety accidents in China from 2010 to 2019[J]. Mineral Engineering Research,2020,35(4):27-33. doi: 10.13582/j.cnki.1674-5876.2020.04.005
[4]	何勇军,易欣,王伟峰,等. 煤矿井下电气火灾智能监控与灭火技术综述[J]. 煤矿安全,2022,53(9):55-64. doi: 10.13347/j.cnki.mkaq.2022.09.008 HE Yongjun,YI Xin,WANG Weifeng,et al. Review of intelligent monitoring and extinguishing technology of electrical fire in coal mine[J]. Safety in Coal Mines,2022,53(9):55-64. doi: 10.13347/j.cnki.mkaq.2022.09.008
[5]	李国伟. 石丘煤矿外因火灾防治措施分析[J]. 能源与节能,2022(6):111-113. doi: 10.3969/j.issn.2095-0802.2022.06.035 LI Guowei. Prevention and control measures of exogenous fire in Shiqiu Coal Mine[J]. Energy and Energy Conservation,2022(6):111-113. doi: 10.3969/j.issn.2095-0802.2022.06.035
[6]	王慧,宋宇宁. D−S证据理论在火灾检测中的应用[J]. 中国安全科学学报,2016,26(5):19-23. WANG Hui,SONG Yuning. Application of D-S evidence theory in fire detection[J]. China Safety Science Journal,2016,26(5):19-23.
[7]	黄圆明,徐泽. 基于DSP的多传感器信息融合的厨房火灾检测系统[J]. 科学技术创新,2020(11):51-53. doi: 10.3969/j.issn.1673-1328.2020.11.028 HUANG Yuanming,XU Ze. Kitchen fire detection system based on DSP multi-sensor information fusion[J]. Scientific and Technological Innovation,2020(11):51-53. doi: 10.3969/j.issn.1673-1328.2020.11.028
[8]	段锁林,杨可,毛丹,等. 基于模糊证据理论算法在火灾检测中的应用[J]. 计算机工程与应用,2017,53(5):231-235. doi: 10.3778/j.issn.1002-8331.1507-0231 DUAN Suolin,YANG Ke,MAO Dan,et al. Fuzzy evidence theory-based algorithm in application of fire detection[J]. Computer Engineering and Applications,2017,53(5):231-235. doi: 10.3778/j.issn.1002-8331.1507-0231
[9]	兰琪,贾敏智. 基于图像型的矿井火灾探测方法研究[J]. 煤炭技术,2016,35(4):187-189. doi: 10.13301/j.cnki.ct.2016.04.076 LAN Qi,JIA Minzhi. Research on fire detection based on image in coal mine[J]. Coal Technology,2016,35(4):187-189. doi: 10.13301/j.cnki.ct.2016.04.076
[10]	孙继平,孙雁宇,范伟强. 基于可见光和红外图像的矿井外因火灾识别方法[J]. 工矿自动化,2019,45(5):1-5,21. doi: 10.13272/j.issn.1671-251x.17435 SUN Jiping,SUN Yanyu,FAN Weiqiang. Mine exogenous fire identification method based on visible light and infrared image[J]. Industry and Mine Automation,2019,45(5):1-5,21. doi: 10.13272/j.issn.1671-251x.17435
[11]	孙继平,崔佳伟. 矿井外因火灾感知方法[J]. 工矿自动化,2021,47(4):1-5,38. doi: 10.13272/j.issn.1671-251x.17760 SUN Jiping,CUI Jiawei. Mine external fire sensing method[J]. Industry and Mine Automation,2021,47(4):1-5,38. doi: 10.13272/j.issn.1671-251x.17760
[12]	何凯,冯旭,高圣楠,等. 基于多尺度特征融合与反复注意力机制的细粒度图像分类算法[J]. 天津大学学报(自然科学与工程技术版),2020,53(10):1077-1085. HE Kai,FENG Xu,GAO Shengnan,et al. Fine-grained image classification algorithm using multi-scale feature fusion and re-attention mechanism[J]. Journal of Tianjin University(Science and Technology),2020,53(10):1077-1085.
[13]	HUANG Chenchen, CHEN Siqi, XU Longtao. Object detection based on multi-source information fusion in different traffic scenes[C]. 12th International Conference on Advanced Computational Intelligence, Dali, 2020: 213-217.
[14]	傅天驹,郑嫦娥,田野,等. 复杂背景下基于深度卷积神经网络的森林火灾识别[J]. 计算机与现代化,2016(3):52-57. doi: 10.3969/j.issn.1006-2475.2016.03.011 FU Tianju,ZHENG Chang'e,TIAN Ye,et al. Forest fire recognition based on deep convolutional neural network under complex background[J]. Computer and Modernization,2016(3):52-57. doi: 10.3969/j.issn.1006-2475.2016.03.011
[15]	喻丽春,刘金清. 基于改进Mask R−CNN的火焰图像识别算法[J]. 计算机工程与应用,2020,56(21):194-198. doi: 10.3778/j.issn.1002-8331.2006-0194 YU Lichun,LIU Jinqing. Fire image recognition algorithm based on improved Mask R-CNN[J]. Computer Engineering and Applications,2020,56(21):194-198. doi: 10.3778/j.issn.1002-8331.2006-0194
[16]	侯易呈,王慧琴,王可. 改进的多尺度火焰检测方法[J]. 液晶与显示,2021,36(5):751-759. doi: 10.37188/CJLCD.2020-0221 HOU Yicheng,WANG Huiqin,WANG Ke. Improved multi-scale flame detection method[J]. Chinese Journal of Liquid Crystals and Displays,2021,36(5):751-759. doi: 10.37188/CJLCD.2020-0221
[17]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[EB/OL]. [2022-07-21]. https://arxiv.org/abs/1506.02640.
[18]	WANG C-Y, LIAO H-Y M, WU Y-H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, 2020: 390-391.
[19]	LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[EB/OL]. [2022-07-20]. https://arxiv.org/abs/1803.01534.
[20]	柳小军,鲍鸿. 基于ARM9多传感器数据融合火灾报警系统的实现[J]. 工业控制计算机,2009,22(3):52-53. doi: 10.3969/j.issn.1001-182X.2009.03.028 LIU Xiaojun,BAO Hong. Implementation of fire alarm system based on ARM9 multi-sensor data fusion[J]. Industrial Control Computer,2009,22(3):52-53. doi: 10.3969/j.issn.1001-182X.2009.03.028
[21]	HOWARD A G, ZHU Menglong, CHEN Bo, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. [2022-07-20]. https://arxiv.org/abs/1704.04861.
[22]	LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]. European Conference on Computer Vision, 2016: 21-37.