煤矿运输系统多元异常图像检测研究

吕东翰; 胡而已; 黄一珀; 李汶璋

doi:10.13272/j.issn.1671-251x.2024050001

煤矿运输系统多元异常图像检测研究

1.
中科慧拓（北京）科技有限公司，北京　100190
2.
应急管理部信息研究院，北京　100029
3.
西安电子科技大学人工智能学院，陕西西安　710000
4.
北京纽科曼科技有限公司，北京　100029

基金项目: 国家自然科学基金资助项目(52274159，52374165)。

详细信息

作者简介:
吕东翰（1996—），男，江苏连云港人，硕士，研究方向为矿山智能化、辅助运输及数据融合等，E-mail：lvdonghan5@163.com

通讯作者:
胡而已(1982—)，男，安徽桐城人，研究员，博士，研究方向为矿山智能化、机器人及智能传感等，E-mail：horyhu@126.com。

中图分类号: TD528
计量
- 文章访问数: 95
- HTML全文浏览量: 26
- PDF下载量: 17
出版历程
- 收稿日期: 2024-05-02
- 修回日期: 2024-06-21
- 网络出版日期: 2024-07-09
- 刊出日期: 2024-06-29

Research on multivariate abnormal image detection in coal mine transportation system

1.
Beijing Waytous Technologies Co., Ltd., Beijing 100190, China
2.
Information Institute, Ministry of Emergency Management, Beijing 100029, China
3.
School of Artificial Intelligence, Xi'an University of Electronic Science and Technology, Xi'an 710000, China
4.
Beijing New Coal Mine Technologies Co., Ltd., Beijing 100029, China

摘要

摘要: 煤矿运输系统的异常险情种类繁多、场景多样，煤矿现场异常事故的发生具有偶然性，异常样本的获取其数量远小于正常样本，造成正负样本不平衡。针对上述问题，提出一种基于超球重构数据描述（HRDD）的煤矿运输系统多元异常图像检测方法。在全卷积数据描述（FCDD）基础上引入图像重构辅助任务，选用均方差损失函数作为图像重构辅助任务的目标函数，将异常图像检测和定位量化为一个不等式约束优化问题。采用无缝融合技术将辅助数据集、异常样本融合到正常样本中，以缩小异常融合样本与正常样本的差异，扩大异常样本总量，平衡异常样本、正常样本的比例。通过多组噪声模拟实验和现场实验证明，以一定概率在抵抗区添加高斯噪声进行增强训练，可提高HRDD模型的抗噪效能、泛化能力、检测准确率等。消融实验结果表明：辅助数据集有效地改善了样本不平衡问题，准确率提高了36.5%；引入图像重构辅助任务可保证深层特征能够准确映射到异常位置，交并比（IoU）提升了33.4%；辅助数据集与图像重构辅助任务之间存在强耦合作用，二者组合使用能进一步激发HRDD算法的性能潜力；添加无缝融合样本、高斯噪声增强等在一定程度上提高了HRDD模型的泛化能力。对照实验结果表明，HRDD算法准确率及IoU均优于其他主流算法，相比FCDD算法，HRDD算法准确率、IoU分别提高了4.6%，7.0%，更适用于煤矿现场。
- 煤矿运输系统 /
- 异常图像检测 /
- 全卷积数据描述 /
- 超球重构数据描述 /
- HRDD /
- 图像重构 /
- 无缝融合技术 /
- 高斯噪声增强
Abstract: There are various types and scenarios of abnormal risks in coal mine transportation systems. The occurrence of abnormal accidents at coal mine sites is accidental, and the number of abnormal samples obtained is much smaller than that of normal samples, resulting in an imbalance of positive and negative samples. In order to solve the above problems, a multivariate abnormal image detection method for coal mine transportation systems based on hypersphere reconstructed data description (HRDD) is proposed. On the basis of full convolutional data description (FCDD), an image reconstruction auxiliary task is introduced. The mean square error loss function is selected as the objective function of the image reconstruction auxiliary task. Abnormal image detection and positioning are quantified as an inequality constrained optimization problem. The seamless fusion technology is used to fuse auxiliary datasets and abnormal samples into normal samples, in order to reduce the difference between abnormal fusion samples and normal samples, expand the total number of abnormal samples, and balance the proportion of abnormal and normal samples. Through multiple sets of noise simulation experiments and on-site experiments, it has been proven that adding Gaussian noise to the resistance zone with a certain probability for enhanced training can improve the noise resistance efficiency, generalization capability, detection accuracy, and other aspects of the HRDD model. The results of the ablation experiment show that the auxiliary dataset effectively improves the problem of sample imbalance, with an accuracy increase of 36.5%. The introduction of image reconstruction auxiliary tasks can ensure that deep features can be accurately mapped to abnormal positions, resulting in an IoU improvement of 33.4%. There is a strong coupling effect between the auxiliary dataset and the image reconstruction auxiliary task. The combination of the two can further stimulate the performance potential of the HRDD algorithm. The addition of seamless fusion samples and Gaussian noise enhancement has to some extent improved the generalization capability of the HRDD model. The comparative experimental results show that the accuracy and IoU of the HRDD algorithm are better than other mainstream algorithms. Compared with the FCDD algorithm, the accuracy and IoU of the HRDD algorithm have increased by 4.6% and 7.0% respectively, making it more suitable for coal mine sites.
- coal mine transportation system /
- abnormal image detection /
- full convolutional data description /
- hypersphere reconstruction data description /
- HRDD /
- image reconstruction /
- seamless integration technology /
- Gaussian noise enhancement

HTML全文

图 1 HRDD基本形式

Figure 1. Basic form of hypersphere reconstructed data description (HRDD)

下载: 全尺寸图片幻灯片

图 2 使用U−net结构的HRDD模型

Figure 2. Hypersphere reconstructed data description model using U-net structure

下载: 全尺寸图片幻灯片

图 3 煤矿现场采集的样本

Figure 3. Samples collected on site in coal mines

下载: 全尺寸图片幻灯片

图 4 辅助数据集样本

Figure 4. Auxiliary dataset samples

下载: 全尺寸图片幻灯片

图 5 无缝融合样本

Figure 5. Seamless fusion of samples

下载: 全尺寸图片幻灯片

图 6 异常样本检测准确率曲线

Figure 6. Accuracy curves of abnormal samples detection

下载: 全尺寸图片幻灯片

图 7 高斯噪声曲线

Figure 7. Gaussian noise curve

下载: 全尺寸图片幻灯片

图 8 高斯噪声图像对比

Figure 8. Comparison of Gaussian noise images

下载: 全尺寸图片幻灯片

图 9 高斯噪声增强训练

Figure 9. Gaussian noise enhancement training

下载: 全尺寸图片幻灯片

图 10 噪声模拟实验

Figure 10. Noise simulation experiments

下载: 全尺寸图片幻灯片

图 11 验证集上的可视化检测结果

Figure 11. Visual detection results on the validation set

下载: 全尺寸图片幻灯片

图 12 异常检测结果对比

Figure 12. Comparison of anomaly detection results

下载: 全尺寸图片幻灯片

表 1 模拟噪声来源

Table 1 Simulate noise sources

噪声类型	来源
高斯噪声	图像、通信干扰等
泊松噪声	光电脉冲干扰
椒盐噪声	强电干扰
旋转图片	相机位置挪动

下载: 导出CSV

表 2 原始模型和增强模型的准确率对比

Table 2 Comparison of accuracy between the original model and the enhanced model

模型	准确率/%	HSC损失	HRDD损失
原始模型	82.5	0.042	0.061
增强模型	95.0	0.029	0.044

下载: 导出CSV

表 3 HRDD模型训练参数

Table 3 Training parameters of hypersphere reconstructed data description model

参数名称	参数值	参数名称	参数值
学习率	10⁻⁴	噪声概率	0.2
平滑常数	（0.9，0.999）	融合样本占比	0.3
批量数	4	迭代次数	150

下载: 导出CSV

表 4 消融实验结果

Table 4 Results of ablation experiments %

实验序号	引入图像重构任务	使用辅助数据集	添加无缝融合样本	高斯噪声增强训练	IoU	准确率
1	−	−	−	−	24.5	62.3
2	√	−	−	−	26.5	60.3
3	−	√	−	−	33.5	85.1
4	√	√	−	−	43.1	91.3
5	√	√	√	−	46.2	94.3
6	−	√	√	√	36.2	90.1
7	√	√	√	√	48.3	96.6

下载: 导出CSV

表 5 4种算法性能评估

Table 5 Performance evaluation of four algorithms %

算法	准确率	IoU
HRDD	96.6	48.3
FCDD	92.3	44.5
嵌入注意力机制的FCDD	93.1	39.3
YOLOv5	90.5	46.8

下载: 导出CSV

表 6 MVTec AD数据集上的性能评估

Table 6 Performance evaluation on the MVTec AD dataset

算法	AUROC	PRO-score
AE	0.817	0.790
AnoGan^[19]	0.743	0.443
CAVGA^[20]	0.930	−
FCDD	0.960	−
HRDD	0.969	0.912
PaDiM^[21]	0.975	0.921

下载: 导出CSV

参考文献(21)

[1]	毛清华,郭文瑾,翟姣,等. 煤矿带式输送机异常状态视频AI识别技术研究[J]. 工矿自动化,2023,49(9):36-46. MAO Qinghua,GUO Wenjin,ZHAI Jiao,et al. Research on video AI recognition technology for abnormal state of coal mine belt conveyors[J]. Journal of Mine Automation,2023,49(9):36-46.
[2]	DAI Lili,ZHANG Xu,GARDONI P,et al. A new machine vision detection method for identifying and screening out various large foreign objects on coal belt conveyor lines[J]. Complex & Intelligent Systems,2023,9(5):5221-5234.
[3]	MAO Qinghua,LI Shikun,HU Xin,et al. Coal mine belt conveyor foreign objects recognition method of improved YOLOv5 algorithm with defogging and deblurring[J]. Energies,2022,15(24). DOI:10. 3390/en15249504.
[4]	郝帅,张旭,马旭,等. 基于CBAM−YOLOv5的煤矿输送带异物检测[J]. 煤炭学报,2022,47(11):4147-4156. HAO Shuai,ZHANG Xu,MA Xu,et al. Foreign object detection in coal mine conveyor belt based on CBAM-YOLOv5[J]. Journal of China Coal Society,2022,47(11):4147-4156.
[5]	HAO Bonan,ZHANG Liya,PENG Ran. An improved vibe algorithm to detect personnel underground in coal mines[J]. Journal of Physics:Conference Series,2021,2025(1). DOI: 10.1088/1742-6596/2025/1/012032.
[6]	刘浩,刘海滨,孙宇,等. 煤矿井下员工不安全行为智能识别系统[J]. 煤炭学报,2021,46(增刊2):1159-1169. LIU Hao,LIU Haibin,SUN Yu,et al. Intelligent recognition system of unsafe behavior of underground coal miners[J]. Journal of China Coal Society,2021,46(S2):1159-1169.
[7]	KIM B,LEE J. A video-based fire detection using deep learning models[J]. Applied Sciences,2019,9(14). DOI: 10.3390/app9142862.
[8]	梁煜,陈童,张为. 融合自适应注意力的多尺度火灾检测算法[J]. 北京理工大学学报,2024,44(1):91-101. LIANG Yu,CHEN Tong,ZHANG Wei. Multi-scale fire detection algorithm with adaptive attention[J]. Transactions of Beijing Institute of Technology,2024,44(1):91-101.
[9]	LIZNERSKI P,RUFF L,VANDERMEULEN R A. Explainable deep one-class classification[EB/OL]. [2024-04-10]. https://arxiv.org/abs/2007.01760.
[10]	来杰,王晓丹,向前,等. 自编码器及其应用综述[J]. 通信学报,2021,42(9):218-230. DOI: 10.11959/j.issn.1000-436x.2021160 LAI Jie,WANG Xiaodan,XIANG Qian,et al. Review on autoencoder and its application[J]. Journal on Communications,2021,42(9):218-230. DOI: 10.11959/j.issn.1000-436x.2021160
[11]	殷晓航,王永才,李德英. 基于U-Net结构改进的医学影像分割技术综述[J]. 软件学报,2021,32(2):519-550. YIN Xiaohang,WANG Yongcai,LI Deying. Suvery of medical image segmentation technology based on U-net structure improvement[J]. Journal of Software,2021,32(2):519-550.
[12]	刘英,何雪,李单阳,等. 注意力引导全尺度连接网络的高分辨率影像变化检测[J]. 遥感学报,2024,28(4):1052-1065. LIU Ying,HE Xue,LI Danyang,et al. CBAM UNet+++:attention mechanism to guide change detection studies of full-scale connected networks[J]. National Remote Sensing Bulletin,2024,28(4):1052-1065.
[13]	WOO S H,PARK J Y,LEE J Y,et al. CBAM:convolutional block attention module[C]. European Conference on Computer Vision,Munich,2018:3-19.
[14]	RUFF L,VANDERMEULEN R A,FRANKS B J,et al. Rethinking assumptions in deep anomaly detection[EB/OL]. [2024-04-10]. https://arxiv.org/abs/2006.00339v3.
[15]	傅新元,郭禾,王宇新,等. 基于抠像技术的图像无缝融合算法[J]. 中国图象图形学报,2008,13(6):1082-1089. DOI: 10.11834/jig.20080609 FU Xinyuan,GUO He,WANG Yuxin,et al. Seamless cloning algorithm based on image matting technique[J]. Journal of Image and Graphics,2008,13(6):1082-1089. DOI: 10.11834/jig.20080609
[16]	邵延华,张铎,楚红雨,等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报,2022,44(10):3697-3708. DOI: 10.11999/JEIT210790 SHAO Yanhua,ZHANG Duo,CHU Hongyu,et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology,2022,44(10):3697-3708. DOI: 10.11999/JEIT210790
[17]	BERGMANN P,FAUSER M,SATTLEGGER D,et al. MVTec AD-a comprehensive real-world dataset for unsupervised anomaly detection[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Long Beach,2019:9584-9592.
[18]	吕承侃,沈飞,张正涛,等. 图像异常检测研究现状综述[J]. 自动化学报,2022,48(6):1402-1428. LYU Chengkan,SHEN Fei,ZHANG Zhengtao,et al. Review of image anomaly detection[J]. Acta Automatica Sinica,2022,48(6):1402-1428.
[19]	LI Dan,CHEN Dacheng,GOH J,et al. Anomaly detection with generative adversarial networks for multivariate time series[EB/OL]. [2024-04-10]. https://arxiv.org/abs/1809.04758.
[20]	VENKATARAMANAN S,PENG Kuanchuan,SINGH R V,et al. Attention guided anomaly localization in images[EB/OL]. [2024-04-10]. https://arxiv.org/abs/1911.08616.
[21]	DEFARD T,SETKOV A,LOESCH A,et al. PaDiM:a patch distribution modeling framework for anomaly detection and localization[EB/OL]. [2024-04-10]. https://arxiv.org/abs/2011.08785.

施引文献(14)

期刊类型引用(8)

1.	鞠慕涵，刘万科，胡捷，谷宇鹏. 改进D~*算法的未知场景机器人运动规划. 导航定位学报. 2024(03): 145-153 . 百度学术
2.	陈思成，纪玉杰，路达. 采样机器人路径规划研究. 内燃机与配件. 2022(22): 96-98 . 百度学术
3.	郝天轩，赵立桢. 跨平台矿井应急救援路径寻优方案研究. 工矿自动化. 2020(05): 108-112 . 本站查看
4.	熊雄. 多出口建筑疏散最优路径分层搜索算法仿真. 计算机仿真. 2020(08): 419-423 . 百度学术
5.	王飞，江明. 基于一种改进的蚁群算法的移动机器人三维路径规划研究. 安徽工程大学学报. 2019(03): 43-48 . 百度学术
6.	张苏英，赵国花，郭宝樑，于佳兴，刘慧贤. 基于改进的蚁群算法的移动机器人路径规划. 河北工业科技. 2019(06): 390-395 . 百度学术
7.	沈显庆，孙启智. BSO算法在移动机器人三维路径规划中的应用. 黑龙江科技大学学报. 2019(06): 747-751 . 百度学术
8.	龚星宇，常心坦，贾澎涛，罗碧波. 基于蚁群算法的井下救援路径优化方法. 工矿自动化. 2018(03): 76-81 . 本站查看