Research on coal-gangue identification technology driven by multi-source fusion of image features and vibration spectrum
-
摘要:
针对目前图像与振动信号融合的方法在煤矸识别领域应用存在特征融合困难、实时性和模型复杂度不满足实际应用要求等问题，设计了基于多头注意力（MA）的多层长短期记忆（ML−LSTM）模型MA−ML−LSTM。采用经粒子群优化（PSO）算法优化的变分模态分解（VMD）算法对振动信号进行处理，将能量、能量矩、峭度、波形因数与矩阵奇异值作为特征量，并采用一维卷积网络获取振动信息；在多分类网络ResNet−18基础上删除最后的全连接层，用于对煤矸图像进行深度特征提取；通过MA机制和ML−LSTM网络实现图像与振动双通道特征融合，强化各通道重要特征信息的表达。实验结果表明：MA−ML−LSTM模型的平均识别准确率达98.72%，相比传统单一的ResNet，MobilenetV3，1D−CNN，LSTM模型分别高4.60%，7.96%，5.37%，6.11%，相比EMD−RF，IMF−SVM，CSPNet−YOLOv7分别高4.18%，4.45%，3.46%，验证了图像特征与振动频谱多源融合驱动的煤矸识别技术的有效性。
Abstract:To address the challenges of feature fusion, real-time performance, and model complexity in the application of image and vibration signal fusion for coal-gangue identification, a multi-head attention (MA)-based multi-layer long short-term memory (ML-LSTM) model, i.e., MA-ML-LSTM, was proposed. The variational mode decomposition (VMD) algorithm, optimized by particle swarm optimization (PSO), was employed to process vibration signals. Features such as energy, energy moment, kurtosis, waveform factor, and matrix singular values were extracted. A one-dimensional convolutional network was used to acquire vibration information. For image feature extraction, the fully connected layer of the multi-classification network ResNet-18 was removed, enabling the extraction of deep features from coal-gangue images. Dual-channel feature fusion of images and vibration signals was achieved using the MA mechanism and the ML-LSTM network, enhancing the expression of significant features in each channel. Experimental results demonstrated that the MA-ML-LSTM model achieved an average recognition accuracy of 98.72%, which was 4.60%, 7.96%, 5.37%, and 6.11% higher than traditional single models ResNet, MobilenetV3, 1D-CNN, and LSTM, respectively. Compared to EMD-RF, IMF-SVM, and CSPNet-YOLOv7 models, accuracy improved by 4.18%, 4.45%, and 3.46%, respectively. These findings validate the effectiveness of the coal-gangue identification technology driven by multi-source fusion of image features and vibration spectrum.
-
-
![]()
图 1 MA−ML−LSTM模型
Figure 1. Multi-head attention（MA）-multi-layers（ML）-long short-term memory（LSTM） model
![]()
图 6 实验平台装料箱的初始状态
Figure 6. Initial state of the material box of the experimental platform
表 1 PSO算法重复10次结果
Table 1 Result of PSO algorithm repeated for 10 times
次数 k α 次数 k α 1 10 2 087 6 10 2 062 2 10 2 165 7 10 2 151 3 10 2 094 8 10 2 114 4 10 2 132 9 10 2 173 5 10 2 189 10 10 2 058 
下载: 导出CSV
表 2 各IMF与原始信号的皮尔逊相关系数
Table 2 Pearson's correlation coefficient of each IMF with the original signal
IMF分量 相关系数 IMF分量 相关系数 IMF1 0.7826 IMF6 0.3696 IMF2 0.8519 IMF7 0.1285 IMF3 0.7342 IMF8 0.1324 IMF4 0.5623 IMF9 0.0165 IMF5 0.4676 IMF10 0.0085
下载: 导出CSV
表 3 消融实验分类结果
Table 3 Classification results of ablation experiment
% 方案 准确率 精确度 召回率 F1 A 92.10 91.93 92.16 92.04 B 95.57 95.72 96.17 95.94 C 97.12 97.08 97.15 97.11 D 98.72 98.66 98.62 98.64
下载: 导出CSV
表 4 不同模型的实验结果
Table 4 Experimental results of different models
% 模型 准确率 精确度 召回率 F1 ResNet 94.12 94.17 94.16 94.06 MobileNetV3 90.76 91.11 90.85 90.98 1D−CNN 93.35 93.64 92.79 93.21 LSTM 92.61 92.79 92.36 92.57 EMD−RF 94.54 95.31 94.99 94.65 IMF−SVM 94.27 94.41 94.85 94.13 CSPNet−YOLOv7 95.26 95.14 94.79 95.65 本文模型 98.72 98.66 98.62 98.64
下载: 导出CSV
-
[1] 于斌,徐刚,黄志增,等. 特厚煤层智能化综放开采理论与关键技术架构[J]. 煤炭学报,2019,44(1):42-53. YU Bin,XU Gang,HUANG Zhizeng,et al. Theory and its key technology framework of intelligentized fully-mechanized caving mining in extremely thick coal seam[J]. Journal of China Coal Society,2019,44(1):42-53.
[2] 葛世荣,王世博,管增伦,等. 数字孪生——应对智能化综采工作面技术挑战[J]. 工矿自动化,2022,48(7):1-12. GE Shirong,WANG Shibo,GUAN Zenglun,et al. Digital twin:meeting the technical challenges of intelligent fully mechanized working face[J]. Journal of Mine Automation,2022,48(7):1-12.
[3] 袁永,屠世浩,陈忠顺,等. 薄煤层智能开采技术研究现状与进展[J]. 煤炭科学技术,2020,48(5):1-17. YUAN Yong,TU Shihao,CHEN Zhongshun,et al. Current situation and development of intelligent mining technology for thin coal seams[J]. Coal Science and Technology,2020,48(5):1-17.
[4] 田妍,田丰. 放顶煤开采过程煤矸识别技术发展现状及前景[J]. 煤炭工程,2018,50(10):142-145. TIAN Yan,TIAN Feng. Development status and prospect of coal gangue recognition technology in top-coal caving[J]. Coal Engineering,2018,50(10):142-145.
[5] 沈科,季亮,张袁浩,等. 基于改进YOLOv5s模型的煤矸目标检测[J]. 工矿自动化,2021,47(11):107-111,118. SHEN Ke,JI Liang,ZHANG Yuanhao,et al. Research on coal and gangue detection algorithm based on improved YOLOv5s model[J]. Industry and Mine Automation,2021,47(11):107-111,118.
[6] 王家臣,潘卫东,张国英,等. 图像识别智能放煤技术原理与应用[J]. 煤炭学报,2022,47(1):87-101. WANG Jiachen,PAN Weidong,ZHANG Guoying,et al. Principles and applications of image-based recognition of withdrawn coal and intelligent control of draw opening in longwall top coal caving face[J]. Journal of China Coal Society,2022,47(1):87-101.
[7] 袁永,秦正寒,夏永琪,等. 基于改进U−Net的煤矸图像分割模型与放煤控制技术研究 [J/OL]. 煤炭学报:1-18[2024-10-30]. https://doi.org/10.13225/j.cnki.jccs.2024.0588. YUAN Yong,QIN Zhenghan,XIA Yongqi,et al. Research on coal gangue image recognition model based on improved U-Net and top coal caving control[J/OL]. Journal of China Coal Society:1-18 [2024-10-30]. https://doi. org/10.13225/j.cnki.jccs.2024.0588.
[8] 韦小龙,王方田,何东升,等. 基于CSPNet−YOLOv7目标检测算法的煤矸图像识别模型[J]. 煤炭科学技术,2024,52(增刊1):238-248. WEI Xiaolong,WANG Fangtian,HE Dongsheng,et al. Image recognition model of coal gangue based on CSPNet-YOLOv7 target detection algorithm[J]. Coal Science and Technology,2024,52(S1):238-248.
[9] 马英. 综放工作面自动化放顶煤系统研究[J]. 煤炭科学技术,2013,41(11):22-24,94. MA Ying. Study on automatic top coal caving system in fully-mechanized coal caving face[J]. Coal Science and Technology,2013,41(11):22-24,94.
[10] 万丽荣,陈博,杨扬,等. 单颗粒煤岩冲击放顶煤液压支架尾梁动态响应分析[J]. 煤炭学报,2019,44(9):2905-2913. WAN Lirong,CHEN Bo,YANG Yang,et al. Dynamic response of single coal-rock impacting tail beam of top coal caving hydraulic support[J]. Journal of China Coal Society,2019,44(9):2905-2913.
[11] 万丽荣,尹广俊,杨扬,等. 单颗粒岩石直冲金属板振动特性研究[J]. 煤炭技术,2017,36(11):213-216. WAN Lirong,YIN Guangjun,YANG Yang,et al. Study on vibration characteristics of single granular rock direct impact metal plate[J]. Coal Technology,2017,36(11):213-216.
[12] 曹贯强,尉瑞,孟祥涛,等. 用于煤矸识别的振动传感器设计[J]. 工矿自动化,2021,47(1):118-122. CAO Guanqiang,YU Rui,MENG Xiangtao,et al. Design of vibration sensor for coal gangue identification[J]. Industry and Mine Automation,2021,47(1):118-122.
[13] 窦希杰,王世博,刘后广,等. 基于EMD特征提取与随机森林的煤矸识别方法[J]. 工矿自动化,2021,47(3):60-65. DOU Xijie,WANG Shibo,LIU Houguang,et al. Coal and gangue identification method based on EMD feature extraction and random forest[J]. Industry and Mine Automation,2021,47(3):60-65.
[14] 窦希杰,王世博,谢洋,等. 基于IMF能量矩和SVM的煤矸识别[J]. 振动与冲击,2020,39(24):39-45. DOU Xijie,WANG Shibo,XIE Yang,et al. Coal and gangue identification based on IMF energy moment and SVM[J]. Journal of Vibration and Shock,2020,39(24):39-45.
[15] 薛光辉,柳二猛,赵新赢,等. 基于声压信号时域特征的综放工作面煤岩性状识别方法研究[J]. 煤炭工程,2015,47(6):119-122. XUE Guanghui,LIU Ermeng,ZHAO Xinying,et al. Research of coal-rock character recognition in fully mechanized caving face based on acoustic pressure data time domain feature[J]. Coal Engineering,2015,47(6):119-122.
[16] 袁源,汪嘉文,朱德昇,等. 顶煤放落过程煤矸声信号特征提取与分类方法[J]. 矿业科学学报,2021,6(6):711-720. YUAN Yuan,WANG Jiawen,ZHU Desheng,et al. Feature extraction and classification method of coal gangue acoustic signal during top coal caving[J]. Journal of Mining Science and Technology,2021,6(6):711-720.
[17] CHEN Xu,WANG Shibo,LIU Houguang,et al. Coal gangue recognition using multichannel auditory spectrogram of hydraulic support sound in convolutional neural network[J]. Measurement Science and Technology,2022,33(1). DOI: 10.1088/1361-6501/ac3709.
[18] 张宁波,刘长友,陈现辉,等. 综放煤矸低水平自然射线的涨落规律及测量识别分析[J]. 煤炭学报,2015,40(5):988-993. ZHANG Ningbo,LIU Changyou,CHEN Xianhui,et al. Measurement analysis on the fluctuation characteristics of low level natural radiation from gangue[J]. Journal of China Coal Society,2015,40(5):988-993.
[19] ARANDJELOVIC R,ZISSERMAN A. Look,listen and learn[C]. IEEE International Conference on Computer Vision,Venice,2017:609-617.
[20] OWENS A,EFROS A A. Audio-visual scene analysis with self-supervised multisensory features[EB/OL]. [2024-07-10]. https://arxiv.org/abs/1804.03641.
[21] WANG Wupeng,XING Chao,WANG Dong,et al. A robust audio-visual speech enhancement model[C]. IEEE International Conference on Acoustics,Speech and Signal Processing,Barcelona,2020:7529-7533.
[22] 张慧春,周子阳,边黎明,等. 基于1DCNN融合多源表型数据的杨树干旱胁迫评估方法[J]. 农业机械学报,2024,55(9):286-296. ZHANG Huichun,ZHOU Ziyang,BIAN Liming,et al. Assessment of poplar drought stress level based on 1DCNN fusion of multi-source phenotypic data[J]. Transactions of the Chinese Society for Agricultural Machinery,2024,55(9):286-296.
[23] 徐亮,王晶,杨文镜,等. 基于Conv−TasNet的多特征融合音视频联合语音分离算法[J]. 信号处理,2021,37(10):1799-1805. XU Liang,WANG Jing,YANG Wenjing,et al. Multi feature fusion audio-visual joint speech separation algorithm based on Conv-TasNet[J]. Journal of Signal Processing,2021,37(10):1799-1805.
[24] 樊凤杰,白洋,纪会芳. 基于EEMD−ICA的脑电去噪算法研究[J]. 计量学报,2021,42(3):395-400. FAN Fengjie,BAI Yang,JI Huifang. Denoising method of EEG signal based on EEMD-ICA[J]. Acta Metrologica Sinica,2021,42(3):395-400.
[25] VASWANI A,SHAZEER N M,PARMAR N,et al. Attention is all you need[EB/OL]. [2024-07-10]. https://arxiv.org/abs/1706.03762.
[26] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural Computation,1997,9:1735-1780. DOI: 10.1162/neco.1997.9.8.1735
[27] YANG Fan,SANG Yongsheng,LYU Jiancheng,et al. Prediction of gasoline yield in fluid catalytic cracking based on multiple level LSTM[J]. Chemical Engineering Research and Design,2022,185:119-129. DOI: 10.1016/j.cherd.2022.06.040
[28] 邢致恺,何怡刚,姚其新. 基于多模态信息融合的变压器在线故障诊断方法[J]. 电子测量与仪器学报,2024,38(9):95-103. XING Zhikai,HE Yigang,YAO Qixin. Transformer online fault diagnosis method based on multi-modal information fusion[J]. Journal of Electronic Measurement and Instrumentation,2024,38(9):95-103.
[29] 兰名扬,刘宇龙,金涛,等. 基于可视化轨迹圆和ResNet18的复合电能质量扰动类型识别[J]. 中国电机工程学报,2022,42(17):6274-6286. LAN Mingyang,LIU Yulong,JIN Tao,et al. An improved recognition method based on visual trajectory circle and ResnetN18 for complex power quality disturbances[J]. Proceedings of the CSEE,2022,42(17):6274-6286.
[30] HOWARD A,SANDLER M,CHU G,et al. Searching for MobileNetV3[EB/OL]. [2024-07-10]. https://arxiv.org/abs/1905.02244.
[31] KIRANYAZ S,AVCI O,ABDELJABER O,et al. 1D convolutional neural networks and applications:a survey[J]. Mechanical Systems and Signal Processing,2021,151. DOI:10.1016/j. ymssp.2020.107398.
计量
- 文章访问数: 339
- HTML全文浏览量: 36
- PDF下载量: 26
