Research on anti-occlusion tracking method for underground mine personnel based on adaptive link optimization
-
摘要:
针对煤矿井下行人因遮挡频繁和外观混淆导致轨迹匹配不准确的问题，提出了一种基于自适应链接优化的井下行人抗遮挡跟踪方法。首先，根据目标置信度变化率和交并比计算，对目标进行遮挡判定，筛选出潜在遮挡目标。然后，在匹配级联阶段，引入潜在遮挡目标的非线性动态特征，并结合历史轨迹信息扩展轨迹链接优化模块的轨迹对输入，同时在轨迹对输入进行时域块处理后添加通道先验卷积注意力机制，增强时域表征能力。轨迹对输入向量经压缩与融合处理后，由多层感知器输出轨迹相似性得分，与原有匹配级联阶段中卡尔曼滤波器的总成本函数相结合，优化匹配决策，有效缓解轨迹匹配过程中的错误匹配问题。最后，在交并比匹配阶段，通过计算断裂率和ID切换率的变化量，引入自适应RB因子，构建反馈机制，动态调整匹配决策中的交并比阈值，以适应因长时间遮挡导致的轨迹断裂问题。采用所提方法与DeepSORT，YOLOv7−SAM，OSNet，FuCoLoT对煤矿井下典型视频序列进行对比实验，结果表明，所提方法的跟踪准确度(MOTA)，跟踪精度(MOTP)和身份F1(IDF1)分别为76.17%，84.13%，74.9%，较DeepSORT分别提升了14.9%，1.83%和10.93%，较YOLOv7−SAM分别提升了1.57%，0.4%和0.37%，较OSNet分别提升了2.83%，0.77%和1.27%，较FuCoLoT分别提升了2.5%，0.08%和1.8%，说明所提方法能够有效解决煤矿井下目标在遮挡情形下的跟踪误匹配问题。
Abstract:To address the issue of inaccurate trajectory matching caused by frequent occlusions and appearance confusion of underground mine personnel in coal mines, an anti-occlusion tracking method for underground mine personnel based on adaptive link optimization was proposed. Firstly, occlusion detection of the targets was performed based on the target confidence change rate and intersection-over-union (IoU) calculation to identify potential occluded targets. Secondly, in the matching cascade stage, nonlinear dynamic features of potential occluded targets were introduced, and historical trajectory information was incorporated to expand the trajectory pair input for the trajectory link optimization module. Additionally, after performing time-domain block processing on the trajectory pair input, a channel prior convolutional attention mechanism was added to enhance the time-domain representation capability. After compression and fusion processing of the trajectory pair input vectors, a trajectory similarity score was output by the multilayer perceptron. This score was combined with the total cost function of the Kalman filter in the original matching cascade stage to optimize matching decisions, effectively alleviating the issue of incorrect matching during the trajectory matching process. Finally, in the IoU matching stage, an adaptive RB factor was introduced by calculating the variations in fracture rate and ID switch rate to construct a feedback mechanism. This mechanism dynamically adjusted the IoU threshold in the matching decision to address trajectory fragmentation caused by long-term occlusion. Comparative experiments were conducted on typical video sequences from underground coal mines using the proposed method, DeepSORT, YOLOv7-SAM, OSNet, and FuCoLoT. The results showed that the proposed method achieved the multiple object tracking accuracy (MOTA) of 76.17%, the multiple object tracking precision (MOTP) of 84.13%, and the identity F1 (IDF1) of 74.9%. Compared to DeepSORT, these values improved by 14.9%, 1.83%, and 10.93%, respectively. Compared to YOLOv7-SAM, they improved by 1.57%, 0.4%, and 0.37%, respectively. Compared to OSNet, they improved by 2.83%, 0.77%, and 1.27%, respectively. Compared to FuCoLoT, they improved by 2.5%, 0.08%, and 1.8%, respectively. This demonstrates that the proposed method can effectively address the issue of tracking mismatches in occlusion scenarios in underground coal mine targets.
-
-
![]()
图 2 一段视频序列井下行人目标置信度变化
Figure 2. Variation in underground mine personnel target in a video sequence
![]()
图 3 基于自适应链接优化的井下行人抗遮挡跟踪方法架构
Figure 3. Architecture of anti-occlusion tracking method for underground mine personnel based on adaptive link optimization
![]()
图 7 COLinker，OLinker和AFLink模块的预测概率分布
Figure 7. Predicted probability distribution chart of COLinker, OLinker, and AFLink modules
![]()
图 8 COLinker，OLinker和AFLink模块的综合评估指标
Figure 8. Comprehensive evaluation indicators of COLinker, OLinker, and AFLink modules
![]()
图 9 COLinker，OLinker和AFLink模块的训练结果
Figure 9. Training results of COLinker, OLinker, and AFLink modules
![]()
图 10 不同遮挡程度下各模块的轨迹断裂恢复率对比
Figure 10. Comparison of trajectory fragmentation recovery rates of each module under different occlusion degrees
![]()
图 11 $ {R_k} $，$ {B_k} $和匹配阈值变化率折线图
Figure 11. Line chart of variation rates of Rk ，Bk and matching threshold
![]()
图 12 5种方法对MPDD数据集的目标跟踪结果
Figure 12. Target tracking results of 5 methods on MPDD dataset
![]()
图 13 5种方法对MOT17和MOT15数据集跟踪实验结果
Figure 13. Tracking experiment results of 5 methods on MOT17 and MOT15 datasets
表 1 数据集详细情况与特征描述
Table 1 Dataset details and feature descriptions
场景 视频时长/s 分段数 抽取图像数/张 数据特点 井下闸机出入口 670 7 7 884 视频分辨率高；拍摄区域合适；遮挡情况多；入口处目标运动模糊程度较低 井底硐室 82 3 1 634 视频分辨率低；拍摄区域狭长；遮挡情况较少；入口处目标运动模糊程度较高 井下工作面 1 407 4 3 346 视频分辨率低；拍摄区域合适；遮挡情况很少；目标模糊程度较低；图像质量差，低光照和局部曝光情况严重 
下载: 导出CSV
表 2 一段视频序列中各遮挡程度的分布情况
Table 2 Distribution of occlusion degrees in a video sequence
遮挡程度 无遮挡
（0）轻度遮挡
（1%～10%）部分遮挡
（10%～35%）严重遮挡
（35%～80%）完全遮挡
（≥80%）帧数比/% 4.9 9.1 18.9 53.8 13.3 帧数 135 251 522 1 485 367
下载: 导出CSV
表 3 COLinker，OLinker和AFLink模块的评估指标均值
Table 3 Mean values of evaluation indicators of COlincker, OLinker and AFLink modules
模型 精确率/% 召回率/% F1分数 AFLink 92 89 0.90 OLinker 93 91 0.92 COLinker 96 94 0.95
下载: 导出CSV
表 4 3段井下视频序列下5种方法指标对比
Table 4 Comparison of indicators of 5 methods in 3 underground video sequences
% 视频序列 方法 MOTA MOTP IDF1 Video1 DeepSORT 61.3 81.9 69.7 FuCoLoT 74.9 82.1 74.1 YOLOv7−SAM 75.5 82.3 73.2 OSNet 74.3 82.3 74.1 本文方法 76.9 82.4 75.9 Video2 DeepSORT 53.2 81.5 51.9 FuCoLoT 65.2 84.7 68.8 YOLOv7−SAM 66.7 84.5 67.9 OSNet 64.8 84.3 68.6 本文方法 69.3 85.3 70.2 Video3 DeepSORT 69.3 83.5 70.3 FuCoLoT 80.9 83.7 76.6 YOLOv7−SAM 81.6 84.4 76.5 OSNet 80.9 83.5 78.2 本文方法 82.3 84.7 78.6
下载: 导出CSV
表 5 公共视频序列下5种方法指标对比
Table 5 Comparison of indicators of 5 methods in public video sequence
% 视频序列 方法 MOTA MOTP IDF1 MOT17−09−SDP DeepSORT 67.1 82.3 70.7 FuCoLoT 76.4 82.8 74.5 YOLOv7−SAM 78.6 83.2 75.9 OSNet 76.4 82.9 7.7 本文方法 78.9 83.5 75.6 AVG−TownCentre DeepSORT 69.8 80.1 71.6 FuCoLoT 78.5 83.1 74.7 YOLOv7−SAM 79.5 84.3 76.8 OSNet 80.2 83.9 74.8 本文方法 81.1 84.8 76.4
下载: 导出CSV
-
[1] 张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579-597. ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote controlon roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579-597.
[2] 李翠芸,衡博文,谢金池. 模糊推理优化的抗遮挡PMBM跟踪算法[J]. 西安电子科技大学学报,2023,50(5):54-64. LI Cuiyun,HENG Bowen,XIE Jinchi. Anti-occlusion PMBM tracking algorithm optimized by fuzzy inference[J]. Journal of Xidian University,2023,50(5):54-64.
[3] LU Xiankai,MA Chao,NI Bingbing,et al. Adaptive region proposal with channel regularization for robust object tracking[J]. IEEE Transactions on Circuits and Systems for Video Technology,2019,31(4):1268-1282.
[4] 牟琦,韩嘉嘉,张寒,等. 基于云边协同的煤矿井下尺度自适应目标跟踪方法[J]. 工矿自动化,2023,49(4):50-61. MU Qi,HAN Jiajia,ZHANG Han,et al. A scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration[J]. Journal of Mine Automation,2023,49(4):50-61.
[5] 冯磊,李斌,何勇. 基于注意力机制的煤矿工作面遮挡行人检测[J]. 价值工程,2022,41(14):148-150. DOI: 10.3969/j.issn.1006-4311.2022.14.047 FENG Lei,LI Bin,HE Yong. Pedestrian detection of coal mine working face occlusion based on attention mechanism[J]. Value Engineering,2022,41(14):148-150. DOI: 10.3969/j.issn.1006-4311.2022.14.047
[6] 程健,陈亮,王凯,等. 一种多特征融合的复杂场景动态目标跟踪算法[J]. 中国矿业大学学报,2021,50(5):1002-1010. CHENG Jian,CHEN Liang,WANG Kai,et al. Multi-feature fusion dynamic target tracking algorithm for complex scenes[J]. Journal of China University of Mining & Technology,2021,50(5):1002-1010.
[7] COMANICIU D,MEER P. Mean shift analysis and applications[C]. 7th IEEE International Conference on Computer Vision,Kerkyra,1999:1197-1203.
[8] 陈伟,任鹏,田子建,等. 基于注意力机制的无监督矿井人员跟踪[J]. 煤炭学报,2021,46(增刊1):601-608. CHEN Wei,REN Peng,TIAN Zijian,et al. Unsupervised mine personnel tracking based on attention mechanism[J]. Journal of China Coal Society,2021,46(S1):601-608.
[9] LI Xiaoyu,WANG Shuai,CHEN Wei,et al. An intelligent vision-based tracking method for underground human using infrared videos[J]. Symmetry,2022,14(8). DOI: 10.3390/SYM14081750.
[10] 陈太光,鲍新平,汪涛,等. 基于深度学习的矿井巷道人员计数技术[J]. 煤矿安全,2023,54(2):234-238. CHEN Taiguang,BAO Xinping,WANG Tao,et al. Mine roadway personnel counting technology based on deep learning[J]. Safety in Coal Mines,2023,54(2):234-238.
[11] 张玉涛,张梦凡,史学强,等. 基于深度学习的井下运动目标跟踪算法研究[J]. 煤炭工程,2022,54(10):151-155. ZHANG Yutao,ZHANG Mengfan,SHI Xueqiang,et al. Object tracking algorithm of moving objects in underground mine based on deep learning[J]. Coal Engineering,2022,54(10):151-155.
[12] 邵小强,李鑫,杨涛,等. 改进YOLOv5s和DeepSORT的井下人员检测及跟踪算法[J]. 煤炭科学技术,2023,51(10):291-301. SHAO Xiaoqiang,LI Xin,YANG Tao,et al. Underground personnel detection and tracking based on improved YOLOv5s and DeepSORT[J]. Coal Science and Technology,2023,51(10):291-301.
[13] 范伟强,王雪瑾,张颖慧,等. 改进YOLOv7和DeepSORT的井下人员检测与跟踪算法[J/OL]. 煤炭科学技术:1-13[2024-10-07]. http://kns.cnki.net/kcms/detail/11.2402.TD.20240119.1449.006.html. FAN Weiqiang,WANG Xuejin,ZHANG Yinghui,et al. Underground personnel detection and tracking using improved YOLOv7 and DeepSORT[J/OL]. Coal Science and Technology:1-13[2024-10-07]. http://kns.cnki.net/kcms/detail/11.2402.TD.20240119.1449.006.html.
[14] HUANG Hejun,CHEN Zuguo,ZOU Ying,et al. Channel prior convolutional attention for medical image segmentation[J]. Computers in Biology and Medicine,2024,178. DOI: 10.1016/J.COMPBIOMED.2024.108784.
[15] ZHOU Chunluan,YUAN Junsong. Multi-label learning of part detectors for heavily occluded pedestrian detection[C]. IEEE International Conference on Computer Vision,Venice,2017:3486-3495.
[16] JAHANTIGHY A,TORABI H,MOHANNA F. Multiple targets video tracking based on extended Kalman filter in combination with particle swarm optimization for intelligent applications[J]. SN Applied Sciences,2023,5(3). DOI: 10.1007/S42452-023-05297-9.
[17] 李凡长,刘洋,吴鹏翔,等. 元学习研究综述[J]. 计算机学报,2021,44(2):422-446. DOI: 10.11897/SP.J.1016.2021.00422 LI Fanzhang,LIU Yang,WU Pengxiang,et al. A survey on recent advances in meta-learning[J]. Chinese Journal of Computers,2021,44(2):422-446. DOI: 10.11897/SP.J.1016.2021.00422
[18] WOJKE N,BEWLEY A,PAULUS D. Simple online and realtime tracking with a deep association metric[C]. IEEE International Conference on Image Processing,Beijing,2017:3645-3649.
[19] WANG C Y,YEH I H,MARK LIAO H Y. YOLOv9:learning what you want toLearn using programmable gradient information[C]. European Conferenceon Computer Vision,Cham,2024:1-21.
[20] 李玺,查宇飞,张天柱,等. 深度学习的目标跟踪算法综述[J]. 中国图象图形学报,2019,24(12):2057-2080. DOI: 10.11834/jig.190372 LI Xi,ZHA Yufei,ZHANG Tianzhu,et al. Survey of visual object tracking algorithms based on deep learning[J]. Journal of Image and Graphics,2019,24(12):2057-2080. DOI: 10.11834/jig.190372
[21] DU Yunhao,ZHAO Zhicheng,SONG Yang,et al. StrongSORT:make DeepSORT great again[J]. IEEE Transactions on Multimedia,2023,25:8725-8737. DOI: 10.1109/TMM.2023.3240881
[22] LUKEžIč A,ZAJC L Č,VOJÍř T,et al. FuCoLoT-A fully-correlational long-term tracker[C]. Asian Conference on Computer Vision,Cham,2019:595-611.
[23] NEWBURGER E,CORRELL M,ELMQVIST N. Fitting bell curves to data distributions using visualization[J]. IEEE Transactions on Visualization and Computer Graphics,2023,29(12):5372-5383. DOI: 10.1109/TVCG.2022.3210763
[24] KEUPER M,TANG Siyu,ANDRES B,et al. Motion segmentation & multiple object tracking by correlation co-clustering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(1):140-153. DOI: 10.1109/TPAMI.2018.2876253
计量
- 文章访问数: 32
- HTML全文浏览量: 20
- PDF下载量: 4
