A scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration
-
摘要: 煤矿井下监控视频中的运动目标通常存在较大的尺度变化和形变，导致基于计算机视觉的目标跟踪算法准确率不高，且海量的视频数据导致基于云端的集中式数据处理方式难以满足目标跟踪的实时性要求。针对上述问题，提出了一种基于云边协同的煤矿井下尺度自适应目标跟踪方法。设计了基于深度估计的尺度自适应目标跟踪算法，通过构建深度−尺度估计模型，利用目标深度值估计尺度值，实现尺度自适应目标跟踪，解决了目标尺度变化和形变导致跟踪准确率不高的问题；设计了一种基于云边协同的智能监控系统架构，将尺度自适应目标跟踪算法细粒度划分后的子模块按所需计算资源分别部署在系统的边缘端和云端，通过边缘端和云端的分布式并行处理提高算法运行效率，解决了集中式数据处理方式实时性差的问题。将基于云边协同的煤矿井下尺度自适应目标跟踪方法应用于煤矿井下视频序列，对其跟踪性能和实时性能进行实验验证，结果表明：与核相关滤波(KCF)、判别型尺度空间跟踪(DSST)算法、基于多特征融合的尺度自适应(SAMF)算法3种经典目标跟踪算法相比，基于深度估计的尺度自适应目标跟踪算法在煤矿井下目标出现较大尺度变化和形变时，具有更高的跟踪精度和成功率；与传统的云计算处理方式相比，基于云边协同的尺度自适应目标跟踪算法部署方式使算法总时延降低了32.55%，有效提升了煤矿井下智能监控系统目标跟踪的实时性能。Abstract: The moving targets in coal mine underground monitoring videos often have significant scale changes and deformations. This results in low accuracy of target tracking algorithms based on computer vision. Moreover, the massive amount of video data makes it difficult for centralized cloud-based data processing methods to meet the real-time requirements of target tracking. In order to solve the above problems, a scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration is proposed. A scale-adaptive target tracking algorithm based on depth estimation is designed. The scale-adaptive target tracking is achieved by constructing a depth-scale estimation model, which uses target depth values to estimate scale values. The problem of low tracking accuracy caused by target scale change and deformation is solved. An intelligent monitoring system architecture based on cloud-edge collaboration is designed. The sub-modules of the scale-adaptive target tracking algorithm, which are divided into fine granularity, are deployed at the edge and cloud of the system according to the required computing resources. The algorithm's operational efficiency is improved through distributed parallel processing at the edge and cloud, solving the problem of poor real-time performance in the centralized data processing. The scale-adaptive target tracking method based on cloud-edge collaboration is applied in coal mine underground video sequences. The tracking performance and real-time performance are verified experimentally. The results show that compared with three classic target tracking algorithms, namely kernel correlation filter (KCF), discriminant scale space tracking (DSST) algorithm, and scale adaptive multiple feature (SAMF) algorithm, the scale-adaptive target tracking algorithm based on depth estimation has higher tracking precision and success rate when there are significant scale changes and deformations in coal mine underground targets. Compared with traditional cloud computing processing methods, the deployment method of scale-adaptive target tracking algorithm based on cloud-edge collaboration reduces the total delay of the algorithm by 32.55%. It effectively improves the real-time performance of target tracking of intelligent monitoring system in coal mine underground.
-
图 2 基于深度估计的尺度自适应目标跟踪算法
Figure 2. Scale-adaptive target tracking algorithm based on depth estimation
图 3 目标跟踪算法5个子模块的运行时间
Figure 3. Running time of five sub-modules of the target tracking algorithm
图 4 基于云边协同的智能监控系统架构
Figure 4. Architecture of cloud-edge collaborative intelligent monitoring system
图 6 煤矿井下视频序列部分帧的RGB图像和深度图像
Figure 6. Part of RGB images and depth images of a coal mine underground video sequence
图 8 4种算法对煤矿井下数据集的目标跟踪结果
Figure 8. Target tracking results of four algorithms for coal mine underground data set
图 9 4种算法对OTB−100数据集的目标跟踪结果
Figure 9. Target tracking results of four algorithms for OTB-100 standard data set
图 10 目标跟踪算法在煤矿井下数据集上的精度和成功率
Figure 10. Tracking accuracy and success rate of four target tracking algorithms for coal mine underground data set
图 11 目标跟踪算法在OTB−100数据集上的精度和成功率
Figure 11. Tracking accuracy and success rate of four target tracking algorithms for OTB-100 data set
图 12 目标跟踪算法在2种部署方式下的局部时延曲线
Figure 12. Local delay curves of the proposed target tracking algorithm under two deployment modes
图 13 目标跟踪算法在2种部署方式下的总时延曲线
Figure 13. Overall delay curves of the proposed target tracking algorithm under two deployment mode
图 14 不同边缘服务器数量下目标跟踪算法总时延曲线
Figure 14. Overall delay curves of the proposed target tracking algorithm under different groups of edge servers
表 1 煤矿井下尺度自适应目标跟踪实验环境配置
Table 1. Experimental environment configuration of coal mine underground scale-adaptive target tracking method
实验环境 配置 边缘服务器 CPU AMD Ryzen 5 5600H with Radeon Graphics 3.30 GHz GPU GeForce RTX 3050 系统 Windows10 云端服务器 CPU Intel Xeon E5−2680 v2 GPU NVIDIA GeForce GTX TITAN Z 系统 centos7 编程语言 Matlab 
下载: 导出CSV
表 2 任务卸载相关参数配置
Table 2. Parameters configuration related to task unloading
设备 参数 值 终端设备 $ {B}_{i,j} $/MHz 100 $ {P}_{i,j} $/W 0.1 ${D}_{i,j} $ [0.1,1.0] 边缘服务器 $ {B}_{j,{\rm{c}}} $/MHz 200 $ {P}_{j,{\rm{c}}} $/W 0.5 $ {y}_{j} $/GHz 10 $ {C}_{j} $/($ \mathrm{c}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{s}\cdot {\mathrm{b}\mathrm{i}\mathrm{t}}^{-1} $) 200 $ D_{j,{\rm{c}}} $ [0.1,1.0] 云端服务器 $ {w}_{{\rm{c}}} $/GHz 100 $ {C}_{{\rm{c}}} $/($ \mathrm{c}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{s}\cdot {\mathrm{b}\mathrm{i}\mathrm{t}}^{-1} $) 50 $ {\sigma }^{2} $/dBm 100 其他 $ {b}_{{\rm{v}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $) [1,100] $ {b}_{{\rm{f}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $) [1,5] $ {b}_{{\rm{m}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $) [1,5]
下载: 导出CSV
表 3 煤矿井下视频序列数据信息
Table 3. Video sequence data information of coal mine underground
视频序列 帧数 主要影响因素 Video1 355 尺度变化 Video2 170 目标形变、背景杂乱 Video3 181 光照不均、尺度变化 Video4 297 光照极度不足、目标形变 Video5 215 目标形变 Video6 286 尺度变化
下载: 导出CSV
表 4 目标跟踪算法在2种部署方式下的传输时延和计算时延计算结果
Table 4. Calculated transmission delay and calculation delay of the proposed target tracking algorithm under two deployment modes
ms 视频序列 云边协同方式 云计算方式 传输时延 计算时延 传输时延 计算时延 Video1 22.491 0.012 78 209.559 6 0.004 61 Video2 23.361 0.013 41 190.456 9 0.004 73 Video3 21.436 0.012 13 205.454 9 0.004 47 Video4 22.267 0.011 98 193.576 8 0.004 34 Video5 21.879 0.015 76 200.177 4 0.004 91 Video6 21.253 0.013 05 193.115 1 0.004 39 Total 132.687 0.079 11 1 192.340 7 0.027 45 注：黑体数据为最优结果。
下载: 导出CSV
表 5 目标跟踪算法在2种部署方式下的总时延计算结果
Table 5. Calculated overall delay of the proposed target tracking algorithm under two deployment modes
ms 视频序列 总时延 云计算方式 云边协同方式 Video1 209.567 6 139.916 7 Video2 190.465 3 140.214 6 Video3 205.462 5 130.169 8 Video4 193.584 6 134.247 2 Video5 200.186 1 130.744 2 Video6 193.123 8 128.929 4 Total 1 192.389 9 804.221 9 注：黑体数据为最优结果。
下载: 导出CSV
表 6 不同边缘服务器数量下目标跟踪算法时延
Table 6. Delay of the proposed target tracking algorithm under different groups of edge servers
边缘服务器
数量/台视频序列
数量/个传统云计算
总时延/msM=12时云边协同时延/ms M=18时云边协同时延/ms 边缘端 云端 总时延 边缘端 云端 总时延 传输时延 计算时延 传输时延 计算时延 传输时延 计算时延 传输时延 计算时延 1 6 1192.3899 132.687 0.0791 671.429 0.0263 804.2219 132.687 0.0791 671.429 0.0263 804.2219 2 12 2384.7798 132.687 0.0791 671.429 0.0526 804.2481 132.687 0.0791 671.429 0.0526 804.2481 3 18 3577.1697 265.374 0.0791 1342.859 0.0789 1608.391 132.687 0.0791 671.429 0.0789 804.2744 4 24 4769.5596 265.371 0.0791 1342.859 0.1052 1608.417 265.374 0.0791 1342.859 0.1052 1608.417 5 30 5961.9495 398.061 0.0791 2014.288 0.1315 2412.560 265.374 0.0791 1342.859 0.1315 1608.443 6 36 7154.3394 398.061 0.0791 2014.288 0.1578 2412.586 265.374 0.0791 1342.859 0.1578 1608.470 注：黑体数据为最优结果。
下载: 导出CSV
-
[1] 王国法,赵国瑞,胡亚辉. 5G技术在煤矿智能化中的应用展望[J]. 煤炭学报,2020,45(1):16-23.WANG Guofa,ZHAO Guorui,HU Yahui. Application prospect of 5G technology in coal mine intelligence[J]. Journal of China Coal Society,2020,45(1):16-23. [2] 丁恩杰,俞啸,夏冰,等. 矿山信息化发展及以数字孪生为核心的智慧矿山关键技术[J]. 煤炭学报,2022,47(1):564-578.DING Enjie,YU Xiao,XIA Bing,et al. Development of mine informatization and key technologies of intelligent mines[J]. Journal of China Coal Society,2022,47(1):564-578. [3] 陈龙,王晓,杨健健,等. 平行矿山:从数字孪生到矿山智能[J]. 自动化学报,2021,47(7):1633-1645.CHEN Long,WANG Xiao,YANG Jianjian,et al. Parallel mining operating systems:from digital twins to mining intelligence[J]. Acta Automatica Sinica,2021,47(7):1633-1645. [4] 孙彦景,霍羽,陈岩,等. 矿山动态协同作业场景无线通信关键技术[J]. 煤炭学报,2021,46(1):321-332.SUN Yanjing,HUO Yu,CHEN Yan,et al. Key technology of service attribute driven wireless communication for dynamic cooperative operations in mining scenarios[J]. Journal of China Coal Society,2021,46(1):321-332. [5] 王学文,刘曙光,王雪松,等. AR/VR融合驱动的综采工作面智能监控关键技术研究与试验[J]. 煤炭学报,2022,47(2):969-985.WANG Xuewen,LIU Shuguang,WANG Xuesong,et al. Research and test on key technologies of intelligent monitoring and control driven by AR/VR for fully mechanized coal-mining face[J]. Journal of China Coal Society,2022,47(2):969-985. [6] LIU S,LIU D,SRIVASTAVA G,et al. Overview and methods of correlation filter algorithms in object tracking[J]. Complex & Intelligent Systems,2021,7(4):1895-1917. [7] 张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579-597.ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote controlon roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579-597. [8] 张帆,孙晓辉,崔东林. 基于ORB特征的矿井移动目标双目视觉跟踪与定位[J]. 煤炭学报,2018,43(增刊2):654-662.ZHANG Fan,SUN Xiaohui,CUI Donglin. Method of tracking and positioning for mobile target based on ORB features and binocular vision in mine[J]. Journal of China Coal Society,2018,43(S2):654-662. [9] 刘浩,刘海滨,孙宇,等. 煤矿井下员工不安全行为智能识别系统[J]. 煤炭学报,2021,46(增刊2):1159-1169.LIU Hao,LIU Haibin,SUN Yu,et al. Intelligent recognition system of unsafe behavior of underground coal miners[J]. Journal of China Coal Society,2021,46(S2):1159-1169. [10] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, 2010: 2544-2550. [11] HENRIQUES J F, CASEIRO R, MARTINES P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]. European Conference on Computer Vision, Berlin, 2012: 702-715. [12] HENRIQUES J F,CASEIRO R,MARTINS P,et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):583-596. [13] 覃剑,石昌伟,张媛,等. 边缘视频处理的细粒度划分与重组部署算法[J]. 电子学报,2021,49(11):2152-2159.QIN Jian,SHI Changwei,ZHANG Yuan,et al. Fine-grained partitioning and reorganization deployment strategy of edge video processing[J]. Acta Electronica Sinica,2021,49(11):2152-2159. [14] 卢新明,阚淑婷. 煤炭精准开采地质保障与透明地质云计算技术[J]. 煤炭学报,2019,44(8):2296-2305.LU Xinming,KAN Shuting. Geological guarantee and transparent geological cloud computing technology of precision coal mining[J]. Journal of China Coal Society,2019,44(8):2296-2305. [15] 施巍松,张星洲,王一帆,等. 边缘计算:现状与展望[J]. 计算机研究与发展,2019,56(1):69-89. doi: 10.7544/issn1000-1239.2019.20180760SHI Weisong,ZHANG Xingzhou,WANG Yifan,et al. Edge computing:state-of-the-art and future directions[J]. Journal of Computer Research and Development,2019,56(1):69-89. doi: 10.7544/issn1000-1239.2019.20180760 [16] 王国法,杜毅博,任怀伟,等. 智能化煤矿顶层设计研究与实践[J]. 煤炭学报,2020,45(6):1909-1924.WANG Guofa,DU Yibo,REN Huaiwei,et al. Top level design and practice of smart coal mines[J]. Journal of China Coal Society,2020,45(6):1909-1924. [17] 陈思光,陈佳民,赵传信. 基于深度强化学习的云边协同计算迁移研究[J]. 电子学报,2021,49(1):157-166.CHEN Siguang,CHEN Jiamin,ZHAO Chuanxin. Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism[J]. Acta Electronica Sinica,2021,49(1):157-166. [18] DANELLJAN M, HAGER G, KHAN F, et al. Accurate scale estimation for robust visual tracking[C]. British Machine Vision Conference, Nottingham, 2014: 1-11. [19] LI Yang, ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature integration[C]. European Conference on Computer Vision Workshops, Zurich, 2014: 254-265. [20] 孙继平,贾倪. 矿井视频图像中人员目标匹配与跟踪方法[J]. 中国矿业大学学报,2015,44(3):540-548.SUN Jiping,JIA Ni. Human target matching and tracking method in coal mine video[J]. Journal of China University of Mining & Technology,2015,44(3):540-548. [21] 程健,陈亮,王凯,等. 一种多特征融合的复杂场景动态目标跟踪算法[J]. 中国矿业大学学报,2021,50(5):1002-1010.CHENG Jian,CHEN Liang,WANG Kai,et al. Multi-feature fusion dynamic target tracking algorithm for complex scenes[J]. Journal of China University of Mining & Technology,2021,50(5):1002-1010. [22] 牟琦,张寒,何志强,等. 基于深度估计和特征融合的尺度自适应目标跟踪算法[J]. 图学学报,2021,42(4):563-571.MU Qi,ZHANG Han,HE Zhiqiang,et al. Scale adaptive target tracking algorithm based on depth estimation and feature fusion[J]. Journal of Graphics,2021,42(4):563-571. [23] GOARD C, AODHA O M, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]. Computer Vision and Pattern Recognition, Hawaii, 2017: 6602-6611. [24] HABER E E,NGUYEN T M,ASSI C. Joint optimization of computational cost and devices energy for task offloading in multi-tier edge-clouds[J]. IEEE Transactions on Communications,2019,67(5):3407-3421. doi: 10.1109/TCOMM.2019.2895040 [25] CHEN Siguang,ZHENG Yimin,LU Weifeng,et al. Energy optimal dynamic computation offloading for industrial IoT in fog computing[J]. IEEE Transactions on Green Communications and Networking,2019,4(2):566-576. [26] 张展,张宪琦,左德承,等. 面向边缘计算的目标追踪应用部署策略研究[J]. 软件学报,2020,31(9):2691-2708.ZHANG Zhan,ZHANG Xianqi,ZUO Decheng,et al. Research on target tracking application deployment strategy for edge computing[J]. Journal of Software,2020,31(9):2691-2708. [27] 姜德义,魏立科,王翀,等. 智慧矿山边缘云协同计算技术架构与基础保障关键技术探讨[J]. 煤炭学报,2020,45(1):484-492.JIANG Deyi,WEI Like,WANG Chong,et al. Discussion on the technology architecture and key basic support technology for intelligent mine edge-cloud collaborative computing[J]. Journal of China Coal Society,2020,45(1):484-492. [28] 张文柱,余静华. 移动边缘计算中基于云边端协同的任务卸载策略[J]. 计算机研究与发展,2023,60(2):371-385.ZHANG Wenzhu,YU Jinghua. Task offloading strategy in mobile edge computing based on cloud-edge-end cooperation[J]. Journal of Computer Research and Development,2023,60(2):371-385. -
点击查看大图
计量
- 文章访问数: 215
- HTML全文浏览量: 60
- PDF下载量: 26
- 被引次数: 0
