Volume 50 Issue 3
Mar.  2024
Turn off MathJax
Article Contents
HAN Kang, LI Jingzhao, TAO Rongying. Recognition of unsafe behaviors of key position personnel in coal mines based on improved YOLOv7 and ByteTrack[J]. Journal of Mine Automation,2024,50(3):82-91.  doi: 10.13272/j.issn.1671-251x.2024030015
Citation: HAN Kang, LI Jingzhao, TAO Rongying. Recognition of unsafe behaviors of key position personnel in coal mines based on improved YOLOv7 and ByteTrack[J]. Journal of Mine Automation,2024,50(3):82-91.  doi: 10.13272/j.issn.1671-251x.2024030015

Recognition of unsafe behaviors of key position personnel in coal mines based on improved YOLOv7 and ByteTrack

doi: 10.13272/j.issn.1671-251x.2024030015
  • Received Date: 2024-03-06
  • Rev Recd Date: 2024-03-28
  • Available Online: 2024-04-11
  • The application of artificial intelligence technology can real-time recognize the behavior of key position personnel in coal mines, such as mine hoist drivers, to prevent dangerous situations such as equipment misoperation. It is of great significance for ensuring coal mine safety production. The personnel behavior recognition method based on image features has problems of poor resistance to background interference and insufficient real-time performance. In order to solve the above problems, a coal mine key position personnel unsafe behavior recognition method based on improved YOLOv7 and ByteTrack is proposed. Firstly, based on MobileOne and C3, lightweight improvements are made to the backbone and head network of the YOLOv7 object detection model to improve the inference speed of the model. Secondly, integrating ByteTrack tracking algorithm, to achieve the tracking and locking of personnel is achieved, and the capability to resist background interference is improved. Thirdly, MobileNetV2 is used to optimize the network structure of OpenPose and improve the efficiency of skeleton feature extraction. Finally, the spatial temporal graph convolutional networks (ST−GCN) is used to analyze the spatial structure and dynamic changes of the key points of the human skeleton in the time series, achieving the analysis and recognition of unsafe behaviors. The experimental results show that the precision of the MobileOneC3−YOLO model reaches 93.7%, and the inference speed is improved by 52% compared to the YOLOv7 model. The success rate of personnel locking model integrating ByteTrack reaches 97.1%. The improved OpenPose model reduces memory requirements by 170.3 MiB. The inference speed on CPU and GPU is improved by 74.7% and 54.9%, respectively; The recognition precision of the unsafe behavior recognition model for four types of unsafe behaviors, including fatigue sleeping on duty, leaving work, side talking, and playing with mobile phones, reaches 93.5%, and the inference speed reaches 18.6 frames per second.

     

  • loading
  • [1]
    李琰,刘珍,陈南希. 基于矿工大数据的不安全行为主题挖掘与语义分析[J]. 煤矿安全,2023,54(9):254-257.

    LI Yan,LIU Zhen,CHEN Nanxi. Topic mining and semantic analysis of unsafe behavior based on miner big data[J]. Safety in Coal Mines,2023,54(9):254-257.
    [2]
    黄辉,张雪. 煤矿员工不安全行为研究综述[J]. 煤炭工程,2018,50(6):123-127.

    HUANG Hui,ZHANG Xue. Review of research on unsafe behavior of miners[J]. Coal Engineering,2018,50(6):123-127.
    [3]
    丁恩杰,俞啸,夏冰,等. 矿山信息化发展及以数字孪生为核心的智慧矿山关键技术[J]. 煤炭学报,2022,47(1):564-578.

    DING Enjie,YU Xiao,XIA Bing,et al. Development of mine informatization and key technologies of intelligent mines[J]. Journal of China Coal Society,2022,47(1):564-578.
    [4]
    沈铭华,马昆,杨洋,等. AI智能视频识别技术在煤矿智慧矿山中的应用[J]. 煤炭工程,2023,55(4):92-97.

    SHEN Minghua,MA Kun,YANG Yang,et al. Application of AI identification technology in intelligent coal mine[J]. Coal Engineering,2023,55(4):92-97.
    [5]
    刘浩,刘海滨,孙宇,等. 煤矿井下员工不安全行为智能识别系统[J]. 煤炭学报,2021,46(增刊2):1159-1169.

    LIU Hao,LIU Haibin,SUN Yu,et al. Intelligent recognition system for unsafe behavior of coal mine employees underground[J]. Journal of China Coal Society,2021,46(S2):1159-1169.
    [6]
    温廷新,王贵通,孔祥博,等. 基于迁移学习与残差网络的矿工不安全行为识别[J]. 中国安全科学学报,2020,30(3):41-46.

    WEN Tingxin,WANG Guitong,KONG Xiangbo,et al. Identification of miners' unsafe behaviors based on transfer learning and residual network[J]. China Safety Science Journal,2020,30(3):41-46.
    [7]
    李占利,权锦成,靳红梅. 基于3D−Attention与多尺度的矿井人员行为识别算法[J]. 国外电子测量技术,2023,42(7):95-104. doi: 10.3969/j.issn.1002-8978.2023.07.014

    LI Zhanli,QUAN Jincheng,JIN Hongmei. Mine personnel behavior recognition algorithm based on 3D−Attention and multi-scale[J]. Foreign Electronic Measurement Technology,2023,42(7):95-104. doi: 10.3969/j.issn.1002-8978.2023.07.014
    [8]
    王宇,于春华,陈晓青,等. 基于多模态特征融合的井下人员不安全行为识别[J]. 工矿自动化,2023,49(11):138-144.

    WANG Yu,YU Chunhua,CHEN Xiaoqing,et al. Recognition of unsafe behaviors of underground personnel based on multi modal feature fusion[J]. Journal of Mine Automation,2023,49(11):138-144.
    [9]
    WANG C-Y,BOCHKOVSKIY A,LIAO H-Y M. YOLOv7:trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:7464-7475.
    [10]
    VASU P K A,GABRIEL J,ZHU J,et al. An improved one millisecond mobile backbone[EB/OL]. [2024-02-20]. https://arxiv.org/pdf/2206.04040.pdf.
    [11]
    黄家才,赵雪迪,高芳征,等. 基于改进YOLOv5s的草莓多阶段识别检测轻量化算法[J]. 农业工程学报,2023,39(21):181-187. doi: 10.11975/j.issn.1002-6819.202307186

    HUANG Jiacai, ZHAO Xuedi, GAO Fangzheng, et al. Recognizing and detecting the strawberry at multi-stages using improved lightweight YOLOv5s[J]. Transactions of the Chinese Society of Agricultural Engineering,2023,39(21):181-187. doi: 10.11975/j.issn.1002-6819.202307186
    [12]
    ZHANG Yifu,SUN Peize,JIANG Yi,et al. ByteTrack:multi-object tracking by associating every detection box[EB/OL]. [2024-02-20]. https://arxiv.org/abs/2110.06864v1.
    [13]
    CAO Zhe,SIMON T,WEI S-E,et al. Realtime multi-person 2D pose estimation using part affinity fields[C]. IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,2017:1302-1310.
    [14]
    SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2024-02-20]. https://arxiv.org/pdf/1409.1556.pdf.
    [15]
    SANDLER M,HOWARD A,ZHU Menglong,et al. MobileNetV2:inverted residuals and linear bottlenecks[EB/OL]. [2024-02-20]. http://arxiv.org/pdf/1801.04381.pdf.
    [16]
    YAN Sijie,XIONG Yuanjun,LIN Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]. AAAI Conference on Artificial Intelligence,New Orleans,2018:5361-5368.
    [17]
    BAI Shaojie,KOLTER J Z,KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. [2024-02-20]. https://arxiv.org/pdf/1803.01271.pdf.
    [18]
    MILAN A,LEAL-TAIXÉ L,REID I,et al. MOT16:a benchmark for multi-object tracking[EB/OL]. [2024-02-20]. https://arxiv.org/pdf/1603.00831.pdf.
    [19]
    BEWLEY A,GE Zongyuan,OTT L,et al. Simple online and realtime tracking[C]. IEEE International Conference on Image Processing,Phoenix,2016:3464-3468.
    [20]
    WOJKE N,BEWLEY A,PAULUS D. Simple online and realtime tracking with a deep association metric[C]. IEEE International Conference on Image Processing,Beijing,2017:3645-3649.
    [21]
    LIN T−Y,MAIRE M,BELONGIE S,et al. Microsoft COCO:common objects in context[EB/OL]. [2024-02-20]. https://www.microsoft.com/en-us/research/wp-content/uploads/2014/09/LinECCV14coco.pdf.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(14)  / Tables(7)

    Article Metrics

    Article views (272) PDF downloads(49) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return