基于SEI的煤矿智能监控视频传输方法

陈佳; 王琪; 王鹏

doi:10.13272/j.issn.1671-251x.2023100025

基于SEI的煤矿智能监控视频传输方法

doi: 10.13272/j.issn.1671-251x.2023100025

陈佳^{1, 2,},
王琪^{1, 2},
王鹏^{1, 2}

1.
中煤科工集团常州研究院有限公司，江苏常州　213015
2.
天地（常州）自动化股份有限公司，江苏常州　210315

基金项目: 中煤科工集团常州研究院科研项目（2024GY0003）。

详细信息

作者简介:
陈佳（1994—），男，安徽铜陵人，硕士，主要从事煤矿智能视频平台开发工作，E-mail：cj068@mail.ustc.edu.cn

中图分类号: TD76
计量
- 文章访问数: 58
- HTML全文浏览量: 22
- PDF下载量: 14
- 被引次数: 0
出版历程
- 收稿日期: 2023-10-10
- 修回日期: 2024-05-18
- 网络出版日期: 2024-06-13

SEI based intelligent monitoring video transmission method for coal mines

CHEN Jia^{1, 2
,},
WANG Qi^{1, 2},
WANG Peng^{1, 2}

1.
CCTEG Changzhou Research Institute, Changzhou 213015, China
2.
Tiandi (Changzhou) Automation Co., Ltd., Changzhou 210315 , China

摘要

摘要: 目前煤矿视频监控数据传输存在高延迟问题，而视频传输延迟的主要成因是编码延迟。针对该问题，提出了一种无视频编码的基于媒体补充增强信息（SEI）的煤矿智能监控视频传输方法。该方法在解复用视频流得到视频压缩帧后缓存一份副本，解码视频压缩帧得到视频解码帧，通过SEI存储视频解码帧中AI模型分析结果，根据时间戳对应关系将自定义SEI写入该视频解码帧对应视频压缩帧副本的网络提取层单元，并复用视频压缩帧副本，实现煤矿智能监控视频实时传输。在24核CPU上对该方法进行实验测试，结果表明：对于1 280×720分辨率的视频，采用该方法处理视频时CPU整体使用率由采用传统方法时的24.7%～36.6%降至20.3%～23.9%，端到端延迟由1 946 ms降至345 ms；对于1 920×1 080分辨率的视频，采用该方法处理视频时CPU整体使用率由采用传统方法时的29.2%～41.8%降至18.5%～26.3%，端到端延迟由6 204 ms降至479 ms。该方法通过规避视频编码环节，降低了煤矿智能监控视频传输延迟，且节省了视频编码所需的CPU或GPU资源，降低了智能视频监控系统硬件成本。
- 煤矿智能视频监控 /
- 监控视频传输 /
- 视频AI分析 /
- 无视频编码 /
- 媒体补充增强信息 /
- 端到端延迟
Abstract: Currently, there is a high latency problem in the transmission of video surveillance data in coal mines, and the main cause of video transmission delay is encoding delay. In order to solve the above problems, a intelligent monitoring video transmission method for coal mines based on media supplemental enhancement information(SEI) without video encoding is proposed. This method caches a copy of the compressed video frame obtained by demultiplexing the video stream, and decodes the compressed video frame to obtain the decoded video frame. The method stores the AI model analysis results in the decoded video frame through SEI, writes the custom SEI into the network extraction layer unit corresponding to the compressed video frame copy of the decoded video frame based on the timestamp correspondence. The method multiplexes the compressed video frame copy to achieve real-time transmission of coal mine intelligent monitoring videos. Experimental testing of this method is conducted on a 24 core CPU. The results show that for videos with a resolution of 1280×720, the overall CPU utilization rate for video processing using this method decreases from 24.7% to 36.3% when using traditional methods to 20.3% to 23.9%. The end-to-end delay decreases from 1946 ms to 345 ms. For videos with a resolution of 1920×1080, the overall CPU utilization rate for video processing using this method decreases from 29.2% to 41.8% using traditional methods to 18.5% to 26.3%. The end-to-end latency decreases from 6204 ms to 479 ms. This method reduces the transmission delay of coal mine intelligent monitoring videos by avoiding the video encoding process, saves CPU or GPU resources required for video encoding, and reduces the hardware cost of the intelligent video monitoring system.
- intelligent video surveillance for coal mines /
- monitoring video transmission /
- AI analysis of video /
- no video encoding /
- media supplement and enhance information /
- end to end latency

HTML全文

图 1 基于SEI的煤矿智能监控视频传输方法

Figure 1. Intelligent monitoring video transmission method for coal mine based on supplemental enhancement information (SEI)

下载: 全尺寸图片幻灯片

图 2 解码流程示例

Figure 2. Example of decoding process

下载: 全尺寸图片幻灯片

图 3 自定义SEI结构

Figure 3. Structure of customized SEI

下载: 全尺寸图片幻灯片

图 4 YOLOv5模型检测结果

Figure 4. YOLOv5 model detection results

下载: 全尺寸图片幻灯片

图 5 视频处理过程中CPU使用率

Figure 5. CPU usage in video processing

下载: 全尺寸图片幻灯片

图 6 端到端延迟测试方法

Figure 6. End-to-end latency test method

下载: 全尺寸图片幻灯片

表 1 编码器参数设置

Table 1. Encoder parameters of H.264 encoder

参数值

preset medium
framerate 30
gop_size 30
open-gop false
no-scenecut true
forced-idr true
x264-params keyint=30:keyint_min=30:rc-lookahead=10

下载: 导出CSV

表 2 视频传输端到端延迟测试结果

Table 2. Test results of end-to-end latency of video transmission ms

视频端到端延迟

传统方法本文方法

1 1 946 345

2 6 204 479

下载: 导出CSV

参考文献(20)

[1]	王国法,刘峰,庞义辉,等. 煤矿智能化——煤炭工业高质量发展的核心技术支撑[J]. 煤炭学报,2019,44(2):349-357. WANG Guofa,LIU Feng,PANG Yihui,et al. Coal mine intellectualization:the core technology of high quality development[J]. Journal of China Coal Society,2019,44(2):349-357.
[2]	贺胜宽. 煤矿自动化信息化系统集成软件设计与实现[J]. 电子世界,2016 (19):134,138. HE Shengkuan. Design and implementation of coal mine automation information system integration software[J]. Electronic World,2016 (19):134,138.
[3]	程德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349-365. CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349-365.
[4]	巩师鑫,赵国瑞,王飞. 机器视觉感知理论与技术在煤炭工业领域应用进展综述[J]. 工矿自动化,2023,49(5):7-21. GONG Shixin,ZHAO Guorui,WANG Fei. Review on the application of machine vision perception theory and technology in coal industry[J]. Journal of Mine Automation,2023,49(5):7-21.
[5]	杨景峰. 基于AI视频识别技术的井下规范操作监控系统设计[J]. 陕西煤炭,2021,40(1):4-8,46. doi: 10.3969/j.issn.1671-749X.2021.01.003 YANG Jingfeng. Design of underground standard operation monitoring system based on AI video recognition technology[J]. Shaanxi Coal,2021,40(1):4-8,46. doi: 10.3969/j.issn.1671-749X.2021.01.003
[6]	陈芳. 基于AI图像识别技术的人员防闯入系统在煤矿的研究与应用[J]. 价值工程,2021,40(24):172-174. doi: 10.3969/j.issn.1006-4311.2021.24.056 CHEN Fang. Exploration and application of personnel intrusion prevention system based on AI technology in coal mine[J]. Value Engineering,2021,40(24):172-174. doi: 10.3969/j.issn.1006-4311.2021.24.056
[7]	孔骏儒,郭梦琪,郭梦曦,等. 一种基于AI图像处理技术的煤矿皮带运输系统:CN202210850361. X[P]. 2023-06-28. KONG Junru,GUO Mengqi,GUO Mengxi,et al. A coal mine belt transportation system based on AI image processing technology:CN202210850361. X[P]. 2023-06-28.
[8]	李文峰,路建通,雷文礼,等. 矿用实时视频传输系统设计[J]. 工矿自动化,2020,46(2):18-22. LI Wenfeng,LU Jiantong,LEI Wenli,et al. Design of mine-used real-time video transmission system[J]. Industry and Mine Automation,2020,46(2):18-22.
[9]	李敬兆,秦晓伟,汪磊. 基于边云协同框架的煤矿井下实时视频处理系统[J]. 工矿自动化,2021,47(12):1-7. LI Jingzhao,QIN Xiaowei,WANG Lei. Real-time video processing system in coal mine based on edge-cloud collaborative framework[J]. Industry and Mine Automation,2021,47(12):1-7.
[10]	毛清华,郭文瑾,翟姣,等. 煤矿带式输送机异常状态视频AI识别技术研究[J]. 工矿自动化,2023,49(9):36-46. MAO Qinghua,GUO Wenjin,ZHAI Jiao,et al. Research on video AI recognition technology for abnormal state of coal mine belt conveyors[J]. Journal of Mine Automation,2023,49(9):36-46.
[11]	高飞,赵杰,周幸福,等. 基于H. 264标准的实时数字视频水印方法:CN101860744A[P]. 2010-10-13. GAO Fei,ZHAO Jie,ZHOU Xingfu,et al. Real-time digital video watermarking method based on H.264 standard:CN101860744A[P]. 2010-10-13.
[12]	WIEGAND T,SULLIVAN G J,BJONTEGAARD G,et al. Overview of the H.264/AVC video coding standard[J]. IEEE Transactions on Circuits and Systems for Video Technology,2003,13(7):560-576. doi: 10.1109/TCSVT.2003.815165
[13]	邓立平. 基于H.264的视频加密算法的研究及实现[D]. 南京:南京邮电大学,2011. DENG Liping. Research and implementation of video encryption algorithm based on H.264[D]. Nanjing:Nanjing University of Posts and Telecommunications,2011.
[14]	SULLIVAN G J,OHM J R,HAN W J,et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on Circuits and Systems for Video Technology,2012,22(12):1649-1668. doi: 10.1109/TCSVT.2012.2221191
[15]	DOE J. YOLOv5:a better version of YOLO[J]. IEEE Transactions on Image Processing,2021,30(5):1234-124.
[16]	ALAKUIJALA J,FARRUGGIA A,FERRAGINA P,et al. Brotli:a general-purpose data compressor[J]. ACM Transactions on Information Systems,2018,37(1):1-30.
[17]	COLLET Y,KUCHERAWY M. Standard compression and the application/zstd media type[EB/OL]. [2023-10-02]. https://datatracker.ietf.org/doc/html/draft-kucherawy-dispatch-zstd-00.
[18]	SOWMYALAKSHMI R,WALY M I,SIKKANDAR M Y,et al. An optimal lempel ziv Markov based microarray image compression algorithm[J]. Computers,Materials & Continua,2021,69(2). DOI: 10.32604/cmc.2021.018636.
[19]	LOGUNLEKO K B,ADENIJI O D,LOGUNLEKO A M. A comparative study of symmetric cryptography mechanism on DES, AES and EB64 for information security[J]. International Journal of Scientific Research in Computer Science and Engineering,2020,8(1):45-51.
[20]	朱沙沙. 一种煤矿安全监控系统数据加密算法[J]. 计算机应用与软件,2020,37(11):324-327. doi: 10.3969/j.issn.1000-386x.2020.11.052 ZHU Shasha. A data encryption algorithm for coal mine safety monitoring system[J]. Computer Applications and Software,2020,37(11):324-327. doi: 10.3969/j.issn.1000-386x.2020.11.052