留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于边云协同框架的煤矿井下实时视频处理系统

李敬兆,  秦晓伟,  汪磊

downloadPDF
文章导航 > 工矿自动化  > 2021 >  47(12): 1-7
李敬兆, 秦晓伟, 汪磊. 基于边云协同框架的煤矿井下实时视频处理系统[J]. 工矿自动化, 2021, 47(12): 1-7. doi: 10.13272/j.issn.1671-251x.2021070023
引用本文: 李敬兆, 秦晓伟, 汪磊. 基于边云协同框架的煤矿井下实时视频处理系统[J]. 工矿自动化, 2021, 47(12): 1-7. doi: 10.13272/j.issn.1671-251x.2021070023
shu
LI Jingzhao, QIN Xiaowei, WANG Lei. Real-time video processing system in coal mine based on edge-cloud collaborative framework[J]. Industry and Mine Automation, 2021, 47(12): 1-7. doi: 10.13272/j.issn.1671-251x.2021070023
Citation: LI Jingzhao, QIN Xiaowei, WANG Lei. Real-time video processing system in coal mine based on edge-cloud collaborative framework[J]. Industry and Mine Automation, 2021, 47(12): 1-7. doi: 10.13272/j.issn.1671-251x.2021070023
shu

基于边云协同框架的煤矿井下实时视频处理系统

doi: 10.13272/j.issn.1671-251x.2021070023

  • 安徽理工大学 电气与信息工程学院, 安徽 淮南 232001

基金项目: 

国家自然科学基金项目(51874010)；物联网关键技术研究创新团队项目(201950ZX003)。

详细信息
    作者简介:

    李敬兆(1964-)，男，安徽淮南人，教授，博士研究生导师，博士，主要研究方向为嵌入式系统、计算机视觉，E-mail:jzhli@aust.edu.cn。

    通讯作者:

    秦晓伟(1996-)，男，安徽滁州人，硕士研究生，主要研究方向为人工智能、图像处理，E-mail:lingxu0812@126.com。

  • 中图分类号: TD67

Real-time video processing system in coal mine based on edge-cloud collaborative framework

  • School of Electrical and Information Engineering, Anhui University of Science and Technology, Huainan 232001, China

    摘要
  • 摘要: 目前煤矿井下智能视频监控主要采用云计算方式处理实时视频，视频传输占用的网络资源多，时延高，无法实时响应监控区域发生的紧急事件。针对该问题，提出了基于边云协同框架的煤矿井下实时视频处理系统，将实时性强的目标识别任务下放至边缘端，将计算量大且实时性弱的边缘设备整合等任务放至云端处理。在视频监控现场，利用部署在边缘设备上的神经网络模型对视频监控图像进行本地处理；通过井下异构融合网络将不同网络环境中边缘设备的处理结果和模型参数等信息发送给云服务器；云服务器针对性地对各场景中的边缘设备进行模型更新、推送，最终实现边云数据实时交互和边缘端服务的在线优化。针对目标检测模型Tiny-YOLOv3无法提取到图片的深层特征、易出现梯度消失和过拟合现象等问题，依据残差结构设计了下采样残差模块，对Tiny-YOLOv3进行改进，以提高模型的深度特征提取和泛化能力。在边云数据交互的基础上，对边缘设备上的目标检测模型进行针对性场景优化，以提高边缘设备端模型检测的准确率。测试结果表明：改进型Tiny-YOLOv3模型的稳定性与数据泛化能力优于YOLO和Tiny-YOLOv3；经过单一场景的特化训练后，改进型Tiny-YOLOv3模型的目标识别更加精准；与云计算相比，边云协同框架可显著降低监控视频处理时延。

     

    Abstract: At present, the intelligent video monitoring in coal mine mainly adopts cloud computing to process real-time video, and the video transmission occupies large network resources and has high delay, which can not respond to emergency events in the monitoring area in real time. In order to solve this problem, a real-time video processing system in coal mine based on edge-cloud collaborative framework is proposed. In this system, the real-time target recognition task is sent to the edge, and the tasks with large calculation and weak real-time performance such as edge device integration are sent to the cloud for processing. At the video monitoring site, the neural network model deployed on the edge device is used to process the video monitoring image locally. Through the underground heterogeneous fusion network, the processing results and model parameters of the edge devices in different network environments are sent to the cloud server. The cloud server updates and pushes the model of edge devices in each scene, and finally realizes real-time interaction of edge-cloud data and online optimization of edge services. In order to solve the problems that Tiny-YOLOv3 can not extract the deep characteristic of the image, and is prone to gradient disappearance and over-fitting, a down-sampling residual module is designed according to the residual structure, and Tiny-YOLOv3 is improved to improve the deep characteristic extraction and generalization capability of the model. On the basis of edge-cloud data interaction, the target detection model on the edge device is optimized for the targeted scene to improve the accuracy of model detection on the edge device. The test results show that the stability and data generalization capability of the improved Tiny-YOLOv3 model are better than those of YOLO and Tiny-YOLOv3. After specialized training in a single scene, the improved Tiny-YOLOv3 model is more accurate in target recognition. Compared with cloud computing, the edge-cloud collaborative framework can reduce the latency of monitoring video processing significantly.

     

    • HTML全文
    • 参考文献(17)
  • [1] 张立亚.矿山智能视频分析与预警系统研究[J].工矿自动化,2017,43(11):16-20.

    ZHANG Liya.Research on intelligent video analysis and early warning system for mine[J].Industry and Mine Automation,2017,43(11):16-20.
    [2] 李现国,李斌,刘宗鹏,等.井下视频行人检测方法[J].工矿自动化,2020,46(2):54-58.

    LI Xianguo,LI Bin,LIU Zongpeng,et al.Underground video pedestrian detection method[J].Industry and Mine Automation,2020,46(2):54-58.
    [3] 谭章禄,吴琦,肖懿轩,等.智慧矿山信息可视化研究[J].工矿自动化,2020,46(1):26-31.

    TAN Zhanglu,WU Qi,XIAO Yixuan,et al.Research on information visualization of smart mine[J].Industry and Mine Automation,2020,46(1):26-31.
    [4] 姜德义,魏立科,王翀,等.智慧矿山边缘云协同计算技术架构与基础保障关键技术探讨[J].煤炭学报,2020,45(1):484-492.

    JIANG Deyi,WEI Like,WANG Chong,et al.Discussion on the technology architecture and key basic support technology for intelligent mine edge-cloud collaborative computing[J].Journal of China Coal Society,2020,45(1):484-492.
    [5] 陈思光,陈佳民,赵传信.基于深度强化学习的云边协同计算迁移研究[J].电子学报,2021,49(1):157-166.

    CHEN Siguang,CHEN Jiamin,ZHAO Chuanxin.Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism[J].Acta Electronica Sinica,2021,49(1):157-166.
    [6] 屈世甲,武福生.基于边缘计算的采煤工作面甲烷监测模式研究[J].煤炭科学技术,2020,48(12):161-167.

    QU Shijia,WU Fusheng.Research on methane monitoring mode of coal mining face based on edge computing[J].Coal Science and Technology,2020,48(12):161-167.
    [7] YE Lunqiang.Study on embedded system in monitoring of intelligent city pipeline network[J].Computer Communications,2020,153(4):451-458.
    [8] 邢姗姗,赵文龙.基于YOLO系列算法的复杂场景下无人机目标检测研究综述[J].计算机应用研究,2020,37(增刊2):28-30.

    XING Shanshan,ZHAO Wenlong.Review of UAV target detection in complex scenarios based on YOLO series algorithms[J].Application Research of Computers,2020, 37(S2):28-30.
    [9] HAN B G,LEE J G,LIM K T,et al.Design of a scalable and fast YOLO for edge-computing devices[J].Sensors,2020,20(23):6779-6790.
    [10] KIM W,CHO H,KIM J,et al.YOLO-based simultaneous target detection and classification in automotive FMCW radar systems[J].Sensors,2020,20(10):2897-2906.
    [11] 张斌,苏学贵,段振雄,等.YOLOv2在煤岩智能识别与定位中的应用研究[J].采矿与岩层控制工程学报,2020,2(2):94-101.

    ZHANG Bin,SU Xuegui,DUAN Zhenxiong,et al.Application of YOLOv2 in intelligent recognition and location of coal and rock[J].Journal of Mining and Strata Control Engineering,2020,2(2):94-101.
    [12] SHAO Zhenfeng,WANG Linggang,WANG Zhongyuan,et al.Saliency-aware convolution neural network for ship detection in surveillance video[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,30(3):781-794.
    [13] 施辉,陈先桥,杨英.改进YOLO v3的安全帽佩戴检测方法[J].计算机工程与应用,2019,55(11):213-220.

    SHI Hui,CHEN Xianqiao,YANG Ying.Safety helmet wearing detection method of improved YOLO v3[J].Computer Engineering and Applications,2019,55(11):213-220.
    [14] 颜贝,张礼,张建林,等.基于残差结构的对抗式网络图像生成方法[J].激光与光电子学进展,2020,57(18):310-317.

    YAN Bei,ZHANG Li,ZHANG Jianlin,et al.Image generation method for adversarial network based on residual structure[J].Laser & Optoelectronics Progress,2020,57(18):310-317.
    [15] 赵丽萍,袁霄,祝承,等.面向图像分类的残差网络进展研究[J].计算机工程与应用,2020,56(20):9-19.

    ZHAO Liping,YUAN Xiao,ZHU Cheng,et al.Research on residual networks for image classification[J].Computer Engineering and Applications,2020,56(20):9-19.
    [16] LI M,AO Y,PENG W,et al.Research of status recognition of fiber transfer box based on machine vision and deep learning[J].Multimedia Tools and Applications,2020,79(39):28695-28709.
    [17] LEE D,SUN Y G,KIM S H,et al.Transfer learning-based object detection algorithm using YOLO network[J].The Journal of the Institute of Internet,Broadcasting and Communication,2020,20(1):219-223.
    • 相关文章
    • 施引文献
  • 资源附件(0)
  • 加载中
WeChat 点击查看大图
计量
  • 文章访问数:  323
  • HTML全文浏览量:  26
  • PDF下载量:  45
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-07-07
  • 修回日期:  2021-11-29
  • 刊出日期:  2021-12-20

目录

    /

    返回文章
    返回

    苏ICP备 05016349号-2

    本系统由 北京仁和汇智信息技术有限公司 开发 访问量：     百度统计