留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于SLAM和虚拟现实的综采工作面巡检系统

任伟

任伟. 基于SLAM和虚拟现实的综采工作面巡检系统[J]. 工矿自动化,2023,49(5):59-65.  doi: 10.13272/j.issn.1671-251x.18076
引用本文: 任伟. 基于SLAM和虚拟现实的综采工作面巡检系统[J]. 工矿自动化,2023,49(5):59-65.  doi: 10.13272/j.issn.1671-251x.18076
REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation,2023,49(5):59-65.  doi: 10.13272/j.issn.1671-251x.18076
Citation: REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation,2023,49(5):59-65.  doi: 10.13272/j.issn.1671-251x.18076

基于SLAM和虚拟现实的综采工作面巡检系统

doi: 10.13272/j.issn.1671-251x.18076
基金项目: 山东省重大科技创新工程项目(2019SDZY01);国家能源集团2021年度重点项目(GJNY-21-25)。
详细信息
    作者简介:

    任伟(1979—),男,陕西扶风人,副研究员,硕士,主要从事智能化开采研究工作,E-mail:renwei@tdmarco.com

  • 中图分类号: TD67/82

A fully mechanized working face inspection system based on SLAM and virtual reality

  • 摘要: 针对综采工作面巡检机器人由于缺乏尺度信息导致可靠性较低的问题,将虚拟现实(VR)技术引入综采工作面巡检中,设计了基于即时定位与地图构建(SLAM)和VR的综采工作面巡检系统。该系统包括位于井下的巡检机器人子系统和位于地面的VR实时渲染子系统2个部分。巡检机器人子系统利用激光SLAM技术实现实时三维扫描,并建立三维地图,同时利用全景相机实时捕获综采工作面的场景,将实时获取的激光点云及全景视频传输到VR实时渲染子系统。VR实时渲染子系统采用GPU加速技术对激光点云进行着色,通过对Unreal三维引擎渲染部分进行定制化开发,实现对激光点云的实时渲染,并将激光点云投屏到VR眼镜上。远程操作人员通过VR眼镜实时获取现场三维场景,通过操作手柄远程控制巡检机器人动作,从而实现基于第一视角的综采工作面巡检。井下工业性试验结果表明,该系统可实现视角自由切换,并对场景进行放大,从而能够更好地观察到细节部分,精确性和可靠性更高;采用GPU加速技术进行点云着色,处理时间明显比CPU处理时间短,GPU实时性更高,整个系统的延时能满足巡检任务需求。

     

  • 图  1  基于SLAM和VR的综采工作面巡检系统结构

    Figure  1.  Structure of fully mechanized working face inspection system based on SLAM and VR

    图  2  巡检机器人子系统硬件

    Figure  2.  Hardware of inspection robot subsystem

    图  3  全景激光雷达硬件构成

    Figure  3.  Hardware composition of panoramic LiDAR

    图  4  全景扫描

    Figure  4.  Panoramic scan

    图  5  scan to sweep配准

    Figure  5.  Scan to sweep registration

    图  6  球幕坐标与经纬图坐标的变换

    Figure  6.  The transformation of spherical screen coordinates and latitude map coordinates

    图  7  VR设备

    Figure  7.  Virtual reality device

    图  8  激光点云着色

    Figure  8.  Laser point cloud coloring

    图  9  工作面现场VR景象

    Figure  9.  VR scene of the working face on-site

    图  10  工作面放大场景

    Figure  10.  Enlarged scene of the working face

    表  1  CPU与GPU处理时间对比

    Table  1.   Comparison of CPU and GPU processing time

    scan序号12345
    CPU处理
    时间/s
    121127200167134
    GPU处理
    时间/s
    0.50.450.30.390.36
    下载: 导出CSV

    表  2  系统延时

    Table  2.   System latency

    scan序号12345
    系统延时/s0.900.920.800.980.87
    下载: 导出CSV
  • [1] 王国法,范京道,徐亚军,等. 煤炭智能化开采关键技术创新进展与展望[J]. 工矿自动化,2018,44(2):5-12.

    WANG Guofa,FAN Jingdao,XU Yajun,et al. Innovation progress and prospect on key technologies of intelligent coal mining[J]. Industry and Mine Automation,2018,44(2):5-12.
    [2] 张守祥,张学亮,张磊,等. 综采巡检机器人关键技术研究[J]. 煤炭科学技术,2022,50(1):247-255.

    ZHANG Shouxiang,ZHANG Xueliang,ZHANG Lei,et al. Research on key technology of patrol robot in fully-mechanized mining face[J]. Coal Science and Technology,2022,50(1):247-255.
    [3] 梁占泽. 矿用带式输送机巡检机器人驱动系统设计[J]. 工矿自动化,2021,47(4):108-112. doi: 10.13272/j.issn.1671-251x.2021010003

    LIANG Zhanze. Driving system design of inspection robot for mine belt conveyor[J]. Industry and Mine Automation,2021,47(4):108-112. doi: 10.13272/j.issn.1671-251x.2021010003
    [4] 毛浩,薛忠新,范生军,等. 张家峁煤矿回风巷道智能巡检机器人系统[J]. 煤矿安全,2021,52(7):107-111. doi: 10.13347/j.cnki.mkaq.2021.07.019

    MAO Hao,XUE Zhongxin,FAN Shengjun,et al. Intelligent inspection robot system for return air roadway in Zhangjiamao Coal Mine[J]. Safety in Coal Mines,2021,52(7):107-111. doi: 10.13347/j.cnki.mkaq.2021.07.019
    [5] 商德勇,崔栓伟,周丹,等. 薄煤层巡检机器人行走机构跨沟性能分析及试验研究[J]. 煤炭工程,2017,49(4):136-138.

    SHANG Deyong,CUI Shuanwei,ZHOU Dan,et al. Ditch-crossing performance analysis and experimental study on inspection robot walking mechanism in thin coal seam[J]. Coal Engineering,2017,49(4):136-138.
    [6] 张树生,马静雅,岑强,等. 煤矿综采工作面巡检机器人系统研究[J]. 煤炭科学技术,2019,47(10):136-140.

    ZHANG Shusheng,MA Jingya,CEN Qiang,et al. Research on inspection robot system for fully-mechanized mining face in coal mine[J]. Coal Science and Technology,2019,47(10):136-140.
    [7] 李森,王峰,刘帅,等. 综采工作面巡检机器人关键技术研究[J]. 煤炭科学技术,2020,48(7):218-225.

    LI Sen,WANG Feng,LIU Shuai,et al. Study on key technology of patrol robots for fully-mechanized mining face[J]. Coal Science and Technology,2020,48(7):218-225.
    [8] 郝勇,袁智. 综采工作面自动巡检机器人系统设计[J]. 煤炭科学技术,2020,48(8):145-149. doi: 10.13199/j.cnki.cst.2020.08.018

    HAO Yong,YUAN Zhi. Design of automatic inspection robot system for fully-mechanized coal mining face[J]. Coal Science and Technology,2020,48(8):145-149. doi: 10.13199/j.cnki.cst.2020.08.018
    [9] YASHIN G A, TRINITATOVA D, AGISHEV R T, et al. Aerovr: virtual reality-based teleoperation with tactile feedback for aerial manipulation[C]. The 19th International Conference on Advanced Robotics, Belo Horizonte, 2019: 767-772.
    [10] BARENTINE C, MCNAY A, PFAFFENBICHLER R, et al. A VR teleoperation suite with manipulation assist[C]. The 16th ACM/IEEE International Conference on Human-Robot Interaction, Boulder, 2021: 442-446.
    [11] GUZZI J, ABBATE G, PAOLILLO A, et al. Interacting with a conveyor belt in virtual reality using pointing gestures[C]. The 17th ACM/IEEE International Conference on Human-Robot Interaction, Sapporo, 2022: 1194-1195.
    [12] KENNEL-MAUSHART F, PORANNE R, COROS S. Multi-arm payload manipulation via mixed reality[C]. International Conference on Robotics and Automation, Philadelphia, 2022. DOI: 10.1109/ICRA46639.2022.9811580.
    [13] KUO C-Y, HUANG C-C, TSAI C-H, et al. Development of an immersive SLAM-based VR system for teleoperation of a mobile manipulator in an unknown environment[J]. Computers in Industry, 2021, 132. DOI: 10.1016/j.compind.2021.103502.
    [14] WONSICK M, KELEȘTEMUR T, ALT S, et al. Telemanipulation via virtual reality interfaces with enhanced environment models[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, Prague, 2021: 2999-3004.
    [15] 迟焕磊,袁智,曹琰,等. 基于数字孪生的智能化工作面三维监测技术研究[J]. 煤炭科学技术,2021,49(10):153-161.

    CHI Huanlei,YUAN Zhi,CAO Yan,et al. Study on digital twin-based smart fully-mechanized coal mining workface monitoring technology[J]. Coal Science and Technology,2021,49(10):153-161.
    [16] 李首滨,李森,张守祥,等. 综采工作面智能感知与智能控制关键技术与应用[J]. 煤炭科学技术,2021,49(4):28-39. doi: 10.13199/j.cnki.cst.2021.04.004

    LI Shoubin,LI Sen,ZHANG Shouxiang,et al. Key technology and application of intelligent perception and intelligent control in fully mechanized mining face[J]. Coal Science and Technology,2021,49(4):28-39. doi: 10.13199/j.cnki.cst.2021.04.004
    [17] 杨生华,周永昌,芮丰,等. 薄煤层开采与成套装备技术的发展趋势[J]. 煤炭科学技术,2020,48(3):49-58. doi: 10.13199/j.cnki.cst.2020.03.003

    YANG Shenghua,ZHOU Yongchang,RUI Feng,et al. Development trend of thin coal seam mining and complete equipment technology[J]. Coal Science and Technology,2020,48(3):49-58. doi: 10.13199/j.cnki.cst.2020.03.003
    [18] 亓玉浩,关士远. 基于激光SLAM的综采工作面实时三维建图方法[J]. 工矿自动化,2022,48(11):139-144. doi: 10.13272/j.issn.1671-251x.2022060047

    QI Yuhao,GUAN Shiyuan. Real-time 3D mapping method of fully mechanized working face based on laser SLAM[J]. Journal of Mine Automation,2022,48(11):139-144. doi: 10.13272/j.issn.1671-251x.2022060047
    [19] 任伟. 综采工作面多目全景摄像仪的研制与应用[J]. 煤炭工程,2022,54(12):102-108.

    REN Wei. Development and application of multi view panoramic camera in fully mechanized mining face[J]. Coal Engineering,2022,54(12):102-108.
    [20] CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. AMOEBA: a coarse grained reconfigurable architecture for dynamic GPU scaling[C]. The 34th ACM International Conference on Supercomputing, 2020: 1-13. DOI: 10.1145/3392717.3392738.
    [21] CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. Alleviating bottlenecks for DNN execution on GPUs via opportunistic computing[C]. The 21st International Symposium on Quality Electronic Design, Santa Clara, 2020. DOI: 10.1109/ISQED48828.2020.9136967.
    [22] KIRK D B, HWU W-M W. Programming massively parallel processors[M]. 3rd ed. San Francisco: Morgan Kaufmann, 2016.
  • 加载中
图(10) / 表(2)
计量
  • 文章访问数:  224
  • HTML全文浏览量:  37
  • PDF下载量:  34
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-02-07
  • 修回日期:  2023-05-15
  • 网络出版日期:  2023-05-24

目录

    /

    返回文章
    返回