Volume 49 Issue 5
May  2023
Turn off MathJax
Article Contents
Abstract
References
Related Articles
REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation，2023，49(5)：59-65. DOI: 10.13272/j.issn.1671-251x.18076
Citation: REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation，2023，49(5)：59-65. DOI: 10.13272/j.issn.1671-251x.18076
shu

A fully mechanized working face inspection system based on SLAM and virtual reality

  • Beijing Tianma Intelligent Control Technology Co., Ltd., Beijing 101399, China

More Information
  • Received Date: February 06, 2023
  • Revised Date: May 14, 2023
  • Available Online: May 23, 2023
    Graphical Abstract
    • Abstract
  • The reliability of the inspection robot in the fully mechanized working face is low due to the lack of scale information. In order to solve the above problem, virtual reality (VR) technology is introduced into the inspection of fully mechanized working face. A fully mechanized working face inspection system based on simultaneous localization and mapping (SLAM) and VR is designed. The system includes two parts: an inspection robot subsystem located underground and a VR real-time rendering subsystem located on the ground. The inspection robot subsystem utilizes laser SLAM technology to achieve real-time 3D scanning and establish a 3D map. At the same time, a panoramic camera is used to capture the scene of the fully mechanized working face in real-time. The real-time obtained laser point cloud and panoramic video are transmitted to the VR real-time rendering subsystem. The VR real-time rendering subsystem uses GPU acceleration technology to color laser point clouds. By customizing the rendering part of the Unreal 3D engine, real-time rendering of the laser point cloud is achieved, and the laser point cloud is projected onto the VR glasses. Remote operators obtain real-time 3D scenes through VR glasses, and remotely control the movements of the inspection robot through the operating handle. The fully mechanized working face inspection based on the first perspective is achieved. The underground industrial test results show that the system can achieve free switching of perspectives and zoom in on the scene. It enables better observation of details, with higher accuracy and reliability. By using GPU acceleration technology for point cloud coloring, the processing time is significantly shorter than CPU processing time. GPU has higher real-time performance, and the entire system's latency can meet the requirements of inspection tasks.
    • FullText(HTML)
    • References (22)
  • [1]
    王国法,范京道,徐亚军,等. 煤炭智能化开采关键技术创新进展与展望[J]. 工矿自动化,2018,44(2):5-12.

    WANG Guofa,FAN Jingdao,XU Yajun,et al. Innovation progress and prospect on key technologies of intelligent coal mining[J]. Industry and Mine Automation,2018,44(2):5-12.
    [2]
    张守祥,张学亮,张磊,等. 综采巡检机器人关键技术研究[J]. 煤炭科学技术,2022,50(1):247-255.

    ZHANG Shouxiang,ZHANG Xueliang,ZHANG Lei,et al. Research on key technology of patrol robot in fully-mechanized mining face[J]. Coal Science and Technology,2022,50(1):247-255.
    [3]
    梁占泽. 矿用带式输送机巡检机器人驱动系统设计[J]. 工矿自动化,2021,47(4):108-112. DOI: 10.13272/j.issn.1671-251x.2021010003

    LIANG Zhanze. Driving system design of inspection robot for mine belt conveyor[J]. Industry and Mine Automation,2021,47(4):108-112. DOI: 10.13272/j.issn.1671-251x.2021010003
    [4]
    毛浩,薛忠新,范生军,等. 张家峁煤矿回风巷道智能巡检机器人系统[J]. 煤矿安全,2021,52(7):107-111. DOI: 10.13347/j.cnki.mkaq.2021.07.019

    MAO Hao,XUE Zhongxin,FAN Shengjun,et al. Intelligent inspection robot system for return air roadway in Zhangjiamao Coal Mine[J]. Safety in Coal Mines,2021,52(7):107-111. DOI: 10.13347/j.cnki.mkaq.2021.07.019
    [5]
    商德勇,崔栓伟,周丹,等. 薄煤层巡检机器人行走机构跨沟性能分析及试验研究[J]. 煤炭工程,2017,49(4):136-138.

    SHANG Deyong,CUI Shuanwei,ZHOU Dan,et al. Ditch-crossing performance analysis and experimental study on inspection robot walking mechanism in thin coal seam[J]. Coal Engineering,2017,49(4):136-138.
    [6]
    张树生,马静雅,岑强,等. 煤矿综采工作面巡检机器人系统研究[J]. 煤炭科学技术,2019,47(10):136-140.

    ZHANG Shusheng,MA Jingya,CEN Qiang,et al. Research on inspection robot system for fully-mechanized mining face in coal mine[J]. Coal Science and Technology,2019,47(10):136-140.
    [7]
    李森,王峰,刘帅,等. 综采工作面巡检机器人关键技术研究[J]. 煤炭科学技术,2020,48(7):218-225.

    LI Sen,WANG Feng,LIU Shuai,et al. Study on key technology of patrol robots for fully-mechanized mining face[J]. Coal Science and Technology,2020,48(7):218-225.
    [8]
    郝勇,袁智. 综采工作面自动巡检机器人系统设计[J]. 煤炭科学技术,2020,48(8):145-149. DOI: 10.13199/j.cnki.cst.2020.08.018

    HAO Yong,YUAN Zhi. Design of automatic inspection robot system for fully-mechanized coal mining face[J]. Coal Science and Technology,2020,48(8):145-149. DOI: 10.13199/j.cnki.cst.2020.08.018
    [9]
    YASHIN G A, TRINITATOVA D, AGISHEV R T, et al. Aerovr: virtual reality-based teleoperation with tactile feedback for aerial manipulation[C]. The 19th International Conference on Advanced Robotics, Belo Horizonte, 2019: 767-772.
    [10]
    BARENTINE C, MCNAY A, PFAFFENBICHLER R, et al. A VR teleoperation suite with manipulation assist[C]. The 16th ACM/IEEE International Conference on Human-Robot Interaction, Boulder, 2021: 442-446.
    [11]
    GUZZI J, ABBATE G, PAOLILLO A, et al. Interacting with a conveyor belt in virtual reality using pointing gestures[C]. The 17th ACM/IEEE International Conference on Human-Robot Interaction, Sapporo, 2022: 1194-1195.
    [12]
    KENNEL-MAUSHART F, PORANNE R, COROS S. Multi-arm payload manipulation via mixed reality[C]. International Conference on Robotics and Automation, Philadelphia, 2022. DOI: 10.1109/ICRA46639.2022.9811580.
    [13]
    KUO C-Y, HUANG C-C, TSAI C-H, et al. Development of an immersive SLAM-based VR system for teleoperation of a mobile manipulator in an unknown environment[J]. Computers in Industry, 2021, 132. DOI: 10.1016/j.compind.2021.103502.
    [14]
    WONSICK M, KELEȘTEMUR T, ALT S, et al. Telemanipulation via virtual reality interfaces with enhanced environment models[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, Prague, 2021: 2999-3004.
    [15]
    迟焕磊,袁智,曹琰,等. 基于数字孪生的智能化工作面三维监测技术研究[J]. 煤炭科学技术,2021,49(10):153-161.

    CHI Huanlei,YUAN Zhi,CAO Yan,et al. Study on digital twin-based smart fully-mechanized coal mining workface monitoring technology[J]. Coal Science and Technology,2021,49(10):153-161.
    [16]
    李首滨,李森,张守祥,等. 综采工作面智能感知与智能控制关键技术与应用[J]. 煤炭科学技术,2021,49(4):28-39. DOI: 10.13199/j.cnki.cst.2021.04.004

    LI Shoubin,LI Sen,ZHANG Shouxiang,et al. Key technology and application of intelligent perception and intelligent control in fully mechanized mining face[J]. Coal Science and Technology,2021,49(4):28-39. DOI: 10.13199/j.cnki.cst.2021.04.004
    [17]
    杨生华,周永昌,芮丰,等. 薄煤层开采与成套装备技术的发展趋势[J]. 煤炭科学技术,2020,48(3):49-58. DOI: 10.13199/j.cnki.cst.2020.03.003

    YANG Shenghua,ZHOU Yongchang,RUI Feng,et al. Development trend of thin coal seam mining and complete equipment technology[J]. Coal Science and Technology,2020,48(3):49-58. DOI: 10.13199/j.cnki.cst.2020.03.003
    [18]
    亓玉浩,关士远. 基于激光SLAM的综采工作面实时三维建图方法[J]. 工矿自动化,2022,48(11):139-144. DOI: 10.13272/j.issn.1671-251x.2022060047

    QI Yuhao,GUAN Shiyuan. Real-time 3D mapping method of fully mechanized working face based on laser SLAM[J]. Journal of Mine Automation,2022,48(11):139-144. DOI: 10.13272/j.issn.1671-251x.2022060047
    [19]
    任伟. 综采工作面多目全景摄像仪的研制与应用[J]. 煤炭工程,2022,54(12):102-108.

    REN Wei. Development and application of multi view panoramic camera in fully mechanized mining face[J]. Coal Engineering,2022,54(12):102-108.
    [20]
    CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. AMOEBA: a coarse grained reconfigurable architecture for dynamic GPU scaling[C]. The 34th ACM International Conference on Supercomputing, 2020: 1-13. DOI: 10.1145/3392717.3392738.
    [21]
    CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. Alleviating bottlenecks for DNN execution on GPUs via opportunistic computing[C]. The 21st International Symposium on Quality Electronic Design, Santa Clara, 2020. DOI: 10.1109/ISQED48828.2020.9136967.
    [22]
    KIRK D B, HWU W-M W. Programming massively parallel processors[M]. 3rd ed. San Francisco: Morgan Kaufmann, 2016.
    • Related Articles

  • Related Articles

    [1]YOU Qinghua, WU Peng, MA Bo. Acoustic signal enhancement method for belt conveyor idler bearings[J]. Journal of Mine Automation, 2025, 51(3): 70-77. DOI: 10.13272/j.issn.1671-251x.2024120083
    [2]LI Xia, CUI Jingcheng. Design of intelligent fire prevention and control system outside underground coal mine[J]. Journal of Mine Automation, 2024, 50(S1): 100-103.
    [3]WANG Shubin, WANG Xu, YAN Shiping, WANG Ke. Transformer based time series prediction method for mine internal caused fire[J]. Journal of Mine Automation, 2024, 50(3): 65-70, 91. DOI: 10.13272/j.issn.1671-251x.2023100084
    [4]WU Guoping. Fault detection method for belt conveyor idler[J]. Journal of Mine Automation, 2023, 49(2): 149-156. DOI: 10.13272/j.issn.1671-251x.2022100022
    [5]ZHAO Duan, LI Tao, DONG Yanqiang, WANG Zhiqiang, LIU Chun. Coal mine external fire detection method based on edge intelligence[J]. Journal of Mine Automation, 2022, 48(12): 108-115. DOI: 10.13272/j.issn.1671-251x.2022080046
    [6]CHEN Xiaolin. Staged multi-index comprehensive evaluation method for fire risk of coal working face[J]. Journal of Mine Automation, 2022, 48(7): 90-95, 104. DOI: 10.13272/j.issn.1671-251x.2021120083
    [7]SUN Jiping, LI Yue. Binocular vision-based perception and positioning method of mine external fire[J]. Journal of Mine Automation, 2021, 47(6): 12-16. DOI: 10.13272/j.issn.1671-251x.17766
    [8]SUN Jiping, CUI Jiawei. Mine external fire sensing method[J]. Journal of Mine Automation, 2021, 47(4): 1-5. DOI: 10.13272/j.issn.1671-251x.17760
    [9]SUN Jiping, SUN Yanyu, FAN Weiqiang. Mine exogenous fire identification method based on visible light and infrared image[J]. Journal of Mine Automation, 2019, 45(5): 1-5. DOI: 10.13272/j.issn.1671-251x.17435
    [10]WANG Yuanbin, MA Xianmi. Research of early prediction method for exogenous fire in coal mine[J]. Journal of Mine Automation, 2015, 41(9): 63-66. DOI: 10.13272/j.issn.1671-251x.2015.09.016

Catalog

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (243) PDF downloads (52) Cited by()
    Related

    苏ICP备 05016349号-2

    Supported by: Beijing Renhe Information Technology Co., Ltd. Visits：1171867    Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return