Volume 50 Issue 5
May  2024
Turn off MathJax
Article Contents
MA Aiqiang, YAO Wanqiang. Multi sensor adaptive fusion SLAM method for underground mobile robots in coal mines[J]. Journal of Mine Automation,2024,50(5):107-117.  doi: 10.13272/j.issn.1671-251x.2024050031
Citation: MA Aiqiang, YAO Wanqiang. Multi sensor adaptive fusion SLAM method for underground mobile robots in coal mines[J]. Journal of Mine Automation,2024,50(5):107-117.  doi: 10.13272/j.issn.1671-251x.2024050031

Multi sensor adaptive fusion SLAM method for underground mobile robots in coal mines

doi: 10.13272/j.issn.1671-251x.2024050031
  • Received Date: 2024-05-13
  • Rev Recd Date: 2024-05-31
  • Available Online: 2024-06-13
  • Mobile robots based on simultaneous localization and mapping (SLAM) technology can quickly, accurately, and automatically collect spatial data for spatial intelligent perception and environmental map construction. It is the key to achieving intelligent and unmanned coal mines. However, the current multi sensor fusion SLAM method in coal mines suffers from degradation and failure in robot front-end pose estimation, as well as insufficient precision in back-end fusion. This study proposes a LiDAR-visual-IMU adaptive fusion SLAM method for underground mobile robots in coal mines. The method clusters and segments LiDAR point cloud data, extracts line and surface features, and uses IMU pre integration state for distortion correction. The method uses image enhancement algorithm based on adaptive Gamma correction and contrast limited adaptive histogram equalization (CLAHE) to process low light images, and then extracts visual point and line features. The method provides initial pose values for LiDAR feature matching and visual feature tracking using IMU pre integration state. The pose of the mobile robot is obtained by matching the line and surface features of adjacent frames of LiDAR. Then, visual point and line feature tracking is performed to calculate the LiDAR, visual, and IMU pose changes. The stability of the front-end odometer is detected by setting dynamic thresholds, and the optimal pose is adaptively selected. The method constructs residuals for different sensors, including point cloud matching residuals, IMU pre integration residuals, visual point line residuals, and edge residuals. In order to balance precision and real-time performance, a sliding window based joint nonlinear optimization of multi-source data for laser point cloud features, visual features, and IMU measurements is implemented to achieve continuous and reliable SLAM in coal mines. Experimental verification is conducted on the effects before and after image enhancement. The results show that the image enhancement algorithm based on adaptive Gamma correction and CLAHE can significantly improve the brightness and contrast of the backlight and lighting areas, increase the feature information in the image, and significantly improve the quality of feature point extraction and matching. It achieves a matching success rate of 90.7%. To verify the performance of the proposed method, experimental verification is conducted in narrow corridor and coal mine roadway scenarios. The results show that the root mean square error of the proposed method in narrow corridor scenarios is 0.15 m, and the consistency of the constructed point cloud map is high. The root mean square error of positioning in the coal mine roadway scenario is 0.19 m. The constructed point cloud map can truly reflect the underground environment of the coal mine.

     

  • loading
  • [1]
    肖琳芬. 王国法院士:煤矿智能化技术体系建设进展与煤炭产业数字化转型[J]. 高科技与产业化,2024,30(2):12-15.

    XIAO Linfen. Academician Wang Guofa:progress in the construction of intelligent technology system in coal mines and the digital transformation of coal industry[J]. High-Technology & Commercialization,2024,30(2):12-15.
    [2]
    任满翊. 无人化智能煤矿建设探索与实践[J]. 工矿自动化,2022,48(增刊1):27-29.

    REN Manyi. Exploration and practice of unmanned intelligent coal mine construction[J]. Journal of Mine Automation,2022,48(S1):27-29.
    [3]
    KHATTAK S,NGUYEN H,MASCARICH F,et al. Complementary multi-modal sensor fusion for resilient robot pose estimation in subterranean environments[C]. International Conference on Unmanned Aircraft Systems,Athens,2020:1024-1029.
    [4]
    龚云,颉昕宇. 基于同态滤波方法的煤矿井下图像增强技术研究[J]. 煤炭科学技术,2023,51(3):241-250.

    GONG Yun,JIE Xinyu. Research on coal mine underground image recognition technology based on homomorphic filtering method[J]. Coal Science and Technology,2023,51(3):241-250.
    [5]
    YANG Xin,LIN Xiaohu,YANG Wanqiang,et al. A robust LiDAR SLAM method for underground coal mine robot with degenerated scene compensation[J]. Remote Sensing,2022,15(1). DOI: 10.3390/RS15010186.
    [6]
    WU Weitong,LI Jianping,CHEN Chi,et al. AFLI-Calib:robust LiDAR-IMU extrinsic self-calibration based on adaptive frame length LiDAR odometry[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2023,199:157-181. doi: 10.1016/j.isprsjprs.2023.04.004
    [7]
    COLE D M,NEWMAN P M. Using laser range data for 3D SLAM in outdoor environments[C]. IEEE International Conference on Robotics and Automation,Orlando,2006:1556-1563.
    [8]
    LI Menggang,ZHU Hua,YOU Shaoze,et al. Efficient laser-based 3D SLAM in real time for coal mine rescue robots[C]. 8th Annual International Conference on Technology in Automation,Control,and Intelligent Systems,Tianjin,2018:971-976.
    [9]
    KNEIP L,WEISS S,SIEGWART R. Deterministic initialization of metric state estimation filters for loosely-coupled monocular vision-inertial systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,San Francisco,2011:2235-2241.
    [10]
    BLOESCH M,BURRI M,OMARI S,et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research,2017,36(10):1053-1072. doi: 10.1177/0278364917728574
    [11]
    GOMEZ-OJEDA R,MORENO F A,ZUNIGA-NOëL D,et al. PL-SLAM:a stereo SLAM system through the combination of points and line segments[J]. IEEE Transactions on Robotics,2019,35(3):734-746. doi: 10.1109/TRO.2019.2899783
    [12]
    YANG Gaochao,WANG Qing,LIU Pengfei,et al. An improved monocular PL-SlAM method with point-line feature fusion under low-texture environment[C]. 4th International Conference on Control and Computer Vision,Macau,2021:119-125.
    [13]
    ZHAO Shibo,FANG Zheng,LI Haolai,et al. A robust laser-inertial odometry and mapping method for large-scale highway environments[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Macau,2019:1285-1292.
    [14]
    SHAN Tixiao,ENGLOT B,MEYERS D,et al. Lio-sam:tightly-coupled lidar inertial odometry via smoothing and mapping[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Las Vegas,2020:5135-5142.
    [15]
    ZHANG Ji,SINGH S. Visual-lidar odometry and mapping:low-drift,robust,and fast[C]. IEEE International Conference on Robotics and Automation,Seattle,2015:2174-2181.
    [16]
    LIU Yanqing,YANG Dongdong,LI Jiamao,et al. Stereo visual-inertial SLAM with points and lines[J]. IEEE Access,2018,6:69381-69392. doi: 10.1109/ACCESS.2018.2880689
    [17]
    TIAN Chengjun,LIU Haobo,LIU Zhe,et al. Research on multi-sensor fusion SLAM algorithm based on improved gmapping[J]. IEEE Access,2023,11(9):13690-13703.
    [18]
    SHAN Tixiao,ENGLOT B,RATTI C,et al. Lvi-sam:tightly-coupled lidar-visual-inertial odometry via smoothing and mapping[C]. IEEE International Conference on Robotics and Automation,Xi'an,2021:5692-5698.
    [19]
    隋心,王思语,罗力,等. 基于点云特征的改进RANSAC地面分割算法[J]. 导航定位学报,2024,12(1):106-114l.

    SUI Xin,WANG Siyu,LUO Li,et al. Improved RANSAC ground segmentation algorithm based on point cloud features[J]. Journal of Navigation and Positioning,2024,12(1):106-114.
    [20]
    HUANG S C,CHENG F C,CHIU Y S. Efficient contrast enhancement using adaptive gamma correction with weighting distribution[J]. IEEE Transactions on Image Processing,2013,22(3):1032-1041. doi: 10.1109/TIP.2012.2226047
    [21]
    WANG Jun,WANG Rui,WU Anwen. Improved gamma correction for visual slam in low-light scenes[C]. IEEE 3rd Advanced Information Management,Communicates,Electronic and Automation Control Conference,Chongqing,2019:1159-1163.
    [22]
    HUANG Lidong,ZHAO Wei,WANG Jun,et al. Combination of contrast limited adaptive histogram equalisation and discrete wavelet transform for image enhancement[J]. IET Image Processing,2015,9(10):908-915. doi: 10.1049/iet-ipr.2015.0150
    [23]
    MESSIKOMMER N,FANG C,GEHRIG M,et al. Data-driven feature tracking for event cameras[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:5642-5651.
    [24]
    GIOI R G,JAKUBOWICZ J,MOREL J M,et al. LSD:a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(4):722-732. doi: 10.1109/TPAMI.2008.300
    [25]
    ZHANG Lilian,KOCH R. An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J]. Journal of Visual Communication and Image Representation,2013,24(7):794-805. doi: 10.1016/j.jvcir.2013.05.006
    [26]
    LIU Zhe,SHI Dianxi,LI Ruihao,et al. PLC-VIO:visual-inertial odometry based on point-line constraints[J]. IEEE Transactions on Automation Science and Engineering,2022,19(3):1880-1897. doi: 10.1109/TASE.2021.3077026
    [27]
    TRIGGS B,MCLAUCHLAN P F,HARTLEY R I,et al. Bundle adjustment-a modern synthesis[C]. Vision Algorithms:Theory and Practic,1999. DOI: 10.1007/3-540-44480-7_21.
    [28]
    LI Ang,ZOU Danping,YU Wenxian. Robust initialization of multi-camera slam with limited view overlaps and inaccurate extrinsic calibration[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Pargue,2021:3361-3367.
    [29]
    FURGALE P,REHDER J,SIEGWART R. Unified temporal and spatial calibration for multi-sensor systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Tokyo,2013:1280-1286.
    [30]
    MAJEED S H,ISA N A M. Adaptive entropy index histogram equalization for poor contrast images[J]. IEEE Access,2020,9:6402-6437.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(10)  / Tables(3)

    Article Metrics

    Article views (120) PDF downloads(18) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return