Multi sensor adaptive fusion SLAM method for underground mobile robots in coal mines
-
摘要: 基于同时定位与建图（SLAM）技术的移动机器人能够快速、准确、自动化地采集空间数据，进行空间智能感知和环境地图构建，是实现煤矿智能化和无人化的关键。针对目前煤矿井下多传感器融合SLAM方法存在机器人前端位姿估计退化失效和后端融合精度不足的问题，提出了一种煤矿井下移动机器人激光雷达（LiDAR）−视觉−惯性（IMU）自适应融合SLAM方法。对LiDAR点云数据进行聚类分割，提取线面特征，利用IMU预积分状态进行畸变校正，采用基于自适应Gamma校正和对比度受限的自适应直方图均衡化（CLAHE）的图像增强算法处理低照度图像，再提取视觉点线特征。用IMU预积分状态为LiDAR特征匹配与视觉特征跟踪提供位姿初始值。根据LiDAR相邻帧的线面特征匹配得到移动机器人位姿，之后进行视觉点线特征跟踪，分别计算LiDAR、视觉、IMU位姿变化值，通过设定动态阈值来检测前端里程计的稳定性，自适应选取最优位姿。对不同传感器构建残差项，包括点云匹配残差、IMU预积分残差、视觉点线残差、边缘化残差。为了兼顾精度与实时性，基于滑动窗口实现激光点云特征、视觉特征、IMU测量的多源数据联合非线性优化，实现煤矿井下连续可用、精确可靠的SLAM。对图像增强前后效果进行试验验证，结果表明，基于自适应Gamma校正和CLAHE的图像增强算法能显著提升背光区和光照区的亮度和对比度，增加图像中的特征信息，大幅提升特征点提取数量和匹配质量，匹配成功率达90.7%。为验证所提方法的性能，在狭长走廊和煤矿巷道场景下进行试验验证，结果表明，所提方法在狭长走廊场景的定位均方根误差为0.15 m，构建的点云地图一致性较高；在煤矿巷道场景中的定位均方根误差为0.19 m，构建的点云地图可真实地反映煤矿井下环境。Abstract: Mobile robots based on simultaneous localization and mapping (SLAM) technology can quickly, accurately, and automatically collect spatial data for spatial intelligent perception and environmental map construction. It is the key to achieving intelligent and unmanned coal mines. However, the current multi sensor fusion SLAM method in coal mines suffers from degradation and failure in robot front-end pose estimation, as well as insufficient precision in back-end fusion. This study proposes a LiDAR-visual-IMU adaptive fusion SLAM method for underground mobile robots in coal mines. The method clusters and segments LiDAR point cloud data, extracts line and surface features, and uses IMU pre integration state for distortion correction. The method uses image enhancement algorithm based on adaptive Gamma correction and contrast limited adaptive histogram equalization (CLAHE) to process low light images, and then extracts visual point and line features. The method provides initial pose values for LiDAR feature matching and visual feature tracking using IMU pre integration state. The pose of the mobile robot is obtained by matching the line and surface features of adjacent frames of LiDAR. Then, visual point and line feature tracking is performed to calculate the LiDAR, visual, and IMU pose changes. The stability of the front-end odometer is detected by setting dynamic thresholds, and the optimal pose is adaptively selected. The method constructs residuals for different sensors, including point cloud matching residuals, IMU pre integration residuals, visual point line residuals, and edge residuals. In order to balance precision and real-time performance, a sliding window based joint nonlinear optimization of multi-source data for laser point cloud features, visual features, and IMU measurements is implemented to achieve continuous and reliable SLAM in coal mines. Experimental verification is conducted on the effects before and after image enhancement. The results show that the image enhancement algorithm based on adaptive Gamma correction and CLAHE can significantly improve the brightness and contrast of the backlight and lighting areas, increase the feature information in the image, and significantly improve the quality of feature point extraction and matching. It achieves a matching success rate of 90.7%. To verify the performance of the proposed method, experimental verification is conducted in narrow corridor and coal mine roadway scenarios. The results show that the root mean square error of the proposed method in narrow corridor scenarios is 0.15 m, and the consistency of the constructed point cloud map is high. The root mean square error of positioning in the coal mine roadway scenario is 0.19 m. The constructed point cloud map can truly reflect the underground environment of the coal mine.
-
图 1 LiDAR−视觉−惯性融合SLAM方法原理
Figure 1. Principle of LiDAR-visual -inertial measurement unit (IMU) fusion simultaneous localization and mapping (SLAM) method
图 5 狭长走廊与煤矿巷道环境及控制点布设情况
Figure 5. Control points layout in narrow corridor and coal mine roadway environment
表 1 多源传感器设备型号与信息
Table 1. Sensors model and information
设备 设备型号 设备信息 LiDAR VLP−16 频率：10 Hz ；最大测量距离：150 m IMU Ellipse2−N 频率：200 Hz ；横滚/俯仰误差为±0.1°；航向误差为0.5° 相机 Zed−2i 频率：30 Hz； 图像分辨率（H×V）：1 280×720 控制台 Autolabor−PC CPU：AMD Ryzen3 3 200G、DDR4 8 G 机器人 Autolabor−Pro1 驱动模式：四驱；位移速度：0.5～1.5 m/s；旋转角速度：0.56 rad/s 
下载: 导出CSV
表 2 特征提取与跟踪对比
Table 2. Features extraction and tracking comparison
算法 特征点数量/个 匹配数量/个 匹配成功率% M7 M9 M7 M9 M7 M9 原图 273 308 89 122 32.6 39.6 Retinex算法 368 390 242 269 65.8 69.0 自适应Gamma校正算法 330 367 250 286 75.8 77.9 CLAHE算法 329 356 276 303 83.9 85.1 文献[30]算法 347 374 297 331 85.6 88.5 本文算法 365 392 331 365 90.7 93.1
下载: 导出CSV
表 3 定位轨迹的标准差和均方根误差
Table 3. Root mean squared error and standard deviation of positioning trajectory
m 误差 狭长走廊 煤矿巷道 本文方法 IE−LVI−SAM LIO−Mapping IE−ORB−SLAM2 ORB−SLAM2 本文方法 IE−LVI−SAM LIO−Mapping IE−ORB−SLAM2 ORB−SLAM2 STD 0.06 0.27 0.78 0.89 3.39 0.08 0.10 0.24 0.57 4.05 RMSE 0.15 0.39 1.25 2.58 8.12 0.19 0.28 0.75 0.60 6.82
下载: 导出CSV
-
[1] 肖琳芬. 王国法院士:煤矿智能化技术体系建设进展与煤炭产业数字化转型[J]. 高科技与产业化,2024,30(2):12-15.XIAO Linfen. Academician Wang Guofa:progress in the construction of intelligent technology system in coal mines and the digital transformation of coal industry[J]. High-Technology & Commercialization,2024,30(2):12-15. [2] 任满翊. 无人化智能煤矿建设探索与实践[J]. 工矿自动化,2022,48(增刊1):27-29.REN Manyi. Exploration and practice of unmanned intelligent coal mine construction[J]. Journal of Mine Automation,2022,48(S1):27-29. [3] KHATTAK S,NGUYEN H,MASCARICH F,et al. Complementary multi-modal sensor fusion for resilient robot pose estimation in subterranean environments[C]. International Conference on Unmanned Aircraft Systems,Athens,2020:1024-1029. [4] 龚云,颉昕宇. 基于同态滤波方法的煤矿井下图像增强技术研究[J]. 煤炭科学技术,2023,51(3):241-250.GONG Yun,JIE Xinyu. Research on coal mine underground image recognition technology based on homomorphic filtering method[J]. Coal Science and Technology,2023,51(3):241-250. [5] YANG Xin,LIN Xiaohu,YANG Wanqiang,et al. A robust LiDAR SLAM method for underground coal mine robot with degenerated scene compensation[J]. Remote Sensing,2022,15(1). DOI: 10.3390/RS15010186. [6] WU Weitong,LI Jianping,CHEN Chi,et al. AFLI-Calib:robust LiDAR-IMU extrinsic self-calibration based on adaptive frame length LiDAR odometry[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2023,199:157-181. doi: 10.1016/j.isprsjprs.2023.04.004 [7] COLE D M,NEWMAN P M. Using laser range data for 3D SLAM in outdoor environments[C]. IEEE International Conference on Robotics and Automation,Orlando,2006:1556-1563. [8] LI Menggang,ZHU Hua,YOU Shaoze,et al. Efficient laser-based 3D SLAM in real time for coal mine rescue robots[C]. 8th Annual International Conference on Technology in Automation,Control,and Intelligent Systems,Tianjin,2018:971-976. [9] KNEIP L,WEISS S,SIEGWART R. Deterministic initialization of metric state estimation filters for loosely-coupled monocular vision-inertial systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,San Francisco,2011:2235-2241. [10] BLOESCH M,BURRI M,OMARI S,et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research,2017,36(10):1053-1072. doi: 10.1177/0278364917728574 [11] GOMEZ-OJEDA R,MORENO F A,ZUNIGA-NOëL D,et al. PL-SLAM:a stereo SLAM system through the combination of points and line segments[J]. IEEE Transactions on Robotics,2019,35(3):734-746. doi: 10.1109/TRO.2019.2899783 [12] YANG Gaochao,WANG Qing,LIU Pengfei,et al. An improved monocular PL-SlAM method with point-line feature fusion under low-texture environment[C]. 4th International Conference on Control and Computer Vision,Macau,2021:119-125. [13] ZHAO Shibo,FANG Zheng,LI Haolai,et al. A robust laser-inertial odometry and mapping method for large-scale highway environments[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Macau,2019:1285-1292. [14] SHAN Tixiao,ENGLOT B,MEYERS D,et al. Lio-sam:tightly-coupled lidar inertial odometry via smoothing and mapping[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Las Vegas,2020:5135-5142. [15] ZHANG Ji,SINGH S. Visual-lidar odometry and mapping:low-drift,robust,and fast[C]. IEEE International Conference on Robotics and Automation,Seattle,2015:2174-2181. [16] LIU Yanqing,YANG Dongdong,LI Jiamao,et al. Stereo visual-inertial SLAM with points and lines[J]. IEEE Access,2018,6:69381-69392. doi: 10.1109/ACCESS.2018.2880689 [17] TIAN Chengjun,LIU Haobo,LIU Zhe,et al. Research on multi-sensor fusion SLAM algorithm based on improved gmapping[J]. IEEE Access,2023,11(9):13690-13703. [18] SHAN Tixiao,ENGLOT B,RATTI C,et al. Lvi-sam:tightly-coupled lidar-visual-inertial odometry via smoothing and mapping[C]. IEEE International Conference on Robotics and Automation,Xi'an,2021:5692-5698. [19] 隋心,王思语,罗力,等. 基于点云特征的改进RANSAC地面分割算法[J]. 导航定位学报,2024,12(1):106-114l.SUI Xin,WANG Siyu,LUO Li,et al. Improved RANSAC ground segmentation algorithm based on point cloud features[J]. Journal of Navigation and Positioning,2024,12(1):106-114. [20] HUANG S C,CHENG F C,CHIU Y S. Efficient contrast enhancement using adaptive gamma correction with weighting distribution[J]. IEEE Transactions on Image Processing,2013,22(3):1032-1041. doi: 10.1109/TIP.2012.2226047 [21] WANG Jun,WANG Rui,WU Anwen. Improved gamma correction for visual slam in low-light scenes[C]. IEEE 3rd Advanced Information Management,Communicates,Electronic and Automation Control Conference,Chongqing,2019:1159-1163. [22] HUANG Lidong,ZHAO Wei,WANG Jun,et al. Combination of contrast limited adaptive histogram equalisation and discrete wavelet transform for image enhancement[J]. IET Image Processing,2015,9(10):908-915. doi: 10.1049/iet-ipr.2015.0150 [23] MESSIKOMMER N,FANG C,GEHRIG M,et al. Data-driven feature tracking for event cameras[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:5642-5651. [24] GIOI R G,JAKUBOWICZ J,MOREL J M,et al. LSD:a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(4):722-732. doi: 10.1109/TPAMI.2008.300 [25] ZHANG Lilian,KOCH R. An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J]. Journal of Visual Communication and Image Representation,2013,24(7):794-805. doi: 10.1016/j.jvcir.2013.05.006 [26] LIU Zhe,SHI Dianxi,LI Ruihao,et al. PLC-VIO:visual-inertial odometry based on point-line constraints[J]. IEEE Transactions on Automation Science and Engineering,2022,19(3):1880-1897. doi: 10.1109/TASE.2021.3077026 [27] TRIGGS B,MCLAUCHLAN P F,HARTLEY R I,et al. Bundle adjustment-a modern synthesis[C]. Vision Algorithms:Theory and Practic,1999. DOI: 10.1007/3-540-44480-7_21. [28] LI Ang,ZOU Danping,YU Wenxian. Robust initialization of multi-camera slam with limited view overlaps and inaccurate extrinsic calibration[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Pargue,2021:3361-3367. [29] FURGALE P,REHDER J,SIEGWART R. Unified temporal and spatial calibration for multi-sensor systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Tokyo,2013:1280-1286. [30] MAJEED S H,ISA N A M. Adaptive entropy index histogram equalization for poor contrast images[J]. IEEE Access,2020,9:6402-6437. -
点击查看大图
图(10) / 表(3)
计量
- 文章访问数: 101
- HTML全文浏览量: 23
- PDF下载量: 17
- 被引次数: 0
