Visual simultaneous localization and mapping algorithm of coal mine underground considering image enhancement
-
摘要: 基于特征点法的视觉同时定位与建图（SLAM）算法在煤矿井下有一定应用，但受光照不均、光照多变、明暗区域交错等因素影响，图像质量较差、纹理信息匮乏，导致视觉SLAM前端特征提取与匹配精度较低，易出现跟踪丢失问题，影响视觉SLAM算法的定位精度与建图效果。提出一种顾及图像增强的煤矿井下视觉SLAM算法，通过图像增强处理提升视觉SLAM的整体性能。采用基于改进双边滤波的Retinex算法对煤矿井下图像进行增强处理：将原始RGB图像转换至HSI色彩空间，以改进的双边滤波代替传统Retinex算法的高斯滤波作为中心环绕函数，对图像反射分量进行估计后转换至RGB色彩空间，得到最终增强图像。将基于改进双边滤波的Retinex算法引入经典ORB−SLAM2算法框架进行位姿估计和建图。基于矿井轮式机器人数据采集平台在煤矿井下巷道环境中对顾及图像增强的视觉SLAM算法进行试验，结果表明：与传统Retinex算法相比，经基于改进双边滤波的Retinex算法增强后的煤矿井下图像未出现明显的泛白及光晕现象，图像质量得到提升；与ORB−SLAM2算法相比，顾及图像增强的视觉SLAM算法提高了特征匹配质量和数量，估计轨迹与真实轨迹的重合度更高，绝对轨迹误差平均值下降了76.2%，且建立的井下巷道三维稠密点云地图更加真实和准确。Abstract: The visual simultaneous localization and mapping (SLAM) algorithm based on the feature point method has certain applications in coal mines. However, due to factors such as uneven lighting, variable lighting, and alternating light and dark areas, the image quality is poor and texture information is lacking. This results in low precision of feature extraction and matching in the front end of visual SLAM. The problem of tracking loss is prone to occur, which affects the positioning precision and mapping effect of the visual SLAM algorithm. This study proposes a visual SLAM algorithm of coal mine underground considering image enhancement. The overall performance of visual SLAM is improved through image enhancement processing. Retinex algorithm based on improved bilateral filter is used to enhance the coal mine underground image. The original RGB image is converted to HSI color space, and the improved bilateral filter replaces the Gaussian filter of the traditional Retinex algorithm as the central surrounding function. After the image reflection component is estimated, it is converted to RGB color space to obtain the final enhanced image. Retinex algorithm based on improved bilateral filter is introduced into the classical ORB-SLAM2 algorithm framework for pose estimation and mapping. Based on the data collection platform of the wheeled mine-used robot, the visual SLAM algorithm considering image enhancement is tested in the roadway environment of coal mine underground. The results show that, compared with the traditional Retinex algorithm, the coal mine image enhanced by the Retinex algorithm based on improved bilateral filter does not show obvious whitening and halo, and the image quality is improved. Compared with the ORB-SLAM2 algorithm, the visual SLAM algorithm considering image enhancement improves the quality and quantity of feature matching. It has a higher degree of overlap between estimated trajectories and real trajectories. It reduces the mean absolute trajector error by 76.2%. It establishes a more realistic and accurate 3D dense point cloud map of underground roadway.
-
-
![]()
图 1 顾及图像增强的视觉SLAM算法框架
①−基于改进双边滤波的Retinex算法；②−改进双边滤波算法。
Figure 1. Visual simultaneous localization and mapping (SLAM) algorithm framework considering image enhancement
![]()
图 3 传统Retinex算法的图像处理效果
Figure 3. Image process effect of traditional Retinex algorithm
![]()
图 4 基于改进双边滤波的Retinex算法流程
Figure 4. Retinex algorithm flow based on improved bilateral filter
![]()
图 9 SLAM估计轨迹与真实轨迹对比
Figure 9. Comparison between estimated trajectories and the real ones of SLAM
表 1 欠曝图像客观评价指标得分
Table 1 Objective evaluation index scores of underexposed images
图像 均值 标准差 平均梯度 信息熵 原始图像 44.320 4 53.125 7 4.889 5 4.511 9 SSR算法处理图像 99.605 5 73.976 9 7.347 1 6.609 6 MSR算法处理图像 113.168 1 77.530 6 7.551 2 7.036 1 改进Retinex算法处理图像 122.686 9 97.890 8 8.622 9 7.455 9 
下载: 导出CSV
表 2 过曝图像客观评价指标得分
Table 2 Objective evaluation index scores of overexposed images
图像 均值 标准差 平均梯度 信息熵 原始图像 157.089 6 87.239 1 5.399 5 6.460 7 SSR算法处理图像 163.666 5 56.840 9 5.442 0 5.778 3 MSR算法处理图像 208.409 8 61.418 4 5.448 7 6.211 7 改进Retinex算法处理图像 205.977 4 89.148 5 6.043 4 6.839 6
下载: 导出CSV
表 3 正常曝光图像客观评价指标得分
Table 3 Objective evaluation index scores of normal exposed images
图像 均值 标准差 平均梯度 信息熵 原始图像 118.563 7 92.409 0 6.399 6 7.015 5 SSR算法处理图像 204.135 9 95.734 1 6.984 2 7.247 6 MSR算法处理图像 193.951 6 99.818 8 7.487 7 7.364 9 改进Retinex算法处理图像 153.404 4 100.207 5 8.604 1 7.703 7
下载: 导出CSV
表 4 煤矿井下图像特征匹配统计结果
Table 4 Statistical feature matching results of underground coal mine images
场景 ORB−SLAM2算法 本文算法 整体提高
百分比/%匹配数量 匹配正
确率/%匹配数量 匹配正
确率/%欠曝 100 63.0 125 73.5 20.9 过曝 105 65.5 126 72.7 15.5 正常曝光 113 72.2 126 76.0. 8.4
下载: 导出CSV
表 5 ATE试验结果
Table 5 Test results of absolute trajectory error(ATE)
m 算法 最大值 最小值 平均值 RMSE ORB−SLAM2 2.327 0.003 0.827 1.196 本文算法 1.279 0.002 0.197 0.287
下载: 导出CSV
-
[1] 王龙. 基于航迹推算的井下人员定位关键算法研究[D]. 徐州: 中国矿业大学, 2015. WANG Long. Study on the key algorithms of underground personnel positioning based on dead reckoning[D]. Xuzhou: China University of Mining and Technology, 2015.
[2] 葛世荣,胡而已,裴文良. 煤矿机器人体系及关键技术[J]. 煤炭学报,2020,45(1):455-463. GE Shirong,HU Eryi,PEI Wenliang. Classification system and key technology of coal mine robot[J]. Journal of China Coal Society,2020,45(1):455-463.
[3] 杨必胜,梁福逊,黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报,2017,46(10):1509-1516. YANG Bisheng,LIANG Fuxun,HUANG Ronggang. Progress,challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica,2017,46(10):1509-1516.
[4] 吴东金,夏林元. 面向室内WLAN定位的动态自适应模型[J]. 测绘学报,2015,44(12):1322-1330. WU Dongjin,XIA Linyuan. Dynamic adaptive model for indoor WLAN localization[J]. Acta Geodaetica et Cartographica Sinica,2015,44(12):1322-1330.
[5] 符世琛,李一鸣,杨健健,等. 基于超宽带技术的掘进机自主定位定向方法研究[J]. 煤炭学报,2015,40(11):2603-2610. FU Shichen,LI Yiming,YANG Jianjian,et al. Research on autonomous positioning and orientation method of roadheader based on ultra wide-band technology[J]. Journal of China Coal Society,2015,40(11):2603-2610.
[6] 李论,张著洪,丁恩杰,等. 基于RSSI的煤矿巷道高精度定位算法研究[J]. 中国矿业大学学报,2017,46(1):183-191,200. LI Lun,ZHANG Zhuhong,DING Enjie,et al. Precision positioning algorithm in coal mine tunnel based on RSSI[J]. Journal of China University of Mining & Technology,2017,46(1):183-191,200.
[7] 马宏伟,王岩,杨林. 煤矿井下移动机器人深度视觉自主导航研究[J]. 煤炭学报,2020,45(6):2193-2206. MA Hongwei,WANG Yan,YANG Lin. Research on depth vision based mobile robot autonomous navigation in underground coal mine[J]. Journal of China Coal Society,2020,45(6):2193-2206.
[8] 齐广峰,吕军锋. MEMS惯性技术的发展及应用[J]. 电子设计工程,2015,23(1):87-89,92. QI Guangfeng,LYU Junfeng. Evolution and application of MEMS inertial technology[J]. Electronic Design Engineering,2015,23(1):87-89,92.
[9] 江传龙,黄宇昊,韩超,等. 井下巡检无人机系统设计及定位与避障技术[J]. 机械设计与研究,2021,37(4):38-42,48. JIANG Chuanlong,HUANG Yuhao,HAN Chao,et al. Design of underground inspection UAV system and study of positioning and obstacle avoidance[J]. Machine Design & Research,2021,37(4):38-42,48.
[10] FUENTES-PACHECO J,RUIZ-ASCENCIO J,RENDÓN-MANCHA J M. Visual simultaneous localization and mapping:a survey[J]. Artificial Intelligence Review,2015,43(1):55-81. DOI: 10.1007/s10462-012-9365-8
[11] DURRANT-WHYTE H,BAILEY T. Simultaneous localization and mapping:part I[J]. IEEE Robotics & Automation Magazine,2006,13(2):99-110.
[12] 丁文东,徐德,刘希龙,等. 移动机器人视觉里程计综述[J]. 自动化学报,2018,44(3):385-400. DING Wendong,XU De,LIU Xilong,et al. Review on visual odometry for mobile robots[J]. Acta Automatica Sinica,2018,44(3):385-400.
[13] 尚磊,王杰,宋尊师,等. 基于单目ORB−SLAM2算法的煤矿搜救机器人定位研究[J]. 机床与液压,2020,48(11):49-52. SHANG Lei,WANG Jie,SONG Zunshi,et al. Research on the localization of the coal mine search and rescue robot based on the monocular ORB-SLAM2 algorithm[J]. Machine Tool & Hydraulics,2020,48(11):49-52.
[14] MUR-ARTAL R,TARDOS J D. ORB-SLAM2:an open-source slam system for monocular,stereo,and RGB-D cameras[J]. IEEE Transactions on Robotics,2017,33(5):1255-1262.
[15] ZHU Daixian,JI Kangkang,WU Dong,et al. A coupled visual and inertial measurement units method for locating and mapping in coal mine tunnel[J]. Sensors,2022,22(19):7437. DOI: 10.3390/s22197437
[16] 朱礼义. 矿井图像增强和井下人员检测算法的研究[D]. 徐州: 中国矿业大学, 2019. ZHU Liyi. Research on mine image enhancement and underground personnel detection[D]. Xuzhou: China University of Mining and Technology, 2019.
[17] 洪炎,朱丹萍,龚平顺. 基于TopHat加权引导滤波的Retinex矿井图像增强算法[J]. 工矿自动化,2022,48(8):43-49. HONG Yan,ZHU Danping,GONG Pingshun. Retinex mine image enhancement algorithm based on TopHat weighted guided filtering[J]. Journal of Mine Automation,2022,48(8):43-49.
[18] 张立亚,郝博南,孟庆勇,等. 基于HSV空间改进融合Retinex算法的井下图像增强方法[J]. 煤炭学报,2020,45(增刊1):532-540. ZHANG Liya,HAO Bonan,MENG Qingyong,et al. Method of image enhancement in coal mine based on improved retex fusion algorithm in HSV space[J]. Journal of China Coal Society,2020,45(S1):532-540.
[19] 谢凤英,汤萌,张蕊. 基于Retinex的图像增强方法综述[J]. 数据采集与处理,2019,34(1):1-11. XIE Fengying,TANG Meng,ZHANG Rui. Review of image enhancement algorithms based on Retinex[J]. Journal of Data Acquisition and Processing,2019,34(1):1-11.
[20] GARCIA V, DEBREUVE E, NIELSEN F, et al. K-nearest neighbor search: Fast GPU-based implementations and application to high-dimensional feature matching[C]. IEEE International Conference on Image Processing, Hong Kong, 2010: 3757-3760.
[21] STURM J, ENGELHARD N, ENDRES F, et al. A benchmark for the evaluation of RGBD SLAM systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, New York, 2012: 573-580.
-
期刊类型引用(11)
1. 迟海波. 基于串并联变换的多机械臂矸石分拣系统. 煤矿安全. 2023(12): 239-245 . 
百度学术
2. 曹现刚，刘思颖，王鹏，许罡，吴旭东. 面向煤矸分拣机器人的煤矸识别定位系统研究. 煤炭科学技术. 2022(01): 237-246 . 百度学术
3. 李小龙，夏云凯. 内蒙古地区易泥化动力煤干选工艺发展趋势研究. 煤炭加工与综合利用. 2022(02): 1-6+10 . 百度学术
4. 唐丽均. 模糊决策树在煤矸石分拣中的应用. 湖南工业职业技术学院学报. 2022(04): 9-12 . 百度学术
5. 张烨，马宏伟，王鹏，曹现刚，魏小荣，周文剑. 煤矸石智能分拣机器人研究进展与关键技术. 工矿自动化. 2022(12): 42-48+56 . 本站查看
6. 杜京义，史志芒，郝乐，陈瑞. 轻量化煤矸目标检测方法研究. 工矿自动化. 2021(11): 119-125 . 本站查看
7. 李素环，夏云凯. X射线块煤智能分选机在脏杂煤分选中的应用. 洁净煤技术. 2021(S2): 47-52 . 百度学术
8. 饶中钰，吴景涛，李明. 煤矸石图像分类方法. 工矿自动化. 2020(03): 69-73 . 本站查看
9. 王子军. IDS智能干选机在霍林河南露天煤矿的应用. 选煤技术. 2020(05): 56-60 . 百度学术
10. 邵尚坤，孙学鹏，杜晓光，李玉飞，王亚冰，张晓芸，刘志国，孙天希. 基于X射线透射成像及荧光双模式的珍珠检测. 光谱学与光谱分析. 2020(12): 3936-3940 . 百度学术
11. 武国平，梁兴国，胡金良，葛小冬. 图像处理和SVM应用于煤矸石分选的实验研究. 信息技术. 2019(01): 97-102+107 . 百度学术
其他类型引用(9)
计量
- 文章访问数: 869
- HTML全文浏览量: 252
- PDF下载量: 41
- 被引次数: 20
