顾及图像增强的煤矿井下视觉同时定位与建图算法

冯玮; 姚顽强; 蔺小虎; 郑俊良; 相里海龙; 薛志强

doi:10.13272/j.issn.1671-251x.2022090025

顾及图像增强的煤矿井下视觉同时定位与建图算法

doi: 10.13272/j.issn.1671-251x.2022090025

1.
西安科技大学测绘科学与技术学院, 陕西西安　710054
2.
陕西彬长孟村矿业有限公司, 陕西咸阳　713600

基金项目: 国家自然科学基金资助项目（42201484）。

详细信息

作者简介:
冯玮（1996—），男，河南新乡人，硕士研究生，研究方向为煤矿井下视觉同时定位与建图，E-mail：280155468@qq.com

中图分类号: TD67
计量
- 文章访问数: 851
- HTML全文浏览量: 234
- PDF下载量: 34
- 被引次数: 0
出版历程
- 收稿日期: 2022-09-08
- 修回日期: 2023-05-15
- 网络出版日期: 2022-11-28

Visual simultaneous localization and mapping algorithm of coal mine underground considering image enhancement

1.
College of Geomatics, Xi'an University of Science and Technology, Xi'an 710054, China
2.
Shaanxi Binchang Mengcun Mining Co., Ltd., Xianyang 713600, China

摘要

摘要: 基于特征点法的视觉同时定位与建图（SLAM）算法在煤矿井下有一定应用，但受光照不均、光照多变、明暗区域交错等因素影响，图像质量较差、纹理信息匮乏，导致视觉SLAM前端特征提取与匹配精度较低，易出现跟踪丢失问题，影响视觉SLAM算法的定位精度与建图效果。提出一种顾及图像增强的煤矿井下视觉SLAM算法，通过图像增强处理提升视觉SLAM的整体性能。采用基于改进双边滤波的Retinex算法对煤矿井下图像进行增强处理：将原始RGB图像转换至HSI色彩空间，以改进的双边滤波代替传统Retinex算法的高斯滤波作为中心环绕函数，对图像反射分量进行估计后转换至RGB色彩空间，得到最终增强图像。将基于改进双边滤波的Retinex算法引入经典ORB−SLAM2算法框架进行位姿估计和建图。基于矿井轮式机器人数据采集平台在煤矿井下巷道环境中对顾及图像增强的视觉SLAM算法进行试验，结果表明：与传统Retinex算法相比，经基于改进双边滤波的Retinex算法增强后的煤矿井下图像未出现明显的泛白及光晕现象，图像质量得到提升；与ORB−SLAM2算法相比，顾及图像增强的视觉SLAM算法提高了特征匹配质量和数量，估计轨迹与真实轨迹的重合度更高，绝对轨迹误差平均值下降了76.2%，且建立的井下巷道三维稠密点云地图更加真实和准确。
- 矿井图像 /
- 视觉分析 /
- 图像增强 /
- 同时定位与建图 /
- 改进双边滤波 /
- 改进Retinex算法
Abstract: The visual simultaneous localization and mapping (SLAM) algorithm based on the feature point method has certain applications in coal mines. However, due to factors such as uneven lighting, variable lighting, and alternating light and dark areas, the image quality is poor and texture information is lacking. This results in low precision of feature extraction and matching in the front end of visual SLAM. The problem of tracking loss is prone to occur, which affects the positioning precision and mapping effect of the visual SLAM algorithm. This study proposes a visual SLAM algorithm of coal mine underground considering image enhancement. The overall performance of visual SLAM is improved through image enhancement processing. Retinex algorithm based on improved bilateral filter is used to enhance the coal mine underground image. The original RGB image is converted to HSI color space, and the improved bilateral filter replaces the Gaussian filter of the traditional Retinex algorithm as the central surrounding function. After the image reflection component is estimated, it is converted to RGB color space to obtain the final enhanced image. Retinex algorithm based on improved bilateral filter is introduced into the classical ORB-SLAM2 algorithm framework for pose estimation and mapping. Based on the data collection platform of the wheeled mine-used robot, the visual SLAM algorithm considering image enhancement is tested in the roadway environment of coal mine underground. The results show that, compared with the traditional Retinex algorithm, the coal mine image enhanced by the Retinex algorithm based on improved bilateral filter does not show obvious whitening and halo, and the image quality is improved. Compared with the ORB-SLAM2 algorithm, the visual SLAM algorithm considering image enhancement improves the quality and quantity of feature matching. It has a higher degree of overlap between estimated trajectories and real trajectories. It reduces the mean absolute trajector error by 76.2%. It establishes a more realistic and accurate 3D dense point cloud map of underground roadway.
- mine image /
- visual analysis /
- image enhancement /
- simultaneous localization and mapping /
- improved bilateral filter /
- improved Retinex algorithm

HTML全文

图 1 顾及图像增强的视觉SLAM算法框架

①−基于改进双边滤波的Retinex算法；②−改进双边滤波算法。

Figure 1. Visual simultaneous localization and mapping (SLAM) algorithm framework considering image enhancement

下载: 全尺寸图片幻灯片

图 2 RGB图像增强流程

Figure 2. RGB image enhancement flow

下载: 全尺寸图片幻灯片

图 3 传统Retinex算法的图像处理效果

Figure 3. Image process effect of traditional Retinex algorithm

下载: 全尺寸图片幻灯片

图 4 基于改进双边滤波的Retinex算法流程

Figure 4. Retinex algorithm flow based on improved bilateral filter

下载: 全尺寸图片幻灯片

图 5 视觉SLAM算法流程

Figure 5. Flow of visual SLAM algorithm

下载: 全尺寸图片幻灯片

图 6 矿井轮式机器人数据采集平台

Figure 6. Data collection platform of wheeled mine-used robot

下载: 全尺寸图片幻灯片

图 7 不同算法的图像增强效果

Figure 7. Image enhancement effect of different algorithms

下载: 全尺寸图片幻灯片

图 8 相邻2帧图像的特征匹配结果

Figure 8. Feature matching results of two adjacent image frames

下载: 全尺寸图片幻灯片

图 9 SLAM估计轨迹与真实轨迹对比

Figure 9. Comparison between estimated trajectories and the real ones of SLAM

下载: 全尺寸图片幻灯片

图 10 煤矿井下巷道稠密建图试验结果

Figure 10. Densely mapping test results of underground roadway

下载: 全尺寸图片幻灯片

表 1 欠曝图像客观评价指标得分

Table 1. Objective evaluation index scores of underexposed images

图像	均值	标准差	平均梯度	信息熵
原始图像	44.320 4	53.125 7	4.889 5	4.511 9
SSR算法处理图像	99.605 5	73.976 9	7.347 1	6.609 6
MSR算法处理图像	113.168 1	77.530 6	7.551 2	7.036 1
改进Retinex算法处理图像	122.686 9	97.890 8	8.622 9	7.455 9

下载: 导出CSV

表 2 过曝图像客观评价指标得分

Table 2. Objective evaluation index scores of overexposed images

图像	均值	标准差	平均梯度	信息熵
原始图像	157.089 6	87.239 1	5.399 5	6.460 7
SSR算法处理图像	163.666 5	56.840 9	5.442 0	5.778 3
MSR算法处理图像	208.409 8	61.418 4	5.448 7	6.211 7
改进Retinex算法处理图像	205.977 4	89.148 5	6.043 4	6.839 6

下载: 导出CSV

表 3 正常曝光图像客观评价指标得分

Table 3. Objective evaluation index scores of normal exposed images

图像	均值	标准差	平均梯度	信息熵
原始图像	118.563 7	92.409 0	6.399 6	7.015 5
SSR算法处理图像	204.135 9	95.734 1	6.984 2	7.247 6
MSR算法处理图像	193.951 6	99.818 8	7.487 7	7.364 9
改进Retinex算法处理图像	153.404 4	100.207 5	8.604 1	7.703 7

下载: 导出CSV

表 4 煤矿井下图像特征匹配统计结果

Table 4. Statistical feature matching results of underground coal mine images

场景	ORB−SLAM2算法		本文算法		整体提高百分比/%
场景	匹配数量	匹配正确率/%	匹配数量	匹配正确率/%	整体提高百分比/%
欠曝	100	63.0	125	73.5	20.9
过曝	105	65.5	126	72.7	15.5
正常曝光	113	72.2	126	76.0.	8.4

下载: 导出CSV

表 5 ATE试验结果

Table 5. Test results of absolute trajectory error(ATE) m

算法	最大值	最小值	平均值	RMSE
ORB−SLAM2	2.327	0.003	0.827	1.196
本文算法	1.279	0.002	0.197	0.287

下载: 导出CSV

参考文献(21)

[1]	王龙. 基于航迹推算的井下人员定位关键算法研究[D]. 徐州: 中国矿业大学, 2015. WANG Long. Study on the key algorithms of underground personnel positioning based on dead reckoning[D]. Xuzhou: China University of Mining and Technology, 2015.
[2]	葛世荣,胡而已,裴文良. 煤矿机器人体系及关键技术[J]. 煤炭学报,2020,45(1):455-463. GE Shirong,HU Eryi,PEI Wenliang. Classification system and key technology of coal mine robot[J]. Journal of China Coal Society,2020,45(1):455-463.
[3]	杨必胜,梁福逊,黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报,2017,46(10):1509-1516. YANG Bisheng,LIANG Fuxun,HUANG Ronggang. Progress,challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica,2017,46(10):1509-1516.
[4]	吴东金,夏林元. 面向室内WLAN定位的动态自适应模型[J]. 测绘学报,2015,44(12):1322-1330. WU Dongjin,XIA Linyuan. Dynamic adaptive model for indoor WLAN localization[J]. Acta Geodaetica et Cartographica Sinica,2015,44(12):1322-1330.
[5]	符世琛,李一鸣,杨健健,等. 基于超宽带技术的掘进机自主定位定向方法研究[J]. 煤炭学报,2015,40(11):2603-2610. FU Shichen,LI Yiming,YANG Jianjian,et al. Research on autonomous positioning and orientation method of roadheader based on ultra wide-band technology[J]. Journal of China Coal Society,2015,40(11):2603-2610.
[6]	李论,张著洪,丁恩杰,等. 基于RSSI的煤矿巷道高精度定位算法研究[J]. 中国矿业大学学报,2017,46(1):183-191,200. LI Lun,ZHANG Zhuhong,DING Enjie,et al. Precision positioning algorithm in coal mine tunnel based on RSSI[J]. Journal of China University of Mining & Technology,2017,46(1):183-191,200.
[7]	马宏伟,王岩,杨林. 煤矿井下移动机器人深度视觉自主导航研究[J]. 煤炭学报,2020,45(6):2193-2206. MA Hongwei,WANG Yan,YANG Lin. Research on depth vision based mobile robot autonomous navigation in underground coal mine[J]. Journal of China Coal Society,2020,45(6):2193-2206.
[8]	齐广峰,吕军锋. MEMS惯性技术的发展及应用[J]. 电子设计工程,2015,23(1):87-89,92. QI Guangfeng,LYU Junfeng. Evolution and application of MEMS inertial technology[J]. Electronic Design Engineering,2015,23(1):87-89,92.
[9]	江传龙,黄宇昊,韩超,等. 井下巡检无人机系统设计及定位与避障技术[J]. 机械设计与研究,2021,37(4):38-42,48. JIANG Chuanlong,HUANG Yuhao,HAN Chao,et al. Design of underground inspection UAV system and study of positioning and obstacle avoidance[J]. Machine Design & Research,2021,37(4):38-42,48.
[10]	FUENTES-PACHECO J,RUIZ-ASCENCIO J,RENDÓN-MANCHA J M. Visual simultaneous localization and mapping:a survey[J]. Artificial Intelligence Review,2015,43(1):55-81. doi: 10.1007/s10462-012-9365-8
[11]	DURRANT-WHYTE H,BAILEY T. Simultaneous localization and mapping:part I[J]. IEEE Robotics & Automation Magazine,2006,13(2):99-110.
[12]	丁文东,徐德,刘希龙,等. 移动机器人视觉里程计综述[J]. 自动化学报,2018,44(3):385-400. DING Wendong,XU De,LIU Xilong,et al. Review on visual odometry for mobile robots[J]. Acta Automatica Sinica,2018,44(3):385-400.
[13]	尚磊,王杰,宋尊师,等. 基于单目ORB−SLAM2算法的煤矿搜救机器人定位研究[J]. 机床与液压,2020,48(11):49-52. SHANG Lei,WANG Jie,SONG Zunshi,et al. Research on the localization of the coal mine search and rescue robot based on the monocular ORB-SLAM2 algorithm[J]. Machine Tool & Hydraulics,2020,48(11):49-52.
[14]	MUR-ARTAL R,TARDOS J D. ORB-SLAM2:an open-source slam system for monocular,stereo,and RGB-D cameras[J]. IEEE Transactions on Robotics,2017,33(5):1255-1262.
[15]	ZHU Daixian,JI Kangkang,WU Dong,et al. A coupled visual and inertial measurement units method for locating and mapping in coal mine tunnel[J]. Sensors,2022,22(19):7437. doi: 10.3390/s22197437
[16]	朱礼义. 矿井图像增强和井下人员检测算法的研究[D]. 徐州: 中国矿业大学, 2019. ZHU Liyi. Research on mine image enhancement and underground personnel detection[D]. Xuzhou: China University of Mining and Technology, 2019.
[17]	洪炎,朱丹萍,龚平顺. 基于TopHat加权引导滤波的Retinex矿井图像增强算法[J]. 工矿自动化,2022,48(8):43-49. HONG Yan,ZHU Danping,GONG Pingshun. Retinex mine image enhancement algorithm based on TopHat weighted guided filtering[J]. Journal of Mine Automation,2022,48(8):43-49.
[18]	张立亚,郝博南,孟庆勇,等. 基于HSV空间改进融合Retinex算法的井下图像增强方法[J]. 煤炭学报,2020,45(增刊1):532-540. ZHANG Liya,HAO Bonan,MENG Qingyong,et al. Method of image enhancement in coal mine based on improved retex fusion algorithm in HSV space[J]. Journal of China Coal Society,2020,45(S1):532-540.
[19]	谢凤英,汤萌,张蕊. 基于Retinex的图像增强方法综述[J]. 数据采集与处理,2019,34(1):1-11. XIE Fengying,TANG Meng,ZHANG Rui. Review of image enhancement algorithms based on Retinex[J]. Journal of Data Acquisition and Processing,2019,34(1):1-11.
[20]	GARCIA V, DEBREUVE E, NIELSEN F, et al. K-nearest neighbor search: Fast GPU-based implementations and application to high-dimensional feature matching[C]. IEEE International Conference on Image Processing, Hong Kong, 2010: 3757-3760.
[21]	STURM J, ENGELHARD N, ENDRES F, et al. A benchmark for the evaluation of RGBD SLAM systems[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, New York, 2012: 573-580.