An onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter
-
摘要: 掘进机等煤机装备在行进或作业期间，易因车体振动引起机载相机视频模糊，导致基于机载视频的机器视觉检测精度和可靠性下降。针对该问题，提出一种基于CLAHE与卡尔曼滤波的掘进机机载视频稳像算法。该算法由运动估计、轨迹平滑和运动补偿3个部分组成。在运动估计阶段，先采用限制对比度自适应直方图均衡（CLAHE）算法对井下巷道图像进行增强处理，再利用Shi-Tomasi算法获取每帧图像的特征点，对获取的特征点进行光流追踪和匹配，进而计算出相机的运动轨迹。在轨迹平滑阶段，利用卡尔曼滤波，根据视频前一帧的最优值预测当前时刻值，避免均值滤波需预先存储采样数据的问题，提高稳像的实时性。在运动补偿阶段，根据原始运动路径和平滑路径的关系对抖动视频逐帧补偿，生成稳定的视频序列。实验结果表明：① 经CLAHE增强处理后，特征点匹配成功率比未增强处理时提高了58%，比HE增强处理时提高了43%，说明CLAHE算法可有效提高图像特征点匹配数。② 通过像素偏移分析、差分图分析、峰值信噪比（PSNR）分析，验证了基于CLAHE与卡尔曼滤波的掘进机机载视频稳像算法具有较好的稳像效果。③ 与传统的HE+均值滤波算法相比，基于CLAHE与卡尔曼滤波的算法处理100帧视频图像的整体耗时减少了0.379 s，在去除抖动的同时，有效提高了稳像的实时性。Abstract: During the movement or operation of coal mining equipment such as a roadheader, the vibration of the vehicle body can easily cause blurring of the onboard camera video. This leads to a decrease in the precision and reliability of machine vision detection based on the onboard video. In order to solve the above problem, an onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter is proposed. This algorithm consists of three parts: motion estimation, trajectory smoothing, and motion compensation. In the motion estimation stage, the contrast limited adaptive histogram equalization (CLAHE) algorithm is used to enhance the image of the underground roadway. The Shi-Tomasi algorithm is used to obtain the feature points of each image frame. The obtained feature points are tracked and matched by optical flow, and then the motion trajectory of the camera is calculated. In the trajectory smoothing stage, Kalman filtering is used to predict the current time value based on the optimal value of the previous frame of the video. It avoids the problem of pre-storing sampling data of mean filtering and improves the real-time performance of image stabilization. In the motion compensation stage, the jitter video is compensated frame by frame based on the relationship between the original motion path and the smooth path, generating a stable video sequence. The experimental results show the following points: ① After CLAHE enhancement processing, the success rate of feature point matching is increased by 58% compared to the non-enhancement processing and 43% compared to the HE enhancement processing. It indicates that the CLAHE algorithm can effectively improve the matching number of image feature points. ② Through pixel offset analysis, differential image analysis, and peak signal-to-noise ratio (PSNR) analysis, it is verified that the onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter has a good image stabilization effect. ③ Compared with the traditional HE+mean filtering algorithm, the algorithm based on CLAHE and Kalman filter reduces the overall time consumption of processing 100 frames of video images by 0.379 seconds, effectively improving the real-time performance of the video stabilization while removing jitter.
-
图 1 基于CLAHE与卡尔曼滤波的掘进机机载视频稳像算法流程
Figure 1. Process of onboard video stabilization algorithm for roadheader based on contrast limited adaptive histogram equalization and Kalman filter
图 4 巷道图像增强前后特征点提取对比
Figure 4. Comparison of feature point extraction before and after roadway image enhancement
图 5 基于CLAHE增强图像的特征跟踪
Figure 5. Feature tracking based on contrast limited adaptive histogram equalization enhanced image
图 6 像素点水平及垂直方向运动轨迹
Figure 6. Horizontal and vertical motion trajectories of pixel points
图 8 稳像前后水平方向偏移对比
Figure 8. Comparison of horizontal deviation before and after image stabilization
图 10 稳像前后PSNR值对比
Figure 10. Comparison of PSNR values before and after image stabilization
表 1 滤波前后方差
Table 1. Variance before and after filtering
方向 滤波前方差 滤波后方差 水平 1.121 0.23 垂直 2.820 1.79 
下载: 导出CSV
表 2 各算法耗时
Table 2. Time consuming of each algorithm
s 算法 运动估计耗时 运动平滑耗时 相关计算耗时 整体耗时 HE+均值滤波 2.340 0.003 1.286 3.629 本文
算法1.938 0.002 1.310 3.250
下载: 导出CSV
-
[1] 张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579-597.ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote controlon roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579-597. [2] WANG Guofa,REN Huaiwei,ZHAO Guorui,et al. Research and practice of intelligent coal mine technology systems in China[J]. International Journal of Coal Science & Technology,2022,9(2):19-35. [3] 魏闪闪,谢巍,贺志强. 数字视频稳像技术综述[J]. 计算机研究与发展,2017,54(9):2044-2058. doi: 10.7544/issn1000-1239.2017.20160078WEI Shanshan,XIE Wei,HE Zhiqiang. Digital video stabilization techniques:a survey[J]. Journal of Computer Research and Development,2017,54(9):2044-2058. doi: 10.7544/issn1000-1239.2017.20160078 [4] 熊炜,王传胜,李利荣,等. 结合光流法和卡尔曼滤波的视频稳像算法[J]. 计算机工程与科学,2020,42(3):493-499. doi: 10.3969/j.issn.1007-130X.2020.03.015XIONG Wei,WANG Chuansheng,LI Lirong,et al. Video stabilization algorithm based on optical flow method and Kalman filtering[J]. Computer Engineering & Science,2020,42(3):493-499. doi: 10.3969/j.issn.1007-130X.2020.03.015 [5] 熊炜,王传胜,管来福,等. 基于特征跟踪和网格路径运动的视频稳像算法[J]. 计算机工程与科学,2020,42(5):843-850.XIONG Wei,WANG Chuansheng,GUAN Laifu,et al. A video stabilization algorithm based on feature tracking and mesh path motion[J]. Computer Engineering & Science,2020,42(5):843-850. [6] 朱娟娟,范静,郭宝龙. 抗前景干扰的自适应电子稳像算法[J]. 光子学报,2015,44(6):45-52.ZHU Juanjuan,FAN Jing,GUO Baolong. Adaptive electronic image stabilization algorithm resistant to foreground moving object[J]. Acta Photonica Sinica,2015,44(6):45-52. [7] 范叶平,郭政,张锐. 基于下采样灰度投影的电子稳像算法研究[J]. 工矿自动化,2017,43(4):22-27.FAN Yeping,GUO Zheng,ZHANG Rui. Research on electronic image stabilization algorithm based on subsample gray-scale projection[J]. Industry and Mine Automation,2017,43(4):22-27. [8] 王施鳗,许文海,董丽丽,等. 基于改进的Harris角点的机载红外图像电子稳像[J]. 红外技术,2020,42(6):573-579. doi: 10.3724/SP.J.7102068595WANG Shiman,XU Wenhai,DONG Lili,et al. Electronic image stabilization of airborne infrared images based on improved Harris corner detection[J]. Infrared Technology,2020,42(6):573-579. doi: 10.3724/SP.J.7102068595 [9] 程德强,郭政,刘洁,等. 一种基于改进光流法的电子稳像算法[J]. 煤炭学报,2015,40(3):707-712.CHENG Deqiang,GUO Zheng,LIU Jie,et al. An electronic image stabilization algorithm based on improved optical flow method[J]. Journal of China Coal Society,2015,40(3):707-712. [10] 孙继平,田子建. 矿井图像监视系统与关键技术[J]. 煤炭科学技术,2014,42(1):65-68.SUN Jiping,TIAN Zijian. Image monitoring system and key technology in underground mine[J]. Coal Science and Technology,2014,42(1):65-68. [11] 王诚聪,刘亚静. 矿井复杂环境视频监控图像增强算法研究[J]. 煤炭工程,2021,53(4):147-151.WANG Chengcong,LIU Yajing. Image enhancement algorithm for video surveillance in complex environment of underground mine[J]. Coal Engineering,2021,53(4):147-151. [12] 王殿伟,王晶,许志杰,等. 一种光照不均匀图像的自适应校正算法[J]. 系统工程与电子技术,2017,39(6):1383-1390. doi: 10.3969/j.issn.1001-506X.2017.06.29WANG Dianwei,WANG Jing,XU Zhijie,et al. Adaptive correction algorithm for non-uniform illumination images[J]. Systems Engineering and Electronics,2017,39(6):1383-1390. doi: 10.3969/j.issn.1001-506X.2017.06.29 [13] 余成波,孔庆达,田桐. 基于CLAHE与同态滤波的细胞图像增强新方法[J]. 微型机与应用,2017,36(4):51-52,62.YU Chengbo,KONG Qingda,TIAN Tong. New approach to the cell image enhancement based on adaptive histogram equalization and homomorphic filtering[J]. Microcomputer & Its Applications,2017,36(4):51-52,62. [14] YADAV G, MAHESHWARI S, AGARWAL A. Contrastlimited adaptive histogram equalization based enhancement for real time video system[C]. International Conference on Advances in Computing, Communications and Informatics, Delhi, 2014: 2392-2397. [15] BANSAL M,KUMAR M,KUMAR K,et al. An efficient technique for object recognition using Shi-Tomasi corner detection algorithm[J]. Soft Computing,2021,25(6):4423-4432. doi: 10.1007/s00500-020-05453-y [16] WANG Zhen, YANG Xiaojun. Moving target detection and tracking based on pyramid Lucas-Kanade optical flow[C]. The 3rd IEEE International Conference on Image, Vision and Computing, Chongqing, 2018: 66-69. [17] RODRÍGUEZ-PADILLA I,CASTELLE B,MARIEU V,et al. A simple and efficient image stabilization method for coastal monitoring video systems[J]. Remote Sensing,2019,12(1):1-21. doi: 10.3390/rs12010001 [18] ZHAI Bo,ZHENG Jin,LI Bo. Digital image stabilization based on adaptive motion filtering with feedback correction[J]. Multimedia Tools and Applications,2016,75:12173-12200. doi: 10.1007/s11042-015-3183-3 [19] LI Lengyi, MA Xiaohong, ZHAO Zheng. Real-time video stabilization based on fast block matching and improved Kalman filter[C]. The Fifth International Conference on Intelligent Control and Information Processing, Dalian, 2014: 394-397. [20] 程德强,黄晓丽,厉航,等. 矿井车载视频图像稳像算法研究[J]. 工矿自动化,2017,43(11):21-26.CHENG Deqiang,HUANG Xiaoli,LI Hang,et al. Research on vehicle video image stabilization algorithm for mine[J]. Industry and Mine Automation,2017,43(11):21-26. [21] SOUZA M R E,MAIA H D A,PEDRINI H. Survey on digital video stabilization:concepts,methods,and challenges[J]. ACM Computing Surveys,2022,55(3):1-37. [22] 陈滨,赵建军,杨利斌. 视频稳像评价方法研究[J]. 科学技术与工程,2016,16(22):219-224,230. doi: 10.3969/j.issn.1671-1815.2016.22.039CHEN Bin,ZHAO Jianjun,YANG Libin. Research on evaluation method of video stabilization[J]. Science Technology and Engineering,2016,16(22):219-224,230. doi: 10.3969/j.issn.1671-1815.2016.22.039 -
点击查看大图
图(10) / 表(2)
计量
- 文章访问数: 350
- HTML全文浏览量: 81
- PDF下载量: 19
- 被引次数: 0
