基于改进最佳缝合线的矿井图像拼接方法

张旭辉; 王悦; 杨文娟; 陈鑫; 张超; 黄梦瑶; 刘彦徽; 杨骏豪

doi:10.13272/j.issn.1671-251x.2023120003

基于改进最佳缝合线的矿井图像拼接方法

doi: 10.13272/j.issn.1671-251x.2023120003

张旭辉^{1, 2,},
王悦¹,
杨文娟^{1, 2, ,},
陈鑫¹,
张超¹,
黄梦瑶¹,
刘彦徽¹,
杨骏豪¹

1.
西安科技大学机械工程学院，陕西西安　710054
2.
西安科技大学陕西省矿山机电装备智能检测与控制重点实验室，陕西西安　710054

基金项目: 国家自然科学基金青年项目（52104166）；陕煤联合基金项目（2021JLM-03）；中国博士后科学基金面上项目（2022MD723826）；陕西省重点研发计划项目（2023-YBGY-063）。

详细信息

作者简介:
张旭辉（1972—），男，陕西凤翔人，教授，博士，研究方向为煤矿机电设备智能检测与控制， E-mail：zhangxh@xust.edu.cn

通讯作者:
杨文娟（1989—），女，山西文水人，副教授，研究方向为智能检测与控制，E-mail： yangwenjuan@xust.edu.cn。

中图分类号: TD67
计量
- 文章访问数: 151
- HTML全文浏览量: 71
- PDF下载量: 29
- 被引次数: 0
出版历程
- 收稿日期: 2023-12-02
- 修回日期: 2024-04-15
- 网络出版日期: 2024-05-10

A mine image stitching method based on improved best seam-line

1.
College of Mechanical Engineering, Xi'an University of Science and Technology, Xi'an 710054, China
2.
Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi'an University of Science and Technology, Xi'an 710054, China

摘要

摘要: 煤矿井下掘进工作面高粉尘、低照度的恶劣环境导致图像信噪比较低，且有效特征点数量严重减少，处理后的图像存在较大色差和噪声，在使用最佳缝合线算法进行图像拼接时出现细节错位、缝合线处过渡不自然或拼接痕迹明显的现象。针对上述问题，提出了一种基于改进最佳缝合线的矿井图像拼接方法。首先，对原始图像进行HSV空间变换，采用改进的Retinex算法对亮度分量进行增强，利用双边滤波函数代替中心环绕函数，以解决亮度差异大处产生的光晕问题，通过增强算法有效提高特征点提取数量。然后，采用SIFT算法提取特征点，并以余弦距离作为匹配度指标；引入像素余弦相似度作为约束项，并采用形态学操作对颜色差异强度进行改进，利用动态规划法对最佳缝合线进行搜索，以避免图像拼接处的错位现象。最后，结合渐入渐出融合算法，使图像过渡平滑，实现煤矿井下掘进工作面的图像融合。模拟井下实际工况环境进行实验验证，结果表明：基于改进最佳缝合线的矿井图像拼接方法与传统最佳缝合线算法相比，避免了颜色差异和噪声引起的错位拼接现象，拼接缝处的图像过渡更加自然，避免了“鬼影”和明显拼接缝的产生，且图像平均梯度提高2.38%，拼接时间提高32.5%，使得融合区域更加平滑自然，提高了拼接质量。
- 掘进工作面 /
- 图像拼接 /
- 图像增强 /
- 最佳缝合线 /
- Retinex算法 /
- 改进能量函数 /
- 像素余弦相似度
Abstract: The harsh environment of high dust and low lighting in the coal mine underground excavation working face results in low signal-to-noise ratio of the image, and a serious reduction in the number of effective feature points. The processed image has significant color difference and noise. When using the best seam-line algorithm for image stitching, there are problems such as fine section misalignment, unnatural transitions at the seam line, or obvious stitching traces. In order to solve the above problems, a mine image stitching method based on improved best seam-line is proposed. Firstly, the original image is subjected to HSV spatial transformation, and an improved Retinex algorithm is used for enhancement on the luminance component. Bilateral filtering is used instead of the center surround function to solve the halo problem caused by large brightness differences. The number of feature points extracted is effectively increased through the enhancement algorithm. Secondly, the SIFT algorithm is used to extract feature points, and cosine distance is used as the matching degree indicator. The method introduces pixel cosine similarity as a constraint, and uses morphological operations to improve color difference intensity, uses dynamic programming to search for the best seam-line to avoid misalignment at image stitching. Finally, combined with the gradual in and out algorithm, the image transition is smooth to achieve image fusion of the underground excavation working face. Experimental verification is conducted by simulating the actual working environment underground. The results show that the mine image stitching method based on the improved best seam-line avoids the phenomenon of misalignment stitching caused by color differences and noise compared to the traditional best seam-line algorithm. The image transition at the stitching seam is more natural, avoiding the generation of 'ghosts' and obvious stitching seams. The average gradient of the image is increased by about 2.38%, and the stitching time is increased by about 32.5%, making the fusion area smoother and more natural, improving the stitching quality.
- excavation working face /
- image stitching /
- image enhancement /
- best seam-line /
- Retinex algorithm /
- improved energy function /
- pixel cosine similarity

HTML全文

图 1 基于改进最佳缝合线的煤矿井下图像拼接方法总体方案

Figure 1. The overall scheme of coal mine image stitching method based on improved best seam-line

下载: 全尺寸图片幻灯片

图 2 图像增强算法效果对比

Figure 2. Effect comparison of image enhancement algorithms

下载: 全尺寸图片幻灯片

图 3 原图与增强后图像对比

Figure 3. Comparison between the original image and enhanced image

下载: 全尺寸图片幻灯片

图 4 图像单应性变换

Figure 4. Image homography transformation

下载: 全尺寸图片幻灯片

图 5 最佳缝合线算法

Figure 5. Best seam-line algorithm

下载: 全尺寸图片幻灯片

图 6 能量函数改进前后对比

Figure 6. Comparison before and after energy function improvement

下载: 全尺寸图片幻灯片

图 7 动态规划法寻找最佳缝合线过程

Figure 7. Process of finding the best seam-line using dynamic programming method

下载: 全尺寸图片幻灯片

图 8 部分实验结果

Figure 8. Part of the experiment results

下载: 全尺寸图片幻灯片

图 9 平均梯度对比柱状图

Figure 9. Average gradient contrast histogram

下载: 全尺寸图片幻灯片

图 10 拼接时间对比柱状图

Figure 10. Stitching time contrast histogram

下载: 全尺寸图片幻灯片

表 1 评价结果

Table 1. Evaluating results

评价指标	算法
评价指标	HE	MSR	改进Retinex算法
PSNR	8.786 8	8.731 4	11.647 2
SSIM	0.154 2	0.233 3	0.277 9
MSE	123.148 9	115.643 2	112.313 4
MAE	172.511 7	171.113 5	164.494 3

下载: 导出CSV

表 2 图像增强前后提取特征点的数量

Table 2. Number of feature points extracted before and after image enhancement

	图1	图2	图3	图4
增强前特征点数量/个	103	95	333	373
增强后特征点数量/个	966	806	1005	1017

下载: 导出CSV

表 3 图像拼接质量评价

Table 3. Image stitching quality evaluation

组别	算法	平均梯度	有无明显拼接缝	拼接时间/s
组1	传统最佳缝合线算法	2.5186	有	1.1276
组1	本文算法	2.6365	无	0.9415
组2	传统最佳缝合线算法	3.9214	有	1.7412
组2	本文算法	4.2211	无	1.3056
组3	传统最佳缝合线算法	4.9336	有	1.6361
组3	本文算法	5.0371	无	1.3140
组4	传统最佳缝合线算法	5.4266	有	1.2675
组4	本文算法	5.5375	无	0.9813
组5	传统最佳缝合线算法	2.9191	有	1.3508
组5	本文算法	2.9522	无	0.9205
组6	传统最佳缝合线算法	4.1892	有	1.0856
组6	本文算法	4.2614	无	0.7670
组7	传统最佳缝合线算法	4.8225	有	1.2719
组7	本文算法	4.8572	无	0.9388
组8	传统最佳缝合线算法	4.1991	有	1.2033
组8	本文算法	4.2344	无	0.9120

下载: 导出CSV

参考文献(21)

[1]	王国法. 加快煤矿智能化建设推进煤炭行业高质量发展[J]. 中国煤炭,2021,47(1):2-10. doi: 10.3969/j.issn.1006-530X.2021.01.002 WANG Guofa. Speeding up intelligent construction of coal mine and promoting high-quality development of coal industry[J]. China Coal,2021,47(1):2-10. doi: 10.3969/j.issn.1006-530X.2021.01.002
[2]	张旭辉,杨红强,白琳娜,等. 煤矿掘进工作面低照度视频增强技术研究[J]. 煤田地质与勘探,2023,51(1):309-316. ZHANG Xuhui,YANG Hongqiang,BAI Linna,et al. Research on low illumination video enhancement technology in coal mine heading face[J]. Coal Geology & Exploration,2023,51(1):309-316.
[3]	王国法,张良,李首滨,等. 煤矿无人化智能开采系统理论与技术研发进展[J]. 煤炭学报,2023,48(1):34-53. WANG Guofa,ZHANG Liang,LI Shoubin,et al. Progresses in theory and technological development of unmanned smart mining system[J]. Journal of China Coal Society,2023,48(1):34-53.
[4]	YUAN Yiting,FANG Faming,ZHANG Guixu. Superpixel-based seamless image stitching for UAV images[J]. IEEE Transactions on Geoscience and Remote Sensing,2021,59(2):1565-1576. doi: 10.1109/TGRS.2020.2999404
[5]	ZARAGOZA J,CHIN T J,TRAN Q H,et al. As-projective-as-possible image stitching with moving DLT[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(7). DOI: 10.1109/TPAMI.2013.247.
[6]	张晶晶,翟东海,黄莉芝,等. 基于特征分块的视差图像拼接算法[J]. 计算机工程,2018,44(5):220-226. ZHANG Jingjing,ZHAI Donghai,HUANG Lizhi,et al. Parallax image stitching algorithm based on feature blocking[J]. Computer Engineering,2018,44(5):220-226.
[7]	DUPLAQUET M L,PARK S K,JUDAY R D. Building large image mosaics with invisible seam lines[C]. Proceedings of SPIE,1998. DOI: 10.1117/12.316427.
[8]	瞿中,乔高元,林嗣鹏. 一种消除图像拼接缝和“鬼影”的快速拼接算法[J]. 计算机科学,2015,42(3):280-283. QU Zhong,QIAO Gaoyuan,LIN Sipeng. Fast image stitching algorithm eliminates seam line and ghosting[J]. Computer Science,2015,42(3):280-283.
[9]	袁怡婷. 基于最佳缝合线的大视差图像拼接[D]. 上海:华东师范大学,2020. YUAN Yiting. Optimal seam based image stitching with large parallax[D]. Shanghai:East China Normal University,2020.
[10]	张翔,王伟,肖迪. 基于改进最佳缝合线的图像拼接方法[J]. 计算机工程与设计,2018,39(7):1964-1970. ZHANG Xiang,WANG Wei,XIAO Di. Image mosaic method based on improved best seam-line[J]. Computer Engineering and Design,2018,39(7):1964-1970.
[11]	RAHMAN Z,JOBSON D,WOODELL G. Multiscale Retinex for color image enhancement[C]. 3rd IEEE International Conference on Image Processing,Lausanne,1996:1003-1006.
[12]	LAND E H,MCCANN J J. Lightness and Retinex theory[J]. Journal of the Optical Society of America,1971,61(1):1-11. doi: 10.1364/JOSA.61.000001
[13]	刘晓阳,乔通,乔智. 基于双边滤波函数和Retinex算法的矿井图像增强方法[J]. 工矿自动化,2017,43(2):49-54. LIU Xiaoyang,QIAO Tong,QIAO Zhi. Image enhancement method of mine based on bilateral filtering and Retinex algorithm[J]. Industry and Mine Automation,2017,43(2):49-54.
[14]	LI Zhengguo,SHU Haiyan,ZHENG Chaobing. Multi-scale single image dehazing using laplacian and gaussian pyramids[J]. IEEE Transactions on Image Processing,2021,30:9270-9279.
[15]	张书霞,左海平. SIFT特征匹配算法研究[J]. 现代计算机(专业版),2010(7):64-67. ZHANG Shuxia,ZUO Haiping. Research on SIFT algorithm of feature matching[J]. Modern Computer,2010(7):64-67.
[16]	CAO Mingwei,JIA Wei,LYU Zhihan,et al. Two-pass K nearest neighbor search for feature tracking[J]. IEEE Access,2018(6):72939-72951.
[17]	SHENG Haiyan,WEI Shimin,YU Xiuli,et al. Research on binocular visual system of robotic arm based on improved SURF algorithm[J]. IEEE Sensors Journal,2020,20(20):11849-11855. doi: 10.1109/JSEN.2019.2951601
[18]	杜港,侯凌燕,佟强,等. 基于BRISK和改进RANSAC算法的图像拼接[J]. 液晶与显示,2022,37(6):758-767. doi: 10.37188/CJLCD.2021-0292 DU Gang,HOU Lingyan,TONG Qiang,et al. Image mosaicing based on BRISK and improved RANSAC algorithm[J]. Chinese Journal of Liquid Crystals and Displays,2022,37(6):758-767. doi: 10.37188/CJLCD.2021-0292
[19]	邹攀红,孙晓燕,张雄伟,等. 一种基于数学形态学的二值图像去噪算法[J]. 微计算机信息,2010,26(32):202-203,206. ZOU Panhong,SUN Xiaoyan,ZHANG Xiongwei,et al. A noise reduction algorithm for binary image based on mathematical morphology[J]. Microcomputer Information,2010,26(32):202-203,206.
[20]	卢泉,杨振华,黄粒峰. 改进最佳缝合线的红外图像拼接方法[J]. 红外技术,2022,44(6):580-586. LU Quan,YANG Zhenhua,HUANG Lifeng. Infrared image mosaic method for improving the best seam-line[J]. Infrared Technology,2022,44(6):580-586.
[21]	罗永涛,王艳,张红民. 结合最佳缝合线和改进渐入渐出法的图像拼接算法[J]. 红外技术,2018,40(4):382-387. LUO Yongtao,WANG Yan,ZHANG Hongmin. Image-stitching algorithm by combining the optimal seam and an improved gradual fusion method[J]. Infrared Technology,2018,40(4):382-387.