Research on super-resolution reconstruction of mine images
-
摘要: 受井下粉尘大、照度低等环境影响，矿井图像存在分辨率低、细节模糊等问题，现有的图像超分辨率重建算法应用于矿井图像时，难以获取不同尺度图像信息、网络参数过大而影响重建速度，且重建图像易出现细节丢失、边缘轮廓模糊、伪影等问题。提出了一种基于多尺度密集通道注意力超分辨率生成对抗网络（SRGAN）的矿井图像超分辨率重建算法。设计了多尺度密集通道注意力残差块替代SRGAN原有的残差块，采用2路并行且卷积核大小不同的密集连接块，可充分获取图像特征；融入高效通道注意力模块，加强对高频信息的关注度；采用深度可分离卷积对网络进行轻量化，抑制网络参数的增加；利用纹理损失约束网络训练，避免网络加深时产生伪影。在井下数据集和公共数据集上对提出的矿井图像超分辨率重建算法和经典超分辨率重建算法BICUBIC，SRCNN，SRRESNET，SRGAN进行实验，结果表明：所提算法在主客观评价上总体优于对比算法，网络参数较SRGAN减少了2.54%，峰值信噪比与结构相似度较经典算法指标均值分别提高了0.764 dB和0.053 58，能更好地关注图像的纹理、轮廓等细节信息，重建图像更符合人眼视觉。
-
关键词:
- 矿井图像 /
- 超分辨率重建 /
- 超分辨率生成对抗网络 /
- 多尺度密集通道注意力残差块 /
- 高效通道注意力模块 /
- 深度可分离卷积 /
- 纹理损失
Abstract: Due to the impact of high dust and low illumination in underground environments, mine images have problems such as low resolution and blurry details. When existing image super-resolution reconstruction algorithms are applied to mine images, it is difficult to obtain image information at different scales. The network parameters are too large, which affects the reconstruction speed. The reconstructed images are prone to problems such as detail loss, blurry edge contours, and artifacts. A mine image super-resolution reconstruction algorithm based on multi-scale dense channel attention super-resolution generative adversarial network (SRGAN) is proposed. A multi-scale dense channel attention residual block is designed to replace the original residual block of SRGAN. Two parallel dense connected blocks with different convolutional kernel sizes are used to fully obtain image features. The efficient channel attention modules are integrated to enhance attention to high-frequency information. The depthwise separable convolution is used to lighten the network and suppress the increase of network parameters. The texture loss constraint network training is utilized to avoid artifacts during network deepening. Experiments are conducted on the proposed mine image super-resolution reconstruction algorithm and classic super-resolution reconstruction algorithms BICUBIC, SRCNN, SRRESNET, SRGAN on both underground and public datasets. The results show that the proposed algorithm outperformed the comparative algorithm in both subjective and objective evaluations. Compared to SRGAN, the proposed algorithm reduces network parameters by 2.54%. Compared to the average index values of the classic algorithms, the peak signal-to-noise ratio and structural similarity of the proposed algorithm increase by 0.764 dB and 0.053 58 respectively. It can better focus on the texture, contour and other details of the image, and the reconstructed image is more in line with human vision. -
图 1 改进的SRGAN生成器
Figure 1. The generator of the improved super-resolution generative adversarial network （SRGAN）
图 2 MDRCAB结构
Figure 2. Structure of multi-scale dense residual channel attention block （MDRCAB）
图 6 场景1各算法重建4倍的效果对比
Figure 6. The comparison of 4 times reconstruction of different super-resolution reconstruction algorithms in Scene One
图 7 场景2各算法重建4倍的效果对比
Figure 7. The comparison of 4 times reconstruction of different super-resolution reconstruction algorithms in Scene Two
图 10 各算法对基准数据集的重建效果
Figure 10. The reconstruction effect of different super-resolution reconstruction algorithms in the common data sets
表 1 不同超分辨率重建算法的客观指标对比
Table 1. The comparison of objective indexes of different super-resolution reconstruction algorithms
算法 PSNR/dB SSIM BICUBIC 23.889 0.644 85 SRCNN 26.345 0.834 61 SRRESNET 26.432 0.833 73 SRGAN 26.385 0.830 81 本文算法 26.527 0.839 58 
下载: 导出CSV
表 2 各算法在公共数据集上的PSNR和 SSIM 对比（缩放因子为4）
Table 2. The comparison of PSNR and SSIM of different super-resolution reconstruction algorithms in common data sets （scaling factor is 4）
算法 Set5 Set14 BSD100 Urban100 PSNR/dB SSIM PSNR/dB SSIM PSNR/dB SSIM PSNR/dB SSIM BICUBIC 20.216 0.544 33 19.592 0.471 52 20.271 0.445 42 18.582 0.425 02 SRCNN 27.769 0.807 65 25.113 0.711 13 25.199 0.683 31 22.384 0.670 02 SRRESNET 28.071 0.830 06 25.385 0.727 89 25.481 0.694 45 23.011 0.701 56 SRGAN 27.983 0.818 03 25.369 0.719 52 25.337 0.689 02 22.857 0.693 85 本文算法 28.366 0.832 21 25.594 0.729 29 25.505 0.694 96 22.948 0.701 32 注：粗体为各列最优，下划线为各列次优。
下载: 导出CSV
表 3 不同优化策略的消融实验结果对比
Table 3. The comparison of ablation results of different optimization strategies
模型 多尺度密
集连接DSC ECA 纹理
损失PSNR/dB SSIM 参数量/个 a 26.385 0.830 81 734 219 b √ 26.471 0.836 02 1 656 336 c √ √ 26.464 0.835 13 715 536 d √ √ 26.530 0.840 73 1 656 351 e √ √ √ 26.523 0.838 60 715 551 f √ √ √ √ 26.527 0.839 58 715 551
下载: 导出CSV
-
[1] CHENG Deqiang,CHEN Liangliang,LYU Chen,et al. Light-guided and cross-fusion U-Net for anti-illumination image super-resolution[J]. IEEE Transactions on Circuits and Systems for Video Technology,2022,32(12):8436-8449. doi: 10.1109/TCSVT.2022.3194169 [2] CHEN Liangliang,GUO Lin,CHENG Deqiang,et al. Structure-preserving and color-restoring up-sampling for single low-light image[J]. IEEE Transactions on Circuits and Systems for Video Technology,2022,32(4):1889-1902. doi: 10.1109/TCSVT.2021.3086598 [3] JIANG Nan,WANG Luo. Quantum image scaling using nearest neighbor interpolation[J]. Quantum Information Processing,2015,14(5):1559-1571. doi: 10.1007/s11128-014-0841-8 [4] LI Xin,ORCHARD M T. New edge-directed interpolation[J]. IEEE Transactions on Image Processing,2001,10(10):1521-1527. doi: 10.1109/83.951537 [5] 程相正,曾朝阳,陈杭,等. 基于双线性插值算法的低分辨率传感器标定方法[J]. 激光与光电子学进展,2013,50(7):72-78.CHENG Xiangzheng,ZENG Zhaoyang,CHEN Hang,et al. Calibration method of low-resolution sensor based on bilinear interpolation strategy[J]. Laser & Optoelectronics Progress,2013,50(7):72-78. [6] ZHU Yubin,DAI Yonghang,HAN Kaining,et al. An efficient BICUBIC interpolation implementation for real-time image processing using hybrid computing[J]. Journal of Real-Time Image Processing,2022,19(6):1211-1223. doi: 10.1007/s11554-022-01254-8 [7] NASONOV A V,KRYLOV A. Fast super-resolution using weighted median filtering[C]. 20th International Conference on Pattern Recognition,Istanbul,2010:2230-2233. [8] HUANG Jin,GAO Bo,CHEN Yan,et al. Super-resolution reconstruction of multi-polarization sar images based on projections onto convex sets algorithm[C]. IEEE International Geoscience and Remote Sensing Symposium,Valencia,2018:8456-8459. [9] 郭黎,廖宇,陈为龙,等. 基于最大后验概率的单视频时间超分辨率重建算法[J]. 计算机应用,2014,34(12):3580-3584.GUO Li,LIAO Yu,CHEN Weilong,et al. Single video temporal super-resolution reconstruction algorithm based on maximum a posterior[J]. Journal of Computer Applications,2014,34(12):3580-3584. [10] FREEMAN W T,JONES T R,PASZTOR E C. Example based super-resolution[J]. IEEE Computer Graphics and Applications,2002,22(2):56-65. doi: 10.1109/38.988747 [11] GLASNER D,BAGON S,IRANI M. Super-resolution from a single image[C]. IEEE 12th International Conference on Computer Vision,Kyoto,2009:349-356. [12] 梁美彦,张宇,梁建安,等. 基于稀疏编码非局部注意力对偶网络的病理图像超分辨率重建[J]. 计算机科学,2023,50(增刊1):305-312.LIANG Meiyan,ZHANG Yu,LIANG Jian'an,et al. Pathological image super-resolution reconstruction based on sparse coding non-local attention dual network[J]. Computer Science,2023,50(S1):305-312. [13] 杨雪,李峰,鹿明,等. 混合稀疏表示模型的超分辨率重建[J]. 遥感学报,2022,26(8):1685-1697. doi: 10.11834/jrs.20219409YANG Xue,LI Feng,LU Ming,et al. New super-resolution reconstruction method based on mixed sparse representations[J]. National Remote Sensing Bulletin,2022,26(8):1685-1697. doi: 10.11834/jrs.20219409 [14] DONG Chao,LOY C C,HE Kaiming,et al. Learning a deep convolutional network for image super- resolution[C]. 13th European Conference on Computer Vision,Zurich,2014:184-199. [15] DONG Chao,LOY C C,TANG Xiaoou,et al. Accelerating the super-resolution convolutional neural network[C]. 14th European Conference on Computer Vision,Amsterdam,2016:391-407. [16] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutional networks[C]. IEEE Conference on Computer Visio and Pattern Recognition,Las Vegas,2016:1646-1654. [17] ZHANG Yulun,LI Kunpeng,LI Kai,et al. Image super-resolution using very deep residual channel attention networks[C]. 15th European Conference on Computer Vision,Munich,2018:294-310. [18] 程德强,赵佳敏,寇旗旗,等. 多尺度密集特征融合的图像超分辨率重建[J]. 光学精密工程,2022,30(20):2489-2500. doi: 10.37188/OPE.20223020.2489CHENG Deqiang,ZHAO Jiamin,KOU Qiqi,et al. Multi-scale dense feature fusion network for image super-resolution[J]. Optics and Precision Engineering,2022,30(20):2489-2500. doi: 10.37188/OPE.20223020.2489 [19] LEDIG C,THEIS L,HUSZAR F,et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,2017:105-114. [20] 程德强,陈亮亮,蔡迎春,等. 边缘融合的多字典超分辨率图像重建算法[J]. 煤炭学报,2018,43(7):2084-2090.CHENG Deqiang,CHEN Liangliang,CAI Yingchun,et al. Image super-resolution reconstruction based on multi-dictionary and edge fusion[J]. Journal of China Coal Society,2018,43(7):2084-2090. [21] 张剑英,宋玉龙,蔡迎春,等. 改进的非局部均值视频超分辨率重建算法[J]. 工矿自动化,2018,44(9):37-44.ZHANG Jianying,SONG Yulong,CAI Yingchun,et al. Improved super-resolution reconstruction algorithm of non-local mean video[J]. Industry and Mine Automation,2018,44(9):37-44. [22] 汪海涛,于文洁,张光磊. 基于在线多字典学习的矿井图像超分辨率重建方法[J]. 工矿自动化,2020,46(9):74-78.WANG Haitao,YU Wenjie,ZHANG Guanglei. Super-resolution reconstruction method of mine image based on online multi-dictionary learning[J]. Industry and Mine Automation,2020,46(9):74-78. [23] 程德强,陈杰,寇旗旗,等. 融合层次特征和注意力机制的轻量化矿井图像超分辨率重建方法[J]. 仪器仪表学报,2022,43(8):73-84.CHENG Deqiang,CHEN Jie,KOU Qiqi,et al. Lightweight super-resolution reconstruction method based on hierarchical features fusion and attention mechanism for mine image[J]. Chinese Journal of Scientific Instrument,2022,43(8):73-84. [24] WANG Qilong,WU Banggu,ZHU Pengfei,et al. ECA-Net:efficient channel attention for deep convolutional neural networks [C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,2020:11531-11539. [25] 姜玉宁,李劲华,赵俊莉. 基于生成式对抗网络的图像超分辨率重建算法[J]. 计算机工程,2021,47(3):249-255.JIANG Yuning,LI Jinhua,ZHAO Junli. Image super-resolution reconstruction algorithm based on generative adversarial networks[J]. Computer Engineering,2021,47(3):249-255. [26] 张磊,王浩盛,雷伟强,等. 基于YOLOv5s−SDE的带式输送机煤矸目标检测[J]. 工矿自动化,2023,49(4):106-112.ZHANG Lei,WANG Haosheng,LEI Weiqiang,et al. Coal gangue target detection of belt conveyor based on YOLOv5s-SDE[J]. Journal of Mine Automation,2023,49(4):106-112. [27] HORÉ A,ZIOU D. Image quality metrics:PSNR vs SSIM[C]. International Conference on Pattern Recognition,Istanbul,2010:2366-2369. [28] WANG Zhou,BOVIK A C,SHEIKH H R,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing,2004,13(4):600-612. doi: 10.1109/TIP.2003.819861 -
点击查看大图
图(10) / 表(3)
计量
- 文章访问数: 229
- HTML全文浏览量: 61
- PDF下载量: 60
- 被引次数: 0
