A method for enhancing low light images in coal mines based on Retinex model containing noise
-
摘要: 低光照图像会导致许多计算机 视觉任务达不到预期效果，影响后续图像分析与智能决策。针对现有煤矿井下低光照图像增强方法未考虑图像现实噪声的问题，提出一种基于含噪Retinex模型的煤矿低光照图像增强方法。建立了含噪Retienx模型，利用噪声估计模块（NEM）估计现实噪声，将原图像和估计噪声作为光照分量估计模块（IEM）和反射分量估计模块（REM）的输入，生成光照分量与反射分量并对二者进行耦合，同时对光照分量进行伽马校正等调整，对耦合后的图像及调整后的光照分量进行除法运算，得到最终的增强图像。NEM通过3层CNN对含噪图像进行拜耳采样，然后重构生成与原图像大小一致的三通道特征图。IEM与REM均以ResNet−34作为图像特征提取网络，引入多尺度非对称卷积与注意力模块（MACAM），以增强网络的细节过滤能力及重要特征筛选能力。定性和定量评估结果表明，该方法能够平衡光源与黑暗环境之间的关系，降低现实噪声的影响，在图像自然度、真实度、对比度、结构等方面均具有良好性能，图像增强效果优于Retinex−Net，Zero−DCE，DRBN，DSLR，TBEFN，RUAS等模型。通过消融实验验证了NEM与MACAM的有效性。Abstract: The low light images can lead to many computer vision tasks not achieving the expected results. This can affect subsequent image analysis and intelligent decision-making. The existing low light image enhancement methods for underground coal mines do not consider the real noise of the image. In order to solve this problem, a method for enhancing low light images in coal mines based on Retinex model containing noise is proposed. The Retienx model containing noise is established. The noise estimation module (NEM) is used to estimate real noise. The original image and estimated noise are used as inputs to the illumination component estimation module (IEM) and reflection estimation module (REM) to generate and couple the illumination and reflection components. At the same time, gamma correction and other adjustments are made to the illumination components. And division operations are performed on the coupled image and adjusted illumination components to obtain the final enhanced image. NEM uses a three-layer CNN to perform Bayer sampling on noisy images. It reconstructs them to generate a three channel feature map which is the same size as the original image. Both IEM and REM use ResNet-34 as the image feature extraction network. The multi-scale asymmetric convolution and attention module (MACAM) is introduced to enhance the network's capability to filter details and important features. The qualitative and quantitative evaluation results indicate that this method can balance the relationship between light sources and dark environments, reduce real-world noise's impact, and perform well in image naturalness, realism, contrast, structure, and other aspects. The image enhancement effect is superior to models such as Retinex-Net, Zero-DCE, DRBN, DSLR, TBEFN, RUAS, etc. The effectiveness of NEM and MACAM is verified through ablation experiments.
-
图 6 煤矿不同场景下6种模型与本文方法的对比
Figure 6. Comparison between six models and the method presented in this paper
图 7 多光源、逆光条件下巷道图像增强效果对比
Figure 7. Comparison of image enhancement effects in roadway under multiple light sources and backlight conditions
表 1 不同模型客观评价结果
Table 1. Objective evaluation results of different models
图像集 模型 NIQE NIQMC PSNR SSIM 矿井图像 Retinex−Net 3.37 4.88 14.40 0.59 Zero−DCE 3.62 4.67 15.51 0.58 DRBN 3.54 4.98 15.32 0.70 DSLR 3.68 5.26 13.93 0.49 TBEFN 3.57 5.44 17.14 0.76 RUAS 3.43 5.18 18.32 0.72 本文 3.30 5.63 18.5 0.74 矿井设备图像 Retinex−Net 3.42 4.64 13.09 0.57 Zero−DCE 3.69 4.83 14.58 0.55 DRBN 3.50 5.03 15.66 0.66 DSLR 3.66 5.64 15.38 0.78 TBEFN 3.60 5.40 17.42 0.42 RUAS 3.52 5.29 18.55 0.70 本文 3.28 5.78 18.03 0.77 巷道图像 Retinex−Net 3.53 4.66 13.88 0.56 Zero−DCE 3.76 4.83 13.56 0.54 DRBN 3.53 4.95 15.32 0.68 DSLR 3.60 5.18 14.95 0.73 TBEFN 3.42 5.26 17.83 0.69 RUAS 3.59 5.55 18.66 0.71 本文 3.33 5.83 18.92 0.80 多光源场景图像 Retinex−Net 3.40 4.03 13.58 0.52 Zero−DCE 3.54 4.62 13.04 0.59 DRBN 3.60 5.33 15.42 0.61 DSLR 3.33 5.19 13.11 0.40 TBEFN 3.45 4.86 17.64 0.64 RUAS 3.58 5.05 18.22 0.72 本文 3.25 5.86 19.01 0.77 
下载: 导出CSV
表 2 消融实验结果
Table 2. Results of ablation experiment
模型 NIQE NIQMC PSNR SSIM ResNet 3.35 4.43 14.02 0.51 ResNet +NEM 3.30 4.96 16.71 0.57 ResNet +MACAM 3.26 5.27 17.83 0.62 ResNet +NEM +MACAM 3.22 5.62 18.82 0.70
下载: 导出CSV
-
[1] 赵谦. 煤矿井下动态目标视频监测图像处理研究[D]. 西安: 西安科技大学, 2014.ZHAO Qian. Study on video monitoring and image processing of coal mine dynamic targets[D]. Xi'an: Xi'an University of Science and Technology, 2014. [2] 孙继平. 煤矿安全生产监控与通信技术[J]. 煤炭学报,2010,35(11):1925-1929.SUN Jiping. Technologies of monitoring and communication in the coal mine[J]. Journal of China Coal Society,2010,35(11):1925-1929. [3] 孙继平. 煤矿事故分析与煤矿大数据和物联网[J]. 工矿自动化,2015,41(3):1-5.SUN Jiping. Accident analysis and big data and Internet of things in coal mine[J]. Industry and Mine Automation,2015,41(3):1-5. [4] 孙继平,杜东璧. 基于随机特征的矿井视频图像中的人员跟踪技术[J]. 煤炭科学技术,2015,43(11):91-94.SUN Jiping,DU Dongbi. Tracing technology of personnel in mine video images based on random features[J]. Coal Science and Technology,2015,43(11):91-94. [5] 张谢华,张申,方帅,等. 煤矿智能视频监控中雾尘图像的清晰化研究[J]. 煤炭学报,2014,39(1):198-204. doi: 10.13225/j.cnki.jccs.2013.0150ZHANG Xiehua,ZHANG Shen,FANG Shuai,et al. Clearing research on fog and dust images in coal mine intelligent video surveillance[J]. Journal of China Coal Society,2014,39(1):198-204. doi: 10.13225/j.cnki.jccs.2013.0150 [6] 王殿伟,韩鹏飞,范九伦,等. 基于光照−反射成像模型和形态学操作的多谱段图像增强算法[J]. 物理学报,2018,67(21):104-114.WANG Dianwei,HAN Pengfei,FAN Jiulun,et al. Multispectral image enhancement based on illuminance-reflection imaging model and morphology operation[J]. Acta Physica Sinica,2018,67(21):104-114. [7] 何畏. 基于改进直方图的低照度图像增强算法[J]. 计算机科学,2015,42(增刊1):241-242,262.HE Wei. Low-light image enhancement based on improve histogram[J]. Computer Science,2015,42(S1):241-242,262. [8] ZUO Chao,CHEN Qian,SUI Xiubao. Range limited bi-histogram equalization for image contrast enhancement[J]. Optik-International Journal for Light and Electron Optics,2013,124(5):425-431. doi: 10.1016/j.ijleo.2011.12.057 [9] 刘晓阳,乔通,乔智. 基于双边滤波和Retinex算法的矿井图像增强方法[J]. 工矿自动化,2017,43(2):49-54.LIU Xiaoyang,QIAO Tong,QIAO Zhi. Image enhancement method of mine based on bilateral filtering and Retinex algorithm[J]. Industry and Mine Automation,2017,43(2):49-54. [10] 范凌云,梁修荣. 基于小波分解子带直方图匹配的矿井视频图像增强方法[J]. 金属矿山,2016(6):130-133.FAN Lingyun,LIANG Xiurong. Mine video images enhancement method based on the histogram matching method of the sub-bands of wavelet transform[J]. Metal Mine,2016(6):130-133. [11] 程德强,郑珍,姜海龙. 一种煤矿井下图像增强算法[J]. 工矿自动化,2015,41(12):31-34.CHENG Deqiang,ZHENG Zhen,JIANG Hailong. An image enhancement algorithm for coal mine underground[J]. Industry and Mine Automation,2015,41(12):31-34. [12] 智宁,毛善君,李梅. 基于照度调整的矿井非均匀照度视频图像增强算法[J]. 煤炭学报,2017,42(8):2190-2197.ZHI Ning,MAO Shanjun,LI Mei. Enhancement algorithm based on illumination adjustment for non-uniform illuminance video images in coal mine[J]. Journal of China Coal Society,2017,42(8):2190-2197. [13] LYU Feifan, LU Feng, WU Jianhua, et al. MBLLEN: low-light image/video enhancement using CNNs[C]. British Machine Vision Conference, Newcastle, 2018: 220-233. [14] WANG Yang, CAO Yang, ZHA Zhengjun, et al. Progressive retinex: mutually reinforced illumination-noise perception network for low light image enhancement[C]. Proceedings of the 27th ACM International Conference on Multimedia, Nice, 2019: 2015-2023. [15] FAN Minhao, WANG Wenjing, YANG Wenhan, et al. Integrating semantic segmentation and Retinex model for low-light image enhancement[C]. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, 2020: 2317-2325. [16] 樊占文,刘波. 基于改进的Retinex低照度图像自适应增强技术研究[J]. 工矿自动化,2021,47(增刊1):126-130.FAN Zhanwen,LIU Bo. Research on adaptive enhancement technology of low illumination image based on improved Retinex[J]. Industry and Mine Automation,2021,47(S1):126-130. [17] ZHAO Zunjin,XIONG Bangshu,WANG Lei,et al. RetinexDIP:a unified deep framework for low-light image enhancement[J]. IEEE Transactions on Circuits and Systems for Video Technology,2022,32(3):1076-1088. doi: 10.1109/TCSVT.2021.3073371 [18] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 2818-2826. [19] DING Xiaohan, GUO Yuchen, DING Guiguang, et al. ACNet: strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks[C]. IEEE/CVF International Conference on Computer Vision, Seoul, 2019: 1911-1920. [20] WEI Chen, WANG Wenjing, YANG Wenhan, et al. Deep Retinex decomposition for low-light enhancement[C]. British Machine Vision Conference, 2018. [21] GUO Chunle, LI Chongyi, GUO Jichang, et al. Zero-reference deep curve estimation for low-light image enhancement[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020: 1777-1786. [22] YANG Wenhan, WANG Shiqi, FANG Yuming, et al. From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020: 3060-3069. [23] LIM S,KIM W. DSLR:deep stacked laplacian restorer for low-light image enhancement[J]. IEEE Transactions on Multimedia,2020,23:4272-4284. [24] LU Kun,ZHANG Lihong. TBEFN:a two-branch exposure-fusion network for low-light image enhancement[J]. IEEE Transactions on Multimedia,2020,23:4093-4105. [25] LIU Risheng, MA Long, ZHANG Jia'ao, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 2021. [26] MITTAL A,SOUNDARARAJAN R,BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters,2013,20(3):209-212. doi: 10.1109/LSP.2012.2227726 [27] GU Ke,LIN Weisi,ZHAI Guangtao,et al. No-reference quality metric of contrast-distorted images based on information maximization[J]. IEEE Transactions on Cybernetics,2017,47(12):4559-4565. doi: 10.1109/TCYB.2016.2575544 -
点击查看大图
图(7) / 表(2)
计量
- 文章访问数: 342
- HTML全文浏览量: 129
- PDF下载量: 18
- 被引次数: 0
