Local feature-guided label smoothing and optimization for re-identification of underground personnel with weak features
-
摘要: 煤矿井下低照度、强光扰、高粉尘等环境条件,以及井下人员服装的相似性和脸部落煤现象,导致井下弱特征人员重识别困难。现有人员重识别方法仅提取全局特征,未充分考虑局部特征,使得井下人员重识别准确率较低。针对上述问题,提出了一种局部特征引导标签平滑与优化的井下弱特征人员重识别方法。该方法首先通过卷积神经网络提取井下人员图像的全局特征与局部特征;然后利用k最近邻相似性计算全局特征和局部特征的互补性得分,来衡量全局特征和局部特征的相似程度;最后根据特征互补性得分对局部特征进行标签平滑及对全局特征进行标签优化,即动态调整每个局部特征的权重,以改进每个局部特征的标签,并对局部特征的预测结果进行汇总,利用更可靠的信息来完善标签以作为全局特征的标签,从而减少图像噪声并增强特征识别能力。实验结果表明,该方法在公开数据集和包含井下人员图像的自建数据集上的平均精度均值(mAP)、第一匹配正确率(Rank−1)和平均逆置负样本惩罚率(mINP)总体优于主流人员重识别方法,具有良好的泛化性和鲁棒性,能有效实现井下弱特征人员重识别。Abstract: The low light, strong light disturbance, high dust and other environmental conditions underground in coal mines, as well as the similarity of clothing and coal falling on the face of underground personnel, make it difficult to re identify underground personnel with weak features. The existing personnel re identification methods only extract global features and do not fully consider local features, resulting in low accuracy of underground personnel re identification. In order to solve the above problems, a local feature guided label smoothing and optimization method for re-identification of underground personnel with weak features is proposed. This method first extracts global and local features of underground personnel images through convolutional neural networks. Secondly, the k-nearest neighbor similarity is used to calculate the complementarity score between global and local features, in order to measure the degree of similarity between global and local features. Finally, based on the score of feature complementarity, label smoothing is performed on local features and label optimization is performed on global features. The weight of each local feature is dynamically adjusted to improve the label of each local feature. The prediction results of local features are summarized. The more reliable information is used to improve the label as a global feature label, thereby reducing image noise and enhancing feature identification capability. The experimental results show that the method outperforms mainstream personnel re identification methods in terms of mean average precision (mAP), rank-1 accuracy (Rank-1), and mean inverse negative penalty (mINP) on both publicly available datasets and self built datasets containing images of underground personnel. It has good generalization and robustness, and can effectively achieve underground weak feature personnel re identification.
-
表 1 消融实验结果
Table 1. Ablation experimental results
% 方法 CoalReID Market1501 MSMT17 mAP Rank−1 mINP mAP Rank−1 mINP mAP Rank−1 mINP AGW 86.1 90.7 63.8 87.8 95.1 65.0 69.3 78.3 52.7 AGW+标签优化 89.8 93.4 66.1 91.3 97.0 68.2 74.7 80.6 55. 6 AGW+标签平滑 88.6 93.1 65.8 90.8 96.8 67.7 75.3 83.8 56.7 AGW+标签优化+标签平滑 93.1 97.3 69.1 95.2 98.6 70.3 83.1 86.8 59.8 表 2 不同方法在各数据集上的性能对比
Table 2. Performance comparison of different methods on various datasets
% 方法 CoalReID Market1501 MSMT17 mAP Rank−1 mINP mAP Rank−1 mINP mAP Rank−1 mINP AGW 86.1 90.7 63.8 87.8 95.1 65.0 49.3 68.3 14.7 RGT&RGPG 88.3 93.4 65.3 95.6 96.9 70.3 65.9 86.2 51.3 SOLIDER 83.5 88.6 60.9 95.6 96.7 71.2 86.5 91.7 60.1 BPBreID 79.3 85.3 59.8 95.3 96.4 71.0 73.2 87.3 56.2 UniHCP 84.2 83.1 58.3 90.3 95.8 66.4 67.3 79.3 60.8 st−ReID 87.3 92.9 65.1 95.5 98.0 68.9 71.2 87.1 56.3 LDS 91.2 95.3 66.8 94.9 96.1 68.3 79.1 88.3 58.4 本文方法 93.1 97.3 69.1 95.2 98.6 70.3 83.1 86.8 59.8 表 3 不同方法在仅包含井下人员图像的自建数据集CoalReID上的性能对比
Table 3. Performance comparison of different methods on self-built CoalReID dataset containing only underground personnel images
% 方法 mAP Rank−1 mINP AGW 79.3 83.5 50.1 RGT&RGPG 79.6 84.1 55.3 SOLIDER 82.4 85.1 58.3 BPBreID 73.2 79.1 55.8 UniHCP 80.3 84.2 60.1 st−ReID 85.3 86.9 60.3 LDS 84.2 85.3 59.8 本文方法 90.1 93.3 68.4 -
[1] 易能. 基于深度学习的双目视觉煤矿井下人员定位研究[D]. 徐州:中国矿业大学,2023.YI Neng. Research on binocular vision underground personnel positioning in coal mine based on deep learning[D]. Xuzhou:China University of Mining and Technology,2023. [2] 侯瑞兵,常虹,马丙鹏,等. 基于时序多尺度互补特征的视频行人重识别[J]. 计算机学报,2023,46(1):31-50.HOU Ruibing,CHANG Hong,MA Bingpeng,et al. Temporal multi-scale complementary feature for video person re-identification[J]. Chinese Journal of Computers,2023,46(1):31-50. [3] 王素玉,肖塞. 行人重识别研究综述[J]. 北京工业大学学报,2022,48(10):1100-1112.WANG Suyu,XIAO Sai. Review of person re-identification[J]. Journal of Beijing University of Technology,2022,48(10):1100-1112. [4] DALAL N,TRIGGS B. Histograms of oriented gradients for human detection[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition,San Diego,2005:886-893. [5] LOWE D G. Object recognition from local scale-invariant features[C]. IEEE International Conference on Computer Vision,Kerkyra,1999:1150-1157. [6] LIAO Shengcai,HU Yang,ZHU Xiangyu,et al. Person re-identification by local maximal occurrence representation and metric learning[C]. IEEE Conference on Computer Vision and Pattern Recognition,Boston,2015:2197-2206. [7] MARTIN K,HIRZER M,WOHLHART P,et al. Large scale metric learning from equivalence constraints[C]. IEEE Conference on Computer Vision and Pattern Recognition,Providence,2012:2288-2295. [8] YI Dong,LEI Zhen,LIAO Shengcai,et al. Deep metric learning for person re-identification[C]. The 22nd International Conference on Pattern Recognition,Stockholm,2014:34-39. [9] CHEN Yanbei,ZHU Xiatian,GONG Shaogang. Person re-identification by deep learning multi-scale representations[C]. IEEE International Conference on Computer Vision Workshops,Venice,2017:2590-2600. [10] LUO Hao,GU Youzhi,LIAO Xingyu,et al. Bag of tricks and a strong baseline for deep person re-identification[C]. IEEE Conference on Computer Vision and Pattern Recognition,Long Beach,2019:580-587. [11] YE Mang,SHEN Jianbing,LIN Gaojie,et al. Deep learning for person re-identification:a survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(6):2872-2893. doi: 10.1109/TPAMI.2021.3054775 [12] 孙彦景,魏力,张年龙,等. 联合DD−GAN和全局特征的井下人员重识别方法[J]. 西安电子科技大学学报,2021,48(5):201-211.SUN Yanjing,WEI Li,ZHANG Nianlong,et al. Person re-identification method combining the DD-GAN and global feature in a coal mine[J]. Journal of Xidian University,2021,48(5):201-211. [13] 丁嘉婕. 基于质量增强和注意力机制的井下行人重识别算法[D]. 徐州:中国矿业大学,2022.DING Jiajie. Mine person re-identification algorithm based on quality enhancement and attention mechanism[D]. Xuzhou:China University of Mining and Technology,2022. [14] 张立亚,王寓,郝博南. 基于改进度量学习的煤矿井下行人重识别方法研究[J]. 工矿自动化,2023,49(9):84-89,166.ZHANG Liya,WANG Yu,HAO Bonan. Research on personnel re-recognition method in coal mine underground based on improved metric learning[J]. Journal of Mine Automation,2023,49(9):84-89,166. [15] 熊炜,杨荻椿,熊子婕,等. 基于全局特征拼接的行人重识别算法研究[J]. 计算机应用研究,2021,38(1):316-320.XIONG Wei,YANG Dichun,XIONG Zijie,et al. Person re-identification algorithm based on global feature stitching[J]. Application Research of Computers,2021,38(1):316-320. [16] PEREYRA G,TUCKER G,CHOROWSKI J,et al. Regularizing neural networks by penalizing confident output distributions[EB/OL]. [2023-08-20]. http://arxiv.org/abs/1701.06548. [17] DENG Jia,DONG Wei,SOCHER R,et al. ImageNet:a large-scale hierarchical image database[C]. IEEE Conference on Computer Vision and Pattern Recognition,Miami,2009:242-252. [18] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016:770-778. [19] SHORTEN C,KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning[J]. Journal of Big Data,2019,6(1):532-540. [20] ZHANG Zijun. Improved Adam optimizer for deep neural networks[C]. IEEE/ACM 26th International Symposium on Quality of Service,Banff,2018:1320-1331. [21] GONG Yunpeng,HUANG Liqing,CHEN Lifei. Eliminate deviation with deviation for data augmentation and a general multi-modal data learning method[EB/OL]. [2023-08-20]. https://arxiv.org/abs/2101.08533v5. [22] CHEN Weihua,XU Xianzhe,JIA Jian,et al. Beyond appearance:a semantic controllable self-supervised learning framework for human-centric visual tasks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:15050-15061. [23] SOMERS V,VLEESCHOUWER C D. Body part-based representation learning for occluded person re-identification[C]. IEEE/CVF Winter Conference on Applications of Computer Vision,Waikoloa,2023:1613-1623. [24] CI Yuanzheng,WANG Yizhou,CHEN Meilin,et al. UniHCP:a unified model for human-centric perceptions[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:17840-17852. [25] WANG Guangcong,LAI Jianhuang,HUANG Peigen,et al. Spatial-temporal person re-identification[C]. AAAI Conference on Artificial Intelligence,Honolulu,2019:8933-8940. [26] ZANG Xianghao,LI Ge,GAO Wei,et al. Learning to disentangle scenes for person re-identification[J]. Image and Vision Computing,2021,116. DOI: 10.1016/j.imavis.2021.104330. -