局部特征引导标签平滑与优化的井下弱特征人员重识别

张杰; 缪小然; 赵作鹏; 胡建峰; 闵冰冰; 高宇蒙

doi:10.13272/j.issn.1671-251x.2023080092

局部特征引导标签平滑与优化的井下弱特征人员重识别

doi: 10.13272/j.issn.1671-251x.2023080092

1.
冀中能源股份有限公司邢东矿，河北邢台　054000
2.
中国矿业大学计算机科学与技术学院，江苏徐州　221006

基金项目: 国家自然科学基金资助项目（61976217）。

详细信息

作者简介:
张杰（1968—），男，河北定兴人，高级工程师，主要从事煤矿安全与智能化方面的工作，E-mail：13303193615@163.com

中图分类号: TD672
计量
- 文章访问数: 113
- HTML全文浏览量: 38
- PDF下载量: 7
- 被引次数: 0
出版历程
- 收稿日期: 2023-08-26
- 修回日期: 2024-02-27
- 网络出版日期: 2024-03-06

Local feature-guided label smoothing and optimization for re-identification of underground personnel with weak features

1.
Xingdong Mine, Jizhong Energy Co., Ltd., Xingtai 054000, China
2.
School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221006, China

摘要

摘要: 煤矿井下低照度、强光扰、高粉尘等环境条件，以及井下人员服装的相似性和脸部落煤现象，导致井下弱特征人员重识别困难。现有人员重识别方法仅提取全局特征，未充分考虑局部特征，使得井下人员重识别准确率较低。针对上述问题，提出了一种局部特征引导标签平滑与优化的井下弱特征人员重识别方法。该方法首先通过卷积神经网络提取井下人员图像的全局特征与局部特征；然后利用k最近邻相似性计算全局特征和局部特征的互补性得分，来衡量全局特征和局部特征的相似程度；最后根据特征互补性得分对局部特征进行标签平滑及对全局特征进行标签优化，即动态调整每个局部特征的权重，以改进每个局部特征的标签，并对局部特征的预测结果进行汇总，利用更可靠的信息来完善标签以作为全局特征的标签，从而减少图像噪声并增强特征识别能力。实验结果表明，该方法在公开数据集和包含井下人员图像的自建数据集上的平均精度均值（mAP）、第一匹配正确率（Rank−1）和平均逆置负样本惩罚率（mINP）总体优于主流人员重识别方法，具有良好的泛化性和鲁棒性，能有效实现井下弱特征人员重识别。
- 人员重识别 /
- 弱特征人员 /
- 局部特征 /
- 标签平滑 /
- 标签优化 /
- 特征互补性
Abstract: The low light, strong light disturbance, high dust and other environmental conditions underground in coal mines, as well as the similarity of clothing and coal falling on the face of underground personnel, make it difficult to re identify underground personnel with weak features. The existing personnel re identification methods only extract global features and do not fully consider local features, resulting in low accuracy of underground personnel re identification. In order to solve the above problems, a local feature guided label smoothing and optimization method for re-identification of underground personnel with weak features is proposed. This method first extracts global and local features of underground personnel images through convolutional neural networks. Secondly, the k-nearest neighbor similarity is used to calculate the complementarity score between global and local features, in order to measure the degree of similarity between global and local features. Finally, based on the score of feature complementarity, label smoothing is performed on local features and label optimization is performed on global features. The weight of each local feature is dynamically adjusted to improve the label of each local feature. The prediction results of local features are summarized. The more reliable information is used to improve the label as a global feature label, thereby reducing image noise and enhancing feature identification capability. The experimental results show that the method outperforms mainstream personnel re identification methods in terms of mean average precision (mAP), rank-1 accuracy (Rank-1), and mean inverse negative penalty (mINP) on both publicly available datasets and self built datasets containing images of underground personnel. It has good generalization and robustness, and can effectively achieve underground weak feature personnel re identification.
- personnel re identification /
- personnel with weak features /
- local features /
- smooth labels /
- label optimization /
- feature complementarity

HTML全文

图 1 局部特征引导标签平滑与优化的井下弱特征人员重识别方法原理

Figure 1. Principle of local feature-guided label smoothing and optimization for re-identification of underground personnel with weak features

下载: 全尺寸图片幻灯片

图 2 井下人员重识别可视化结果

Figure 2. Visualization results of underground personnel re-identification

下载: 全尺寸图片幻灯片

图 3 实际场景下人员重识别结果

Figure 3. Result of personnel re-identification in actual scenarios

下载: 全尺寸图片幻灯片

表 1 消融实验结果

Table 1. Ablation experimental results

%
方法	CoalReID			Market1501			MSMT17
方法	mAP	Rank−1	mINP	mAP	Rank−1	mINP	mAP	Rank−1	mINP
AGW	86.1	90.7	63.8	87.8	95.1	65.0	69.3	78.3	52.7
AGW+标签优化	89.8	93.4	66.1	91.3	97.0	68.2	74.7	80.6	55. 6
AGW+标签平滑	88.6	93.1	65.8	90.8	96.8	67.7	75.3	83.8	56.7
AGW+标签优化+标签平滑	93.1	97.3	69.1	95.2	98.6	70.3	83.1	86.8	59.8

下载: 导出CSV

表 2 不同方法在各数据集上的性能对比

Table 2. Performance comparison of different methods on various datasets

%
方法	CoalReID			Market1501			MSMT17
方法	mAP	Rank−1	mINP	mAP	Rank−1	mINP	mAP	Rank−1	mINP
AGW	86.1	90.7	63.8	87.8	95.1	65.0	49.3	68.3	14.7
RGT&RGPG	88.3	93.4	65.3	95.6	96.9	70.3	65.9	86.2	51.3
SOLIDER	83.5	88.6	60.9	95.6	96.7	71.2	86.5	91.7	60.1
BPBreID	79.3	85.3	59.8	95.3	96.4	71.0	73.2	87.3	56.2
UniHCP	84.2	83.1	58.3	90.3	95.8	66.4	67.3	79.3	60.8
st−ReID	87.3	92.9	65.1	95.5	98.0	68.9	71.2	87.1	56.3
LDS	91.2	95.3	66.8	94.9	96.1	68.3	79.1	88.3	58.4
本文方法	93.1	97.3	69.1	95.2	98.6	70.3	83.1	86.8	59.8

下载: 导出CSV

表 3 不同方法在仅包含井下人员图像的自建数据集CoalReID上的性能对比

Table 3. Performance comparison of different methods on self-built CoalReID dataset containing only underground personnel images

%
方法	mAP	Rank−1	mINP
AGW	79.3	83.5	50.1
RGT&RGPG	79.6	84.1	55.3
SOLIDER	82.4	85.1	58.3
BPBreID	73.2	79.1	55.8
UniHCP	80.3	84.2	60.1
st−ReID	85.3	86.9	60.3
LDS	84.2	85.3	59.8
本文方法	90.1	93.3	68.4

下载: 导出CSV

参考文献(26)

[1]	易能. 基于深度学习的双目视觉煤矿井下人员定位研究[D]. 徐州:中国矿业大学,2023. YI Neng. Research on binocular vision underground personnel positioning in coal mine based on deep learning[D]. Xuzhou:China University of Mining and Technology,2023.
[2]	侯瑞兵,常虹,马丙鹏,等. 基于时序多尺度互补特征的视频行人重识别[J]. 计算机学报,2023,46(1):31-50. HOU Ruibing,CHANG Hong,MA Bingpeng,et al. Temporal multi-scale complementary feature for video person re-identification[J]. Chinese Journal of Computers,2023,46(1):31-50.
[3]	王素玉,肖塞. 行人重识别研究综述[J]. 北京工业大学学报,2022,48(10):1100-1112. WANG Suyu,XIAO Sai. Review of person re-identification[J]. Journal of Beijing University of Technology,2022,48(10):1100-1112.
[4]	DALAL N,TRIGGS B. Histograms of oriented gradients for human detection[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition,San Diego,2005:886-893.
[5]	LOWE D G. Object recognition from local scale-invariant features[C]. IEEE International Conference on Computer Vision,Kerkyra,1999:1150-1157.
[6]	LIAO Shengcai,HU Yang,ZHU Xiangyu,et al. Person re-identification by local maximal occurrence representation and metric learning[C]. IEEE Conference on Computer Vision and Pattern Recognition,Boston,2015:2197-2206.
[7]	MARTIN K,HIRZER M,WOHLHART P,et al. Large scale metric learning from equivalence constraints[C]. IEEE Conference on Computer Vision and Pattern Recognition,Providence,2012:2288-2295.
[8]	YI Dong,LEI Zhen,LIAO Shengcai,et al. Deep metric learning for person re-identification[C]. The 22nd International Conference on Pattern Recognition,Stockholm,2014:34-39.
[9]	CHEN Yanbei,ZHU Xiatian,GONG Shaogang. Person re-identification by deep learning multi-scale representations[C]. IEEE International Conference on Computer Vision Workshops,Venice,2017:2590-2600.
[10]	LUO Hao,GU Youzhi,LIAO Xingyu,et al. Bag of tricks and a strong baseline for deep person re-identification[C]. IEEE Conference on Computer Vision and Pattern Recognition,Long Beach,2019:580-587.
[11]	YE Mang,SHEN Jianbing,LIN Gaojie,et al. Deep learning for person re-identification:a survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(6):2872-2893. doi: 10.1109/TPAMI.2021.3054775
[12]	孙彦景,魏力,张年龙,等. 联合DD−GAN和全局特征的井下人员重识别方法[J]. 西安电子科技大学学报,2021,48(5):201-211. SUN Yanjing,WEI Li,ZHANG Nianlong,et al. Person re-identification method combining the DD-GAN and global feature in a coal mine[J]. Journal of Xidian University,2021,48(5):201-211.
[13]	丁嘉婕. 基于质量增强和注意力机制的井下行人重识别算法[D]. 徐州:中国矿业大学,2022. DING Jiajie. Mine person re-identification algorithm based on quality enhancement and attention mechanism[D]. Xuzhou:China University of Mining and Technology,2022.
[14]	张立亚,王寓,郝博南. 基于改进度量学习的煤矿井下行人重识别方法研究[J]. 工矿自动化,2023,49(9):84-89,166. ZHANG Liya,WANG Yu,HAO Bonan. Research on personnel re-recognition method in coal mine underground based on improved metric learning[J]. Journal of Mine Automation,2023,49(9):84-89,166.
[15]	熊炜,杨荻椿,熊子婕,等. 基于全局特征拼接的行人重识别算法研究[J]. 计算机应用研究,2021,38(1):316-320. XIONG Wei,YANG Dichun,XIONG Zijie,et al. Person re-identification algorithm based on global feature stitching[J]. Application Research of Computers,2021,38(1):316-320.
[16]	PEREYRA G,TUCKER G,CHOROWSKI J,et al. Regularizing neural networks by penalizing confident output distributions[EB/OL]. [2023-08-20]. http://arxiv.org/abs/1701.06548.
[17]	DENG Jia,DONG Wei,SOCHER R,et al. ImageNet:a large-scale hierarchical image database[C]. IEEE Conference on Computer Vision and Pattern Recognition,Miami,2009:242-252.
[18]	HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016:770-778.
[19]	SHORTEN C,KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning[J]. Journal of Big Data,2019,6(1):532-540.
[20]	ZHANG Zijun. Improved Adam optimizer for deep neural networks[C]. IEEE/ACM 26th International Symposium on Quality of Service,Banff,2018:1320-1331.
[21]	GONG Yunpeng,HUANG Liqing,CHEN Lifei. Eliminate deviation with deviation for data augmentation and a general multi-modal data learning method[EB/OL]. [2023-08-20]. https://arxiv.org/abs/2101.08533v5.
[22]	CHEN Weihua,XU Xianzhe,JIA Jian,et al. Beyond appearance:a semantic controllable self-supervised learning framework for human-centric visual tasks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:15050-15061.
[23]	SOMERS V,VLEESCHOUWER C D. Body part-based representation learning for occluded person re-identification[C]. IEEE/CVF Winter Conference on Applications of Computer Vision,Waikoloa,2023:1613-1623.
[24]	CI Yuanzheng,WANG Yizhou,CHEN Meilin,et al. UniHCP:a unified model for human-centric perceptions[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,2023:17840-17852.
[25]	WANG Guangcong,LAI Jianhuang,HUANG Peigen,et al. Spatial-temporal person re-identification[C]. AAAI Conference on Artificial Intelligence,Honolulu,2019:8933-8940.
[26]	ZANG Xianghao,LI Ge,GAO Wei,et al. Learning to disentangle scenes for person re-identification[J]. Image and Vision Computing,2021,116. DOI: 10.1016/j.imavis.2021.104330.