Multi-personnel underground trajectory prediction method based on Social Transformer

MA Zheng; YANG Dashan; ZHANG Tianxiang

doi:10.13272/j.issn.1671-251x.2023110084

Volume 50 Issue 5

May 2024

Turn off MathJax

Article Contents

Abstract

References

Journal of Mine Automation > 2024 > 50(5): 67-74. > DOI: 10.13272/j.issn.1671-251x.2023110084

MA Zheng, YANG Dashan, ZHANG Tianxiang. Multi-personnel underground trajectory prediction method based on Social Transformer[J]. Journal of Mine Automation，2024，50（5）：67-74. DOI: 10.13272/j.issn.1671-251x.2023110084

Citation:

PDF (40086 KB)

Multi-personnel underground trajectory prediction method based on Social Transformer

MA Zheng^{1, 2, 3, 4,},
YANG Dashan^{1, 2, 3, 4},
ZHANG Tianxiang⁵

1.
CCTEG China Coal Research Institute, Beijing 100013, China
2.
State Key Laboratory of Intelligent Coal Mining and Strata Control, Beijing 100013, China
3.
Engineering Research Center for Technology Equipment of Emergency Refuge in Coal Mine, Beijing 100013, China
4.
Beijing Engineering and Research Center of Mine Safe, Beijing 100013, China
5.
School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China

More Information

Received Date: November 25, 2023
Revised Date: May 24, 2024
Available Online: June 12, 2024

Graphical Abstract

Abstract

Abstract

Currently, in the prediction methods of underground personnel trajectories in coal mines, Transformer not only has lower computational complexity compared to recurrent neural network（RNN） and long short-term memory (LSTM), but also effectively solves the problem of long-term dependence caused by gradient disappearance when processing data. But when multi personnel are moving simultaneously in the environment, the Transformer's prediction of the future trajectories of all personnel in the scene will have a significant deviation. And currently, there is no model in the field of underground multi personnel trajectory prediction that simultaneously uses Transformer and considers the mutual influence between individuals. In order to solve the above problems, a multi personnel underground trajectory prediction method based on Social Transformer is proposed. Firstly, each individual is independently modeled to obtain their historical trajectory information. Feature extraction is performed using a Transformer encoder, followed by a fully connected layer to better represent the features. Secondly, an interactive layer based on graph convolution is used to connect each other, allowing spatially close networks to share information with each other. This layer calculates the attention that the predicted object allocates to its neighbors when influenced by them, extracts their motion patterns, and updates the feature matrix. Finally, the new feature matrix are decoded by the Transformer decoder to output predictions of future position information. The experimental results show that the average displacement error of Social Transformer is reduced by 45.8% compared to Transformer. Compared with other mainstream trajectory prediction methods such as LSTM, S-GAN, Trajectoron++, and S-STGCNN, the prediction errors are reduced by 67.1%, 35.9%, 30.1%, and 10.9%, respectively. This can effectively overcome the problem of inaccurate prediction trajectories caused by mutual influence among personnel in the underground multi personnel scenario of coal mines and improve prediction precision.
- electronic fence,
- underground multi personnel trajectory prediction,
- Transformer,
- interaction coding,
- Social Transformer

FullText(HTML)

References (21)

References

[1]	刘海忠. 电子围栏中心监控平台的设计与开发[D]. 武汉:华中师范大学,2012. LIU Haizhong. Design and development of center monitoring platform for electronic fence[D]. Wuhan:Central China Normal University,2012.
[2]	JEONG N Y,LIM S H,LIM E,et al. Pragmatic clinical trials for real-world evidence:concept and implementation[J]. Cardiovascular Pevention and Pharmacotherapy,2020,2(3):85-98. DOI: 10.36011/cpp.2020.2.e12
[3]	KLENSKE E D,ZEILINGER M N,SCHOLKOPF B,et al. Gaussian process-based predictive control for periodic error correction[J]. IEEE Transactions on Control Systems Technology,2016,24(1):110-121. DOI: 10.1109/TCST.2015.2420629
[4]	HUNT K J,SBARBARO D,ŻBIKOWSKI R,et al. Neural networks for control systems-a survey[J]. Automatica,1992,28(6):1083-1112. DOI: 10.1016/0005-1098(92)90053-I
[5]	PRESTON D B. Spectral analysis and time series[J]. Technometrics,1983,25(2):213-214. DOI: 10.1080/00401706.1983.10487866
[6]	AKAIKE H. Fitting autoregreesive models for prediction[M]//PARZEN E,TANABE K,KITAGAWA G. Selected papers of Hirotugu Akaike. New York:Springer-Verlag New York Inc,1998:131-135.
[7]	ZHANG Jianjing,LIU Hongyi,CHANG Qing,et al. Recurrent neural network for motion trajectory prediction in human-robot collaborative assembly[J]. CIRP Annals,2020,69(1):9-12. DOI: 10.1016/j.cirp.2020.04.077
[8]	SHERSTINSKY A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D:Nonlinear Phenomena,2020. DOI: 10.1016/j.physd.2019.132306.
[9]	SONG Xiao,CHEN Kai,LI Xu,et al. Pedestrian trajectory prediction based on deep convolutional LSTM network[J]. IEEE Transactions on Intelligent Transportation Systems,2020,22(6):3285-3302.
[10]	SALZMANN T,IVANOVIC B,CHAKRAVARTY P,et al. Trajectron++:dynamically-feasible trajectory forecasting with heterogeneous data[C]. 16th European Conference on Computer Vision,Glasgow,2020:683-700.
[11]	MOHAMED A,QIAN Kun,ELHOSEINY M,et al. Social-STGCNN:a social spatio-temporal graph convolutional neural network for human trajectory prediction[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,2020:14424-14432.
[12]	SHANKAR V,YOUSEFI E,MANASHTY A,et al. Clinical-GAN:trajectory forecasting of clinical events using transformer and generative adversarial networks[J]. Artificial Intelligence in Medicine,2023,138. DOI: 10.1016/j.artmed.2023.102507.
[13]	HAN Kai,WANG Yunhe,CHEN Hanting,et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(1):87-110. DOI: 10.1109/TPAMI.2022.3152247
[14]	GRAHAM B,EL-NOUBY A,TOUVRON H,et al. LeViT:a vision transformer in ConvNet’s clothing for faster inference[C]. IEEE/CVF International Conference on Computer Vision,Montreal,2021:12259-12269.
[15]	ARNAB A,DEHGHANI M,HEIGOLD G,et al. ViViT:a video vision transformer[C]. IEEE/CVF International Conference on Computer Vision,Montreal,2021:6836-6846.
[16]	VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[C]. 31st Conference on Neural Information Processing Systems,Long Beach,2017:5998-6008.
[17]	刘赟. ReLU激活函数下卷积神经网络的不同类型噪声增益研究[D]. 南京:南京邮电大学,2023. LIU Yun. Research on different types of noise gain in convolutional neural networks under ReLU activation function[D]. Nanjing:Nanjing University of Posts and Telecommunications,2023.
[18]	靳晶晶,王佩. 基于卷积神经网络的图像识别算法研究[J]. 通信与信息技术,2022(2):76-81. JIN Jingjing,WANG Pei. Research on image recognition algorithm based on convolutional neural network[J]. Communications and Information Technology,2022(2):76-81.
[19]	ALAHI A,GOEL K,RAMANATHAN V,et al. Social LSTM:human trajectory prediction in crowded spaces[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016:961-971.
[20]	BERGSTRA J,BREULEUX O,BASTIEN F,et al. Theano:a CPU and GPU math compiler in Python[C]. The 9th Python in Science Conference,2010. DOI: 10.25080/majora-92bf1922-003.
[21]	PESARANGHADER A,WANG Yiping,HAVAEI M. CT-SGAN:computed tomography synthesis GAN[C]// ENGELHARDT S,OKSUZ I,ZHU Dajiang,et al. Deep generative models,and data augmentation,labelling,and imperfections. Berlin:Springer-Verlag,2021:67-79.

[1]	WANG Shubin, WANG Xu, YAN Shiping, WANG Ke. Transformer based time series prediction method for mine internal caused fire[J]. Journal of Mine Automation, 2024, 50(3): 65-70, 91. DOI: 10.13272/j.issn.1671-251x.2023100084
[2]	LI Zexi. Ensemble learning mine pressure prediction method based on variable time series shift Transformer-LSTM[J]. Journal of Mine Automation, 2023, 49(7): 92-98. DOI: 10.13272/j.issn.1671-251x.18142
[3]	LI Yongqin, YU Zaichuan. Exploration and practice of intelligent transformation in Jinjie Coal Mine[J]. Journal of Mine Automation, 2022, 48(S1): 33-35.
[4]	ZHANG Guilin, WANG Yiying, LIU Jiangong, CHEN Longfei, LIU Wenzhuang. Mine inverter based on isolation and transformation of power electronic transformer[J]. Journal of Mine Automation, 2021, 47(10): 70-76. DOI: 10.13272/j.issn.1671-251x.2020110059
[5]	XU Qianwen, JI Xingquan, ZHANG Yuzhen, LI Jun, YU Yongjin. Fault diagnosis of mind-used transformer based on stacked sparse auto-encoder[J]. Journal of Mine Automation, 2018, 44(10): 33-37. DOI: 10.13272/j.issn.1671-251x.2018040092
[6]	TAO Fengyuan, ZHANG Dong, DONG Xinsheng, WANG Shirong. Analysis of influence factors of polarization spectrum method on state detection of transformer oil-paper insulatio[J]. Journal of Mine Automation, 2014, 40(10): 33-36. DOI: 10.13272/j.issn.1671-251x.2014.10.010
[7]	SHI Li-ping, JI Feng. Solution of Information Integration of Automation Subsystem for Underground Transformer[J]. Journal of Mine Automation, 2008, 34(1): 73-75.
[8]	LI De-chen, LIAO Hong-mei, WANG Yu-yang, LV Ming. Analysis of Winding Connection Way of Main Transformers of Power Transformer and Its Simulatio[J]. Journal of Mine Automation, 2007, 33(5): 28-29.
[9]	WU Yan-hua, MENG Jiao-ru. The Operating Disturbances Analysis of the Coal Transformer[J]. Journal of Mine Automation, 2001, 27(3): 40-42.
[10]	ZOU Yi-wen, LI Jin-he. Application of Circuit Transformer Bank in Small-scale Heat Power Plant in Coal Mine and Its Protection Distributio[J]. Journal of Mine Automation, 2001, 27(1): 18-20.