基于改进时空图卷积神经网络的钻杆计数方法

杜京义; 党梦珂; 乔磊; 魏美婷; 郝乐

doi:10.13272/j.issn.1671-251x.2022030098

基于改进时空图卷积神经网络的钻杆计数方法

doi: 10.13272/j.issn.1671-251x.2022030098

杜京义^1,,
党梦珂^1, ,,
乔磊²,
魏美婷³,
郝乐⁴

1.
西安科技大学电气与控制工程学院, 陕西西安　710054
2.
西安热工研究院有限公司, 陕西西安　710054
3.
西安工程大学管理学院, 陕西西安　710048
4.
西安科技大学安全科学与工程学院, 陕西西安　710054

基金项目: 陕西省科技厅自然科学基金项目（2018JQ5197）；陕西省重点研发计划项目（2019GY-097）。

详细信息

作者简介:
杜京义(1965—)，男，山东淄博人，教授，硕士研究生导师，主要研究方向为检测技术及其自动化，E-mail：517571853@qq.com

通讯作者:
党梦珂（1998—），男，陕西武功人，硕士研究生，主要研究方向为目标检测与动作识别，E-mail：2447439418@qq.com

中图分类号: TD713
计量
- 文章访问数: 245
- HTML全文浏览量: 156
- PDF下载量: 35
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-29
- 修回日期: 2022-12-28
- 网络出版日期: 2022-09-08

Drill pipe counting method based on improved spatial-temporal graph convolution neural network

DU Jingyi^1
,,
DANG Mengke^{1
, ,},
QIAO Lei²,
WEI Meiting³,
HAO Le⁴

1.
College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China
2.
Xi'an Thermal Power Research Institute Co., Ltd., Xi'an 710054, China
3.
College of Management, Xi'an Polytechnic University, Xi'an 710048, China
4.
College of Safety Science and Engineering, Xi'an University of Science and Technology, Xi'an 710054, China

摘要

摘要: 针对现有钻杆计数方法存在劳动重复、计数误差较大、未考虑动作的时序信息等问题，提出了一种基于改进时空图卷积神经网络（MST−GCN）模型的钻杆计数方法。首先，通过矿用监控摄像头获取井下打钻视频数据，采用Alphapose算法在图像序列中提取人体的关键点信息，得到单帧图像上的人体骨架和连续图像序列上的骨架序列数据，进而构建表征人体动作的骨架序列；然后，在时空图卷积神经网络（ST−GCN）模型的基础上设计了MST−GCN模型，采用远空间分区策略关注骨架上距离较远的关键点运动信息，通过注意力机制网络SENet融合原空间特征与远空间特征，从而有效识别骨架序列上的动作类别；最后，在打钻视频上利用支持向量机辨识打钻姿势来决定是否保存骨架序列，若骨架序列长度保存到150帧则使用MST−GCN模型识别动作类别，并根据实际打钻时间设置相邻动作的识别间隔，从而记录动作数量，实现钻杆计数。实验结果表明：在自建的数据集上，MST−GCN模型的识别准确率为91.1%，比ST−GCN、Alphapose−LSTM和NST−GCN动作识别模型的准确率分别提升了6.2%，19.0%和4.8%，模型的损失值收敛在0.2以下，学习能力更强。在相同条件的打钻视频上，MST−GCN模型、人工方法和改进ResNet方法的平均误差依次为0.25，0.75，21次，说明MST−GCN模型的计数效果更好。在真实打钻1 300根的现场应用中，MST−GCN模型的平均误差为9根，误计数结果低，能够满足实际需求。
- 瓦斯抽采 /
- 钻孔 /
- 钻杆计数 /
- 打钻姿势 /
- 人体骨架 /
- 动作识别 /
- MST−GCN模型
Abstract: There are some problems in the existing drill pipe counting methods, such as repeated labor, large counting error, and failure to consider the timing information of actions. In order to solve the above problems, a drill pipe counting method based on an improved multi spatial-temporal graph convolution neural network (MST-GCN) model is proposed. Firstly, the video data of underground drilling is obtained through the mine monitoring camera. The Alphabose algorithm is used to extract the key points of the human body from the image sequence. The human skeleton on a single frame image and the skeleton sequence data on a continuous image sequence are obtained. The skeleton sequence representing human actions is built. Secondly, the MST-GCN model is designed based on the spatial-temporal graph convolution neural network (ST-GCN) model. The far space partition strategy is used to focus on the motion information of the key points that are far away from the skeleton. The squeeze and excitation network (SENet) is used to fuse the original space features and the far space features, so as to effectively identify the action categories on the skeleton sequence. Finally, support vector machine is used to identify the drilling pose on the drilling video to decide whether to save the skeleton sequence. If the sequence length is saved to 150 frames, the MST-GCN model is used to identify the action category. The identification interval of adjacent actions is set according to the actual drilling time, so as to record the number of actions and realize the drill pipe counting. The experimental results show that the recognition accuracy of the MST-GCN model is 91.1% on the self-built data set, which is 6.2%, 19.0% and 4.8% higher than that of ST-GCN, Alphapose-LSTM and NST-GCN, respectively. The loss value of the MST-GCN model converges below 0.2, and the learning capability is stronger. On the drilling videos under the same conditions, the average error values of the MST-GCN model, the artificial method and the improved ResNet method are 0.25, 0.75 and 21 respectively, which shows that the counting effect of the MST-GCN model is better. The average error of MST-GCN model is 9 and the miscount is low in the field application of drilling 1 300 pieces, which can meet the actual requirements.
- gas extraction /
- drilling /
- drill pipe counting /
- drilling pose /
- human skeleton /
- action recognition /
- MST-GCN model

HTML全文

图 1 打钻过程中的3种动作

Figure 1. Three kinds of action during drilling

下载: 全尺寸图片幻灯片

图 2 基于改进时空图卷积神经网络的钻杆计数方法原理

Figure 2. Principle of drill pipe counting method based on improved multi spatial-temporal graph convolution neural network

下载: 全尺寸图片幻灯片

图 3 人体骨架数据

Figure 3. Skeleton data of human body

下载: 全尺寸图片幻灯片

图 4 MST−GCN模型原理

Figure 4. Principle of multi spatial-temporal graph convolution neural network model

下载: 全尺寸图片幻灯片

图 5 原空间分区策略

Figure 5. Original spatial partitioning strategy

下载: 全尺寸图片幻灯片

图 6 远空间分区策略

Figure 6. Remote spatial partitioning strategy

下载: 全尺寸图片幻灯片

图 7 SENet模块融合空间特征

Figure 7. Fusion spatial features of the SENet module

下载: 全尺寸图片幻灯片

图 8 时空特征提取单元结构

Figure 8. Structure of spatio-temporal feature extraction unit

下载: 全尺寸图片幻灯片

图 9 打钻姿势与非打钻姿势

Figure 9. Drilling posture and non-drilling posture

下载: 全尺寸图片幻灯片

图 10 打钻姿势识别流程

Figure 10. Drill pose recognition process

下载: 全尺寸图片幻灯片

图 11 MST−GCN与ST−GCN模型的训练损失曲线

Figure 11. Training loss curves of MST-GCN model and ST-GCN model

下载: 全尺寸图片幻灯片

图 12 现场测试打钻画面

Figure 12. Field test drilling screen

下载: 全尺寸图片幻灯片

表 1 消融实验结果比较

Table 1. Comparison of ablation experiment results

基准模型	编号	分区策略		特征融合		准确率/%
基准模型	编号	原空间分区策略	远空间分区策略	ADD	SENet	准确率/%
ST−GCN	1	√	×	×	×	84.9
	2	×	√	×	×	83.1
	3	√	√	√	×	87.9
	4	√	√	×	√	91.1

下载: 导出CSV

表 2 不同模型的动作识别结果

Table 2. Action recognition results of different models

模型方法	准确率/%
Alphapose−LSTM^[15]	72.1
NST−GCN^[16]	86.3
MST−GCN	91.1

下载: 导出CSV

表 3 3种计数方法的动作识别结果

Table 3. Action recognition results of three counting methods

打钻视频	真实钻杆/根	人工方法		改进ResNet^[9]		本文方法
打钻视频	真实钻杆/根	上杆/次	卸杆/次	上杆/次	卸杆/次	上杆/次	卸杆/次
1	80	80	80	0	88	79	80
2	100	102	101	0	134	98	102
平均误差		0.75		21		0.25

下载: 导出CSV

表 4 全部钻孔的实验结果

Table 4. The results of all drilling experiments

钻孔编号	真实钻杆/根	识别结果
钻孔编号	真实钻杆/根	上杆/次	卸杆/次	均值/次
1	40	40	38	39
2	45	45	45	45
3	43	43	43	43
4	53	53	53	53
5	47	50	43	46.5
6	56	54	54	54
7	46	46	45	45.5
8	106	119	100	109.5
9	106	113	99	106
10	80	79	80	79.5
11	80	82	79	80.5
12	80	84	80	82
13	80	83	81	82
14	80	82	80	81
15	80	83	79	81
16	80	78	75	76.5
17	98	105	102	103.5
18	100	107	96	101.5
总计	1300	1346	1272	1309

下载: 导出CSV

参考文献(16)

[1]	梁运培, 郑梦浩, 李全贵, 等. 我国煤与瓦斯突出预测与预警研究综述[J/OL]. 煤炭学报: 1-24[2022-12-08]. DOI: 10.13225/j.cnki.jccs.2022.0965. LIANG Yunpei, ZHENG Menghao, LI Quangui, et al. A review on prediction and early warning methods of coal and gas outburst[J/OL]. Journal of China Coal Society: 1-24[2022-12-08]. DOI: 10.13225/j.cnki.jccs.2022.0965.
[2]	PAN Xiaokang,CHENG Hao,CHEN Jie,et al. An experimental study of the mechanism of coal and gas outbursts in the tectonic regions[J]. Engineering Geology,2020,279:105883. DOI: 10.1016/j.enggeo.2020.105883.
[3]	谢和平,周宏伟,薛东杰,等. 我国煤与瓦斯共采:理论、技术与工程[J]. 煤炭学报,2014,39(8):1391-1397. XIE Heping,ZHOU Hongwei,XUE Dongjie,et al. Theory,technology and engineering of simultaneous exploitation of coal and gas in China[J]. Journal of China Coal Society,2014,39(8):1391-1397.
[4]	李东前. 煤矿瓦斯防治技术研究[J]. 当代化工研究,2021(10):99-100. doi: 10.3969/j.issn.1672-8114.2021.10.049 LI Dongqian. Study on coal mine gas prevention and control technology[J]. Modern Chemical Research,2021(10):99-100. doi: 10.3969/j.issn.1672-8114.2021.10.049
[5]	李树刚,包若羽,张天军,等. 本煤层瓦斯抽采钻孔合理密封深度确定[J]. 西安科技大学学报,2019,39(2):183-188,216. LI Shugang,BAO Ruoyu,ZHANG Tianjun,et al. Determining the rational sealing depth for horizontal gas drainage borehole[J]. Journal of Xi'an University of Science and Technology,2019,39(2):183-188,216.
[6]	孙志飞,吴银成,胡云. 钻杆长度测量方法[J]. 工矿自动化,2015,41(3):51-53. SUN Zhifei,WU Yincheng,HU Yun. Length measurement methods of drill pipe[J]. Industry and Mine Automation,2015,41(3):51-53.
[7]	徐钊,房咪咪,周红伟,等. 基于电驻波的锚杆长度无损测量方法[J]. 工矿自动化,2013,39(9):112-115. doi: 10.7526/j.issn.1671-251X.2013.09.029 XU Zhao,FANG Mimi,ZHOU Hongwei,et al. Non-destructive measurement method of anchor stock length based on electricity standing wave[J]. Industry and Mine Automation,2013,39(9):112-115. doi: 10.7526/j.issn.1671-251X.2013.09.029
[8]	董立红,王杰,厍向阳. 基于改进Camshift算法的钻杆计数方法[J]. 工矿自动化,2015,41(1):71-76. DONG Lihong,WANG Jie,SHE Xiangyang. Drill counting method based on improved Camshift algorithm[J]. Industry and Mine Automation,2015,41(1):71-76.
[9]	高瑞,郝乐,刘宝,等. 基于改进ResNet网络的井下钻杆计数方法[J]. 工矿自动化,2020,46(10):32-37. GAO Rui,HAO Le,LIU Bao,et al. Research on underground drill pipe counting method based on improved ResNet network[J]. Industry and Mine Automation,2020,46(10):32-37.
[10]	党伟超,姚远,白尚旺,等. 煤矿探水卸杆动作识别研究[J]. 工矿自动化,2020,46(7):107-112. DANG Weichao,YAO Yuan,BAI Shangwang,et al. Research on unloading drill-rod action identification in coal mine water exploration[J]. Industry and Mine Automation,2020,46(7):107-112.
[11]	YAN Sijie, XIONG Yuanjun, LIN Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, 2018: 5361-5368.
[12]	FANG Haoshu, XIE Shuqin, TAI Yuwing, et al. RMPE: regional multi-person pose estimation[C]. Proceedings of the IEEE International Conference on Computer Vision, Venice, 2017: 2334-2343.
[13]	HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 7132-7141.
[14]	KE Qiuhong, BENNAMOUN M, AN Senjian, et al. A new representation of skeleton sequences for 3D action recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 3288-3297.
[15]	卫少洁,周永霞. 一种结合Alphapose和LSTM的人体摔倒检测模型[J]. 小型微型计算机系统,2019,40(9):1886-1890. doi: 10.3969/j.issn.1000-1220.2019.09.014 WEI Shaojie,ZHOU Yongxia. Human body fall detection model combining Alphapose and LSTM[J]. Journal of Chinese Computer Systems,2019,40(9):1886-1890. doi: 10.3969/j.issn.1000-1220.2019.09.014
[16]	杨世强, 李卓, 王金华, 等. 基于新分区策略的ST−GCN人体动作识别[J/OL]. 计算机集成制造系统: 1-16[2022-03-29]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211022.1500.014.html. YANG Shiqiang, LI Zhuo, WANG Jinhua, et al. ST-GCN human action based on new partition strategy[J/OL]. Computer Integrated Manufacturing Systems: 1-16[2022-03-29]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211022.1500.014.html.