Adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning
-
摘要: 受深部开采冲击地压等地质灾害扰动的影响，存在矿井超前支护系统自感知能力差、智能抗冲自适应能力弱、缺乏决策控制能力等问题。针对上述问题，提出了一种基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护方法。通过多源传感器感知巷道环境和超前液压支架支护状态，在虚拟世界中创建物理实体的数字孪生模型，其中物理模型精确展现超前液压支架的结构特征和细节，控制模型实现超前液压支架的自适应控制，机理模型实现对超前液压支架自适应支护的逻辑描述和机理解释，数据模型存储超前液压支架实体运行数据和孪生数据，仿真模型完成超前液压支架立柱仿真以实现超前液压支架与数字孪生模型虚实交互。根据基于深度Q网络（DQN）的超前液压支架自适应抗冲决策算法，对仿真环境中巷道抗冲支护进行智能决策，并依据决策结果对物理实体和数字孪生模型下达调控指令，实现超前液压支架智能控制。实验结果表明：立柱位移与压力变化一致，说明超前液压支架立柱仿真模型设计合理，从而验证了数字孪生模型的准确性；基于DQN的矿井超前液压支架自适应抗冲决策算法可通过调节液压支架控制器PID参数，自适应调控立柱压力，提升巷道安全等级，实现超前液压支架自适应抗冲支护。Abstract: Due to the disturbance of geological disasters such as deep mining and rock burst, there are problems such as poor self perception capability, weak intelligent anti impact adaptive capability, and lack of decision-making and control capability in the advanced support system of the mine. In order to solve the above problems, a adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning is proposed. By sensing the roadway environment and advanced hydraulic support status through multiple sensors, a digital twin model of a physical entity is created in a virtual world. The physical model accurately displays the structural features and details of the advanced hydraulic support. The control model realizes adaptive control of the advanced hydraulic support. The mechanism model realizes logical description and mechanism explanation of the adaptive support of the advanced hydraulic support. The data model stores the physical operation data and twin data of the advanced hydraulic support. The simulation model completes the simulation of the advanced hydraulic support column to achieve virtual real interaction between the advanced hydraulic support and the digital twin model. According to the adaptive impact resistance decision-making algorithm based on deep Q-network (DQN) for advanced hydraulic support, intelligent decision-making is made for roadway impact resistance support in the simulation environment. Based on the decision results, control instructions are issued to physical entities and digital twin models to achieve intelligent control of advanced hydraulic support. The experimental results show that the displacement and pressure changes of the column are consistent, indicating that the simulation model design of the advanced hydraulic support column is reasonable, thereby verifying the accuracy of the digital twin model. The adaptive impact resistance decision-making algorithm for advanced hydraulic supports in mines based on DQN can adjust the PID parameters of the hydraulic support controller, adaptively regulate the column pressure, improve the safety level of roadways, and achieve adaptive impact resistance support for advanced hydraulic supports.
-
-
![]()
图 1 基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护技术架构
Figure 1. Adaptive impact resistance support technology architecture for mine advanced hydraulic support based on digital twin and deep reinforcement learning
![]()
图 4 超前液压支架自适应抗冲决策流程
Figure 4. Adaptive impact resistance decision-making flow of advanced hydraulic support
![]()
图 5 基于DQN的超前液压支架自适应抗冲决策算法原理
Figure 5. Principle of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
![]()
图 7 基于DQN的超前液压支架自适应抗冲决策算法奖励收敛曲线
Figure 7. Reward convergence curve of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
表 1 深度强化学习算法对比
Table 1 Comparison of deep reinforcement learning algorithms
算法 优点 缺点 DQN 通过经验池进行随机小批量的采样训练，打破数据之间的相关性；采用每隔N次迭代更新目标Q网络的方法，提高Q网络稳定性 对目标Q网络采取最大化操作，对Q值的估计过高；训练数据规模较小 DDPG 同时建立Q值函数和策略函数；适用于连续动作的任务，稳定性好 对参数敏感；处理规模较大、时间较长，需要很强的GPU PPO 对智能体的策略函数进行优化，减少数据收集难度；适用于复杂场景和大规模训练 序列数据收集困难；对超参数选择敏感 
下载: 导出CSV
表 2 超前液压支架立柱仿真模型参数
Table 2 Simulation model parameters of advanced hydraulic support column
参数 值 参数 值 阀芯质量/kg 0.2 活塞杆直径/mm 10 活塞杆质量/kg 15 间隙直径/mm 0.02 反馈信号比例系数 5 比例阀阀杆直径/mm 10 阀芯控制比例系数 2 比例阀阀芯直径/mm 15 液压油密度/（kg·m−3） 920 顶板刚度系数/（N·mm−1） 15000 液压油压力/MPa 41.50 顶板阻尼系数/（N·s·m−1 ） 20 活塞直径/mm 30
下载: 导出CSV
表 3 巷道应力参数及安全系数
Table 3 Stress parameters and safety factors of roadway
巷道
区间/m围岩
应力/MPa支护
应力/MPa临界
应力/MPa应力安
全系数能量安
全系数安全
等级0～33 17.50 0.28 33.67 1.9 1.09 A 33～53 26.25 0.28 33.67 1.3 1.09 C 53～65 17.50 0.28 33.67 1.9 1.09 A 65～238 24.50 0.28 33.67 1.4 1.09 C 238～338 29.75 0.28 33.67 0.9 1.09 C 338～613 24.50 0.28 33.67 1.4 1.09 C 613～713 29.75 0.28 33.67 0.9 1.09 C 713～1186 24.50 0.28 33.67 1.4 1.09 C 1186～1732 17.50 0.28 33.67 1.9 1.09 A 1732～1763 21.00 0.28 33.67 1.6 1.09 A 1763～2660 17.50 0.28 33.67 1.9 1.09 A 2660～2680 21.00 0.28 33.67 1.6 1.09 A
下载: 导出CSV
表 4 基于DQN的超前液压支架自适应抗冲决策算法训练参数
Table 4 Training parameters of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
参数 值 参数 值 经验池大小 2 000 最终概率 0.1 批量大小 64 目标值Q网络更新频率/步 50 折扣因子 0.95 训练轮次 480 初始概率 0.9 最大步数 100 概率的衰减系数 0.99
下载: 导出CSV
表 5 智能决策调控前后巷道状态对比
Table 5 Comparison of roadway state before and after intelligent decision control
巷道区间/m 初始巷道状态 调控后PID参数 调控后巷道状态 立柱
压力/MPa临界
应力/MPa应力安
全系数安全等级 KP KI KD 立柱
压力/MPa临界
应力/MPa应力安
全系数安全等级 65～238 17.21 33.67 1.4 C 0.6193 0.5841 0.6736 40.28 37.76 1.6 A 65～238 22.90 33.67 1.4 C 0.6187 0.5843 0.6738 40.31 37.76 1.6 A 338～613 21.25 33.67 1.4 C 0.6197 0.5861 0.6766 40.26 37.76 1.6 A 338～613 23.80 33.67 1.4 C 0.6193 0.5841 0.6736 40.32 37.76 1.6 A 713～1 186 17.21 33.67 1.4 C 0.6198 0.5861 0.6766 40.28 37.76 1.6 A 713～1 186 22.55 33.67 1.4 C 0.6187 0.5851 0.6746 40.41 37.76 1.6 A
下载: 导出CSV
-
[1] 吴拥政,付玉凯,何杰,等. 深部冲击地压巷道“卸压−支护−防护”协同防控原理与技术[J]. 煤炭学报,2021,46(1):132-144. WU Yongzheng,FU Yukai,HE Jie,et al. Principle and technology of "pressure relief-support-protection" collaborative prevention and control in deep rock burst roadway[J]. Journal of China Coal Society,2021,46(1):132-144.
[2] 王国法,牛艳奇. 超前液压支架与围岩耦合支护系统及其适应性研究[J]. 煤炭科学技术,2016,44(9):19-25. WANG Guofa,NIU Yanqi. Study on advance hydraulic powered support and surrounding rock coupling support system and suitability[J]. Coal Science and Technology,2016,44(9):19-25.
[3] 唐治,潘一山,朱小景,等. 自移式吸能防冲巷道超前支架设计与研究[J]. 煤炭学报,2016,41(4):1032-1037. TANG Zhi,PAN Yishan,ZHU Xiaojing,et al. Design and study of self-moving energy absorption and antiimpact roadway advanced support[J]. Journal of China Coal Society,2016,41(4):1032-1037.
[4] 潘一山,齐庆新,王爱文,等. 煤矿冲击地压巷道三级支护理论与技术[J]. 煤炭学报,2020,45(5):1585-1594. PAN Yishan,QI Qingxin,WANG Aiwen,et al. Theory and technology of three levels support in bump-prone roadway[J]. Journal of China Coal Society,2020,45(5):1585-1594.
[5] 杜明超,张坤,黄梁松,等. 新型抗冲击吸能装置手性结构吸能特性对比分析[J/OL]. 煤炭科学技术:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138. DU Mingchao,ZHANG Kun,HUANG Liangsong,et al. Comparative analysis of energy absorption characteristics of chiral structures of new impact absorbing devices[J/OL]. Coal Science and Technology:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138.
[6] 宿吉奎,廉自生. 乳化液泵站的模糊PID控制系统仿真[J]. 煤矿机械,2012,33(7):54-56. DOI: 10.3969/j.issn.1003-0794.2012.07.026 SU Jikui,LIAN Zisheng. Simulation of fuzzy-PID control system in emulsion pump station[J]. Coal Mine Machinery,2012,33(7):54-56. DOI: 10.3969/j.issn.1003-0794.2012.07.026
[7] ESSA M E S M,ABOELELA M A,HASSAN M A M. Position control of hydraulic servo system using proportional-integral-derivative controller tuned by some evolutionary techniques[J]. Journal of Vibration and Control,2016,22(12):2946-2957. DOI: 10.1177/1077546314551445
[8] KIM S W,CHO B,SHIN S,et al. Force control of a hydraulic actuator with a neural network inverse model[J]. IEEE Robotics and Automation Letters,2021,6(2):2814-2821. DOI: 10.1109/LRA.2021.3062353
[9] FENG Hao,SONG Qianyu,MA Shoulei,et al. A new adaptive sliding mode controller based on the RBF neural network for an electro-hydraulic servo system[J]. ISA Transactions,2022,129:472-484. DOI: 10.1016/j.isatra.2021.12.044
[10] LIU Huaiqin,YU Qinghe,WU Qu. PID control model based on back propagation neural network optimized by adversarial learning-based grey wolf optimization[J]. Applied Sciences,2023,13(8). DOI: 10.3390/app13084767.
[11] 薛光辉,管健,柴敬轩,等. 基于神经网络PID综掘巷道超前支架支撑力自适应控制[J]. 煤炭学报,2019,44(11):3596-3603. XUE Guanghui,GUAN Jian,CHAI Jingxuan,et al. Adaptive control of advance bracket support force in fully mechanized roadway based on neural network PID[J]. Journal of China Coal Society,2019,44(11):3596-3603.
[12] 胡相捧,刘新华,庞义辉,等. 基于BP神经网络PID的液压支架初撑力自适应控制[J]. 矿业科学学报,2020,5(6):662-671. HU Xiangpeng,LIU Xinhua,PANG Yihui,et al. Adaptive control of setting load of hydraulic support based on BP neural network PID[J]. Journal of Mining Science and Technology,2020,5(6):662-671.
[13] 徐亚军,张坤,李丁一,等. 超前支架自适应支护理论与应用[J]. 煤炭学报,2020,45(10):3615-3624. XU Yajun,ZHANG Kun,LI Dingyi,et al. Theory and application of self-adaptive support for advanced powered support[J]. Journal of China Coal Society,2020,45(10):3615-3624.
[14] 张帆,葛世荣,李闯. 智慧矿山数字孪生技术研究综述[J]. 煤炭科学技术,2020,48(7):168-176. ZHANG Fan,GE Shirong,LI Chuang. Research summary on digital twin technology for smart mines[J]. Coal Science and Technology,2020,48(7):168-176.
[15] EL BAZI N,MABROUKI M,LAAYATI O,et al. Generic multi-layered digital-twin-framework-enabled asset lifecycle management for the sustainable mining industry[J]. Sustainability,2023,15(4). DOI: 10.3390/su15043470.
[16] 葛世荣,张帆,王世博,等. 数字孪生智采工作面技术架构研究[J]. 煤炭学报,2020,45(6):1925-1936. GE Shirong,ZHANG Fan,WANG Shibo,et al. Digital twin for smart coal mining workface:technological frame and construction[J]. Journal of China Coal Society,2020,45(6):1925-1936.
[17] 张帆,葛世荣. 矿山数字孪生构建方法与演化机理[J]. 煤炭学报,2023,48(1):510-522. ZHANG Fan,GE Shirong. Construction method and evolution mechanism of mine digital twins[J]. Journal of China Coal Society,2023,48(1):510-522.
[18] 陈博,朱斌,王蒙飞,等. 基于DDPG的矿井智能通风决策算法研究[J]. 煤矿机械,2023,44(10):201-204. CHEN Bo,ZHU Bin,WANG Mengfei,et al. Research on intelligent ventilation decision-making algorithm for mine based on DDPG[J]. Coal Mine Machinery,2023,44(10):201-204.
[19] CAO Jiaqi,WANG Xiaolan,WANG Yansong,et al. An improved dueling deep Q-network with optimizing reward functions for driving decision method[J]. Proceedings of the Institution of Mechanical Engineers,Part D:Journal of Automobile Engineering,2023,237(9):2295-2309. DOI: 10.1177/09544070221106037
[20] 刘清,张龙,李天越,等. 综采工作面三机数字孪生及协同建模方法[J]. 工矿自动化,2023,49(2):47-55. LIU Qing,ZHANG Long,LI Tianyue,et al. A three machine digital twin and collaborative modeling method for fully mechanized working face[J]. Journal of Mine Automation,2023,49(2):47-55.
[21] 潘一山,肖永惠,罗浩,等. 冲击地压矿井安全性研究[J]. 煤炭学报,2023,48(5):1846-1860. PAN Yishan,XIAO Yonghui,LUO Hao,et al. Study on safety of rockburst mine[J]. Journal of China Coal Society,2023,48(5):1846-1860.
-
期刊类型引用(35)
1. 张文科，郭瑜，赵辉. 基于图像识别的煤矿带式输送机自适应调速系统设计. 煤炭工程. 2024(01): 220-224 . 
百度学术
2. 刘锋，白金牛. 基于视觉技术的胶带输送机煤量检测方法. 陕西煤炭. 2024(01): 52-57+64 . 百度学术
3. 刘文梅. 基于煤流量监控的带式输送机监测方案设计. 机械管理开发. 2024(02): 243-244+247 . 百度学术
4. 岑梁，尹小明，黄海棠，胡文博，张世荣. 带式输送系统节能优化调度研究. 制造业自动化. 2024(05): 114-119 . 百度学术
5. 尚伟栋，杨大山，张坤. 基于深度神经网络的带式输送机煤量检测方法. 工矿自动化. 2024(S1): 139-145 . 本站查看
6. 张克亮. 基于MT-CNN的矿井带式输送机输煤量检测技术. 中国矿业. 2024(06): 137-142 . 百度学术
7. 王涛. 基于改进MobileNet的带式输送机煤量检测研究. 能源与环保. 2024(07): 198-202 . 百度学术
8. 尹瑞，张冬雪，倪强. 基于数组的刮板输送机运载模型及煤量计算算法研究. 工矿自动化. 2024(08): 84-90 . 本站查看
9. 段树深. 基于ROI边缘图像直线特征的井下带式输送机跑偏故障检测方法. 中国矿业. 2024(10): 162-167 . 百度学术
10. 刘西尧，纪妙，陈恩博，余宝意，曾庆杰，李海生. 基于机器视觉的烟草输送机皮带跑偏识别方法研究. 设备管理与维修. 2024(19): 165-169 . 百度学术
11. 左明明，张习，杨子豪，孙其飞，张梦超，张媛，李虎. 基于深度学习的输送带跑偏状态智能监测方法. 工矿自动化. 2024(12): 166-172+182 . 本站查看
12. 尚伟栋. 矿用带式输送机过载传感检测中的模糊阈值判断. 煤矿机械. 2023(04): 197-200 . 百度学术
13. 秦亚敏，张雷，赵东哲，张金辉，贾英新，靳晔. 基于机器视觉的带式输送机跑偏监测预警方法. 煤矿机械. 2023(05): 165-167 . 百度学术
14. 曾飞，胡文祥，高彦鑫，宋杰杰. 基于激光扫描的输送带横向跑偏检测系统. 制造业自动化. 2023(05): 21-24 . 百度学术
15. 吕晨辉，李新，刘新龙，赵安新，张晨阳. 基于煤量检测与变频一体机的煤流自适应智能调速. 煤矿机械. 2023(08): 213-216 . 百度学术
16. 李阿红. 基于图像处理的矿用带式输送机系统设计及其能效分析. 能源与环保. 2023(07): 228-232+238 . 百度学术
17. 曾飞，陶玉衡，苏俊彬，李翔. 融合ResNet18和Deconvolution的输送带横向跑偏检测方法. 现代制造工程. 2023(08): 121-126 . 百度学术
18. 毛清华，郭文瑾，翟姣，王荣泉，尚新芒，李世坤，薛旭升. 煤矿带式输送机异常状态视频AI识别技术研究. 工矿自动化. 2023(09): 36-46 . 本站查看
19. 高京，董华丛. 基于目标检测的铁路货车承载鞍错位故障识别. 自动化技术与应用. 2023(10): 63-67 . 百度学术
20. 周坪，马国庆，周公博，马天兵，李远博. 智能化带式输送机健康监测技术研究综述. 仪器仪表学报. 2023(12): 1-21 . 百度学术
21. 杨志方，张立亚，郝博南，刘渊，赵青. 基于双流融合网络的输送带跑偏检测方法. 煤炭科学技术. 2023(S2): 259-267 . 百度学术
22. 王桂梅，李学晖，杨立洁，刘杰辉. 基于深度学习的永磁直驱带式输送机煤量检测方法研究. 煤炭技术. 2022(01): 188-190 . 百度学术
23. 邢建旭，岑梁，卢峰，黄益军，胡文博，邱泽晶，张世荣. 基于带式输送机的仓储转运系统优化调度研究. 机电工程. 2022(03): 402-410 . 百度学术
24. 屠鑫. 基于图像识别的中长跑运动员摆臂高度校准方法研究. 河北北方学院学报(自然科学版). 2022(07): 7-12+18 . 百度学术
25. 白光星，陈炜乐，孙勇，宋双林，李国芳，王彩萍，王伟峰，康付如. 煤矿带式输送机运输火灾风险智能监测与早期预警技术研究进展. 煤矿安全. 2022(09): 47-54 . 百度学术
26. 王海军，王洪磊. 带式输送机智能化关键技术现状与展望. 煤炭科学技术. 2022(12): 225-239 . 百度学术
27. 刘孝军，王飞. 基于AI的煤矿视频智能分析技术. 煤炭科学技术. 2022(S2): 260-264 . 百度学术
28. 姜阔胜，李良和，王开松，胡坤. 煤矿输送带修复点定位及监测系统设计. 煤矿机械. 2021(02): 1-4 . 百度学术
29. 王平. 基于数字图像处理的输送带跑偏状态实时监测技术. 煤矿机械. 2021(02): 168-170 . 百度学术
30. 云艳. 矿用带式输送机跑偏机理及纠偏系统设计. 能源与环保. 2021(12): 264-268 . 百度学术
31. 李林. 矿用皮带输送机纠偏装置的分析研究. 机械管理开发. 2020(08): 74-75+78 . 百度学术
32. 高云飞. 带式输送机托辊结构受力分析. 中国石油和化工标准与质量. 2020(15): 157-158 . 百度学术
33. 张凯，田原，贾曲. 机器视觉在煤机装备中的应用现状与趋势. 煤矿机械. 2020(12): 123-125 . 百度学术
34. 陈小霞，徐伟同，侯飞，高奎，刘兆雪，温玺杰，王兵，王延勇，王晓倩. 基于图像处理的煤炭量AI识别系统的设计. 选煤技术. 2020(06): 75-80 . 百度学术
35. 成新文. 带式输送机防跑偏技术分析研究. 内蒙古煤炭经济. 2020(15): 47-48 . 百度学术
其他类型引用(14)
计量
- 文章访问数: 516
- HTML全文浏览量: 42
- PDF下载量: 80
- 被引次数: 49
