Adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning
-
摘要: 受深部开采冲击地压等地质灾害扰动的影响，存在矿井超前支护系统自感知能力差、智能抗冲自适应能力弱、缺乏决策控制能力等问题。针对上述问题，提出了一种基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护方法。通过多源传感器感知巷道环境和超前液压支架支护状态，在虚拟世界中创建物理实体的数字孪生模型，其中物理模型精确展现超前液压支架的结构特征和细节，控制模型实现超前液压支架的自适应控制，机理模型实现对超前液压支架自适应支护的逻辑描述和机理解释，数据模型存储超前液压支架实体运行数据和孪生数据，仿真模型完成超前液压支架立柱仿真以实现超前液压支架与数字孪生模型虚实交互。根据基于深度Q网络（DQN）的超前液压支架自适应抗冲决策算法，对仿真环境中巷道抗冲支护进行智能决策，并依据决策结果对物理实体和数字孪生模型下达调控指令，实现超前液压支架智能控制。实验结果表明：立柱位移与压力变化一致，说明超前液压支架立柱仿真模型设计合理，从而验证了数字孪生模型的准确性；基于DQN的矿井超前液压支架自适应抗冲决策算法可通过调节液压支架控制器PID参数，自适应调控立柱压力，提升巷道安全等级，实现超前液压支架自适应抗冲支护。Abstract: Due to the disturbance of geological disasters such as deep mining and rock burst, there are problems such as poor self perception capability, weak intelligent anti impact adaptive capability, and lack of decision-making and control capability in the advanced support system of the mine. In order to solve the above problems, a adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning is proposed. By sensing the roadway environment and advanced hydraulic support status through multiple sensors, a digital twin model of a physical entity is created in a virtual world. The physical model accurately displays the structural features and details of the advanced hydraulic support. The control model realizes adaptive control of the advanced hydraulic support. The mechanism model realizes logical description and mechanism explanation of the adaptive support of the advanced hydraulic support. The data model stores the physical operation data and twin data of the advanced hydraulic support. The simulation model completes the simulation of the advanced hydraulic support column to achieve virtual real interaction between the advanced hydraulic support and the digital twin model. According to the adaptive impact resistance decision-making algorithm based on deep Q-network (DQN) for advanced hydraulic support, intelligent decision-making is made for roadway impact resistance support in the simulation environment. Based on the decision results, control instructions are issued to physical entities and digital twin models to achieve intelligent control of advanced hydraulic support. The experimental results show that the displacement and pressure changes of the column are consistent, indicating that the simulation model design of the advanced hydraulic support column is reasonable, thereby verifying the accuracy of the digital twin model. The adaptive impact resistance decision-making algorithm for advanced hydraulic supports in mines based on DQN can adjust the PID parameters of the hydraulic support controller, adaptively regulate the column pressure, improve the safety level of roadways, and achieve adaptive impact resistance support for advanced hydraulic supports.
-
图 1 基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护技术架构
Figure 1. Adaptive impact resistance support technology architecture for mine advanced hydraulic support based on digital twin and deep reinforcement learning
图 4 超前液压支架自适应抗冲决策流程
Figure 4. Adaptive impact resistance decision-making flow of advanced hydraulic support
图 5 基于DQN的超前液压支架自适应抗冲决策算法原理
Figure 5. Principle of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
图 7 基于DQN的超前液压支架自适应抗冲决策算法奖励收敛曲线
Figure 7. Reward convergence curve of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
表 1 深度强化学习算法对比
Table 1. Comparison of deep reinforcement learning algorithms
算法 优点 缺点 DQN 通过经验池进行随机小批量的采样训练，打破数据之间的相关性；采用每隔N次迭代更新目标Q网络的方法，提高Q网络稳定性 对目标Q网络采取最大化操作，对Q值的估计过高；训练数据规模较小 DDPG 同时建立Q值函数和策略函数；适用于连续动作的任务，稳定性好 对参数敏感；处理规模较大、时间较长，需要很强的GPU PPO 对智能体的策略函数进行优化，减少数据收集难度；适用于复杂场景和大规模训练 序列数据收集困难；对超参数选择敏感 
下载: 导出CSV
表 2 超前液压支架立柱仿真模型参数
Table 2. Simulation model parameters of advanced hydraulic support column
参数 值 参数 值 阀芯质量/kg 0.2 活塞杆直径/mm 10 活塞杆质量/kg 15 间隙直径/mm 0.02 反馈信号比例系数 5 比例阀阀杆直径/mm 10 阀芯控制比例系数 2 比例阀阀芯直径/mm 15 液压油密度/（kg·m−3） 920 顶板刚度系数/（N·mm−1） 15000 液压油压力/MPa 41.50 顶板阻尼系数/（N·s·m−1 ） 20 活塞直径/mm 30
下载: 导出CSV
表 3 巷道应力参数及安全系数
Table 3. Stress parameters and safety factors of roadway
巷道
区间/m围岩
应力/MPa支护
应力/MPa临界
应力/MPa应力安
全系数能量安
全系数安全
等级0～33 17.50 0.28 33.67 1.9 1.09 A 33～53 26.25 0.28 33.67 1.3 1.09 C 53～65 17.50 0.28 33.67 1.9 1.09 A 65～238 24.50 0.28 33.67 1.4 1.09 C 238～338 29.75 0.28 33.67 0.9 1.09 C 338～613 24.50 0.28 33.67 1.4 1.09 C 613～713 29.75 0.28 33.67 0.9 1.09 C 713～1186 24.50 0.28 33.67 1.4 1.09 C 1186～1732 17.50 0.28 33.67 1.9 1.09 A 1732～1763 21.00 0.28 33.67 1.6 1.09 A 1763～2660 17.50 0.28 33.67 1.9 1.09 A 2660～2680 21.00 0.28 33.67 1.6 1.09 A
下载: 导出CSV
表 4 基于DQN的超前液压支架自适应抗冲决策算法训练参数
Table 4. Training parameters of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)
参数 值 参数 值 经验池大小 2 000 最终概率 0.1 批量大小 64 目标值Q网络更新频率/步 50 折扣因子 0.95 训练轮次 480 初始概率 0.9 最大步数 100 概率的衰减系数 0.99
下载: 导出CSV
表 5 智能决策调控前后巷道状态对比
Table 5. Comparison of roadway state before and after intelligent decision control
巷道区间/m 初始巷道状态 调控后PID参数 调控后巷道状态 立柱
压力/MPa临界
应力/MPa应力安
全系数安全等级 KP KI KD 立柱
压力/MPa临界
应力/MPa应力安
全系数安全等级 65～238 17.21 33.67 1.4 C 0.6193 0.5841 0.6736 40.28 37.76 1.6 A 65～238 22.90 33.67 1.4 C 0.6187 0.5843 0.6738 40.31 37.76 1.6 A 338～613 21.25 33.67 1.4 C 0.6197 0.5861 0.6766 40.26 37.76 1.6 A 338～613 23.80 33.67 1.4 C 0.6193 0.5841 0.6736 40.32 37.76 1.6 A 713～1 186 17.21 33.67 1.4 C 0.6198 0.5861 0.6766 40.28 37.76 1.6 A 713～1 186 22.55 33.67 1.4 C 0.6187 0.5851 0.6746 40.41 37.76 1.6 A
下载: 导出CSV
-
[1] 吴拥政,付玉凯,何杰,等. 深部冲击地压巷道“卸压−支护−防护”协同防控原理与技术[J]. 煤炭学报,2021,46(1):132-144.WU Yongzheng,FU Yukai,HE Jie,et al. Principle and technology of "pressure relief-support-protection" collaborative prevention and control in deep rock burst roadway[J]. Journal of China Coal Society,2021,46(1):132-144. [2] 王国法,牛艳奇. 超前液压支架与围岩耦合支护系统及其适应性研究[J]. 煤炭科学技术,2016,44(9):19-25.WANG Guofa,NIU Yanqi. Study on advance hydraulic powered support and surrounding rock coupling support system and suitability[J]. Coal Science and Technology,2016,44(9):19-25. [3] 唐治,潘一山,朱小景,等. 自移式吸能防冲巷道超前支架设计与研究[J]. 煤炭学报,2016,41(4):1032-1037.TANG Zhi,PAN Yishan,ZHU Xiaojing,et al. Design and study of self-moving energy absorption and antiimpact roadway advanced support[J]. Journal of China Coal Society,2016,41(4):1032-1037. [4] 潘一山,齐庆新,王爱文,等. 煤矿冲击地压巷道三级支护理论与技术[J]. 煤炭学报,2020,45(5):1585-1594.PAN Yishan,QI Qingxin,WANG Aiwen,et al. Theory and technology of three levels support in bump-prone roadway[J]. Journal of China Coal Society,2020,45(5):1585-1594. [5] 杜明超,张坤,黄梁松,等. 新型抗冲击吸能装置手性结构吸能特性对比分析[J/OL]. 煤炭科学技术:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138.DU Mingchao,ZHANG Kun,HUANG Liangsong,et al. Comparative analysis of energy absorption characteristics of chiral structures of new impact absorbing devices[J/OL]. Coal Science and Technology:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138. [6] 宿吉奎,廉自生. 乳化液泵站的模糊PID控制系统仿真[J]. 煤矿机械,2012,33(7):54-56. doi: 10.3969/j.issn.1003-0794.2012.07.026SU Jikui,LIAN Zisheng. Simulation of fuzzy-PID control system in emulsion pump station[J]. Coal Mine Machinery,2012,33(7):54-56. doi: 10.3969/j.issn.1003-0794.2012.07.026 [7] ESSA M E S M,ABOELELA M A,HASSAN M A M. Position control of hydraulic servo system using proportional-integral-derivative controller tuned by some evolutionary techniques[J]. Journal of Vibration and Control,2016,22(12):2946-2957. doi: 10.1177/1077546314551445 [8] KIM S W,CHO B,SHIN S,et al. Force control of a hydraulic actuator with a neural network inverse model[J]. IEEE Robotics and Automation Letters,2021,6(2):2814-2821. doi: 10.1109/LRA.2021.3062353 [9] FENG Hao,SONG Qianyu,MA Shoulei,et al. A new adaptive sliding mode controller based on the RBF neural network for an electro-hydraulic servo system[J]. ISA Transactions,2022,129:472-484. doi: 10.1016/j.isatra.2021.12.044 [10] LIU Huaiqin,YU Qinghe,WU Qu. PID control model based on back propagation neural network optimized by adversarial learning-based grey wolf optimization[J]. Applied Sciences,2023,13(8). DOI: 10.3390/app13084767. [11] 薛光辉,管健,柴敬轩,等. 基于神经网络PID综掘巷道超前支架支撑力自适应控制[J]. 煤炭学报,2019,44(11):3596-3603.XUE Guanghui,GUAN Jian,CHAI Jingxuan,et al. Adaptive control of advance bracket support force in fully mechanized roadway based on neural network PID[J]. Journal of China Coal Society,2019,44(11):3596-3603. [12] 胡相捧,刘新华,庞义辉,等. 基于BP神经网络PID的液压支架初撑力自适应控制[J]. 矿业科学学报,2020,5(6):662-671.HU Xiangpeng,LIU Xinhua,PANG Yihui,et al. Adaptive control of setting load of hydraulic support based on BP neural network PID[J]. Journal of Mining Science and Technology,2020,5(6):662-671. [13] 徐亚军,张坤,李丁一,等. 超前支架自适应支护理论与应用[J]. 煤炭学报,2020,45(10):3615-3624.XU Yajun,ZHANG Kun,LI Dingyi,et al. Theory and application of self-adaptive support for advanced powered support[J]. Journal of China Coal Society,2020,45(10):3615-3624. [14] 张帆,葛世荣,李闯. 智慧矿山数字孪生技术研究综述[J]. 煤炭科学技术,2020,48(7):168-176.ZHANG Fan,GE Shirong,LI Chuang. Research summary on digital twin technology for smart mines[J]. Coal Science and Technology,2020,48(7):168-176. [15] EL BAZI N,MABROUKI M,LAAYATI O,et al. Generic multi-layered digital-twin-framework-enabled asset lifecycle management for the sustainable mining industry[J]. Sustainability,2023,15(4). DOI: 10.3390/su15043470. [16] 葛世荣,张帆,王世博,等. 数字孪生智采工作面技术架构研究[J]. 煤炭学报,2020,45(6):1925-1936.GE Shirong,ZHANG Fan,WANG Shibo,et al. Digital twin for smart coal mining workface:technological frame and construction[J]. Journal of China Coal Society,2020,45(6):1925-1936. [17] 张帆,葛世荣. 矿山数字孪生构建方法与演化机理[J]. 煤炭学报,2023,48(1):510-522.ZHANG Fan,GE Shirong. Construction method and evolution mechanism of mine digital twins[J]. Journal of China Coal Society,2023,48(1):510-522. [18] 陈博,朱斌,王蒙飞,等. 基于DDPG的矿井智能通风决策算法研究[J]. 煤矿机械,2023,44(10):201-204.CHEN Bo,ZHU Bin,WANG Mengfei,et al. Research on intelligent ventilation decision-making algorithm for mine based on DDPG[J]. Coal Mine Machinery,2023,44(10):201-204. [19] CAO Jiaqi,WANG Xiaolan,WANG Yansong,et al. An improved dueling deep Q-network with optimizing reward functions for driving decision method[J]. Proceedings of the Institution of Mechanical Engineers,Part D:Journal of Automobile Engineering,2023,237(9):2295-2309. doi: 10.1177/09544070221106037 [20] 刘清,张龙,李天越,等. 综采工作面三机数字孪生及协同建模方法[J]. 工矿自动化,2023,49(2):47-55.LIU Qing,ZHANG Long,LI Tianyue,et al. A three machine digital twin and collaborative modeling method for fully mechanized working face[J]. Journal of Mine Automation,2023,49(2):47-55. [21] 潘一山,肖永惠,罗浩,等. 冲击地压矿井安全性研究[J]. 煤炭学报,2023,48(5):1846-1860.PAN Yishan,XIAO Yonghui,LUO Hao,et al. Study on safety of rockburst mine[J]. Journal of China Coal Society,2023,48(5):1846-1860. -
点击查看大图
计量
- 文章访问数: 152
- HTML全文浏览量: 29
- PDF下载量: 52
- 被引次数: 0
