Volume 49 Issue 4
Apr.  2023
Turn off MathJax
Article Contents
Article Navigation >  Journal of Mine Automation  > 2023  >  49(4): 78-85
YANG Yi, WANG Shengwen, CUI Kefei, et al. Intelligent decision-making method for coal caving based on fuzzy deep Q-network[J]. Journal of Mine Automation，2023，49(4)：78-85.  doi: 10.13272/j.issn.1671-251x.2022090068
Citation: YANG Yi, WANG Shengwen, CUI Kefei, et al. Intelligent decision-making method for coal caving based on fuzzy deep Q-network[J]. Journal of Mine Automation，2023，49(4)：78-85.  doi: 10.13272/j.issn.1671-251x.2022090068
shu

Intelligent decision-making method for coal caving based on fuzzy deep Q-network

doi: 10.13272/j.issn.1671-251x.2022090068
  • 1.

    School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454003, China

  • 2.

    Henan Key Laboratory of Intelligent Detection and Control of Coal Mine Equipment, Henan Polytechnic University, Jiaozuo 454003, China

  • 3.

    Zhengzhou Coal Mining Machinery Group Co., Ltd., Zhengzhou 450000, China

  • 4.

    Zhengzhou Coal Machine Hydraulic Control Group Co., Ltd., Zhengzhou 450013, China

  • 5.

    School of Automation, Southeast University, Nanjing 210096, China

  • Received Date: 2022-09-22
  • Rev Recd Date: 2023-04-19
    • Available Online: 2023-04-27
    Abstract
  • During the coal caving process in the fully mechanized caving face, due to the impact of coal dust and dust water mist on the workers' line of sight, there are problems of over-caving and under-caving in manually controlled coal caving. In order to solve this problem, the tail beam of the hydraulic support is regarded as an intelligent agent, and the coal caving process is abstracted as a Markov optimal decision. A deep Q-network (DQN) is used to make decisions on the action of the coal drawing port. However, there is an overestimation problem in the DQN algorithm. A fuzzy deep Q-network (FDQN) algorithm is proposed and applied to intelligent decision-making of coal caving. The fuzzy control system is constructed by using the fuzzy features of the coal seam status in the coal caving process. The coal quantity and the coal gangue ratio in the coal seam state are taken as the inputs of the fuzzy control system. The output action of the fuzzy control system is replaced with the action of the DQN algorithm using the max operation to select the output Q value of the target network. It improves the online learning rate of the agent and increases the reward value of coal caving action. The coal caving model for the fully mechanized caving face is constructed. The three-dimensional numerical simulation of the coal caving process based on DQN, double depth Q-network (DDQN), and FDQN algorithms is conducted respectively. The results show that the FDQN algorithm has the fastest convergence speed, which is 31.6% faster than the DQN algorithm. It increases the online learning rate of the intelligent agent. The coal caving effect based on the FDQN algorithm is the best from three aspects: the straightness of the coal gangue boundary, the remaining coal above the tail beam, and the amount of gangue in the released body. The extraction rate based on the FDQN algorithm is the highest and the gangue content is the lowest. Compared with the DQN algorithm and DDQN algorithm, the extraction rate of the FDQN algorithm has increased by 2.8% and 0.7% respectively, and the gangue content has decreased by 2.1% and 13.2% respectively. The FDQN-based intelligent decision-making method for coal caving can adjust the action of the hydraulic support tail beam based on the coal seam occurrence status. It effectively solves the problems of over-caving and under-caving during the coal caving process.

     

    • FullText(HTML)
  • loading
    • References(21)
  • [1]
    李爽,薛广哲,方新秋,等. 煤矿智能化安全保障体系及关键技术[J]. 煤炭学报,2020,45(6):2320-2330.

    LI Shuang,XUE Guangzhe,FANG Xinqiu,et al. Coal mine intelligent safety system and key technologies[J]. Journal of China Coal Society,2020,45(6):2320-2330.
    [2]
    葛世荣,郝尚清,张世洪,等. 我国智能化采煤技术现状及待突破关键技术[J]. 煤炭科学技术,2020,48(7):28-46.

    GE Shirong,HAO Shangqing,ZHANG Shihong,et al. Status of intelligent coal mining technology and potential key technologies in China[J]. Coal Science and Technology,2020,48(7):28-46.
    [3]
    张守祥,张学亮,刘帅,等. 智能化放顶煤开采的精确放煤控制技术[J]. 煤炭学报,2020,45(6):2008-2020.

    ZHANG Shouxiang,ZHANG Xueliang,LIU Shuai,et al. Intelligent precise control technology of fully mechanized top coal caving face[J]. Journal of China Coal Society,2020,45(6):2008-2020.
    [4]
    LIANG Minfu,HU Chengjun,YU Rui,et al. Optimization of the process parameters of fully mechanized top-coal caving in thick-seam coal using BP neural networks[J]. Sustainability,2022,14(3):1340-1357. doi: 10.3390/su14031340
    [5]
    王国法,庞义辉. 特厚煤层大采高综采综放适应性评价和技术原理[J]. 煤炭学报,2018,43(1):33-42.

    WANG Guofa,PANG Yihui. Full-mechanized coal mining and caving mining method evaluation and key technology for thick coal seam[J]. Journal of China Coal Society,2018,43(1):33-42.
    [6]
    霍昱名. 厚煤层综放开采顶煤破碎机理及智能化放煤控制研究[D]. 太原: 太原理工大学, 2021.

    HUO Yuming. Research on failure mechanism and intelligent drawing control of top coal in thick coal seam[D]. Taiyuan: Taiyuan University of Technology, 2021.
    [7]
    马英. 基于记忆放煤时序控制的智能放煤模式研究[J]. 煤矿机电,2015,36(2):1-5. doi: 10.3969/j.issn.1001-0874.2015.02.001

    MA Ying. Research on intelligent coal caving system based on memory coal caving sequential control[J]. Colliery Mechanical & Electrical Technology,2015,36(2):1-5. doi: 10.3969/j.issn.1001-0874.2015.02.001
    [8]
    李庆元,杨艺,李化敏,等. 基于Q-learning模型的智能化放顶煤控制策略[J]. 工矿自动化,2020,46(1):72-79.

    LI Qingyuan,YANG Yi,LI Huamin,et al. Intelligent control strategy for top coal caving based on Q-learning model[J]. Industry and Mine Automation,2020,46(1):72-79.
    [9]
    罗开成,高阳,杨艺,等. 基于均值偏差奖赏函数的放煤口控制策略研究[J]. 煤炭工程,2022,54(9):105-111.

    LUO Kaicheng,GAO Yang,YANG Yi,et al. Intelligent control strategy of drawing window in top-coal caving based on mean deviation reward function[J]. Coal Engineering,2022,54(9):105-111.
    [10]
    杨艺,李庆元,李化敏,等. 基于批量式强化学习的群组放煤智能决策研究[J]. 煤炭科学技术,2022,50(10):188-197. doi: 10.13199/j.cnki.cst.2020-1438

    YANG Yi,LI Qingyuan,LI Huamin,et al. Research on intelligent decision-making for group top-coal caving based on batch reinforcement learning[J]. Coal Science and Technology,2022,50(10):188-197. doi: 10.13199/j.cnki.cst.2020-1438
    [11]
    YANG Yi,LI Xinwei,LI Huaming,et al. Deep Q-network for optimal decision for top-coal caving[J]. Energies,2020,13(7):1618-1630. doi: 10.3390/en13071618
    [12]
    YANG Yi,LIN Zhiwei,LI Bingfeng,et al. Hidden Markov random field for multi-agent optimal decision in top-coal caving[J]. IEEE Access,2020,8:76596-76609. doi: 10.1109/ACCESS.2020.2984786
    [13]
    WANG Haixing,YANG Yi,LIN Zhiwei,et al. Multi-agent reinforcement learning with optimal equivalent action of neighborhood[J]. Actuators,2022,11(4):99. DOI: 10.3390/act11040099.
    [14]
    袁甜甜,李凤莲,张雪英,等. 特征降维的深度强化学习脑卒中分类预测研究[J]. 重庆理工大学学报(自然科学),2023,37(3):194-203.

    YUAN Tiantian,LI Fenglian,ZHANG Xueying,et al. Classification and prediction research of stroke based on deep reinforcement learning with feature dimension reduction[J]. Journal of Chongqing University of Technology(Natural Science),2023,37(3):194-203.
    [15]
    SUTTON R S. Learning to predict by the methods of temporal differences[J]. Machine Learning,1988,3(1):9-44.
    [16]
    MNIH V,KAVUKCUOGLU K,SLIVER D,et al. Human-level control through deep reinforcement learning[J]. Nature,2015,518:529-533. doi: 10.1038/nature14236
    [17]
    封硕,舒红,谢步庆. 基于改进深度强化学习的三维环境路径规划[J]. 计算机应用与软件,2021,38(1):250-255. doi: 10.3969/j.issn.1000-386x.2021.01.042

    FENG Shuo,SHU Hong,XIE Buqing. 3D environment path planning based on improved deep reinforcement learning[J]. Computer Applications and Software,2021,38(1):250-255. doi: 10.3969/j.issn.1000-386x.2021.01.042
    [18]
    黎声益,马玉敏,刘鹃. 基于双深度Q学习网络的面向设备负荷稳定的智能车间调度方法[J]. 计算机集成制造系统,2023,29(1):91-99. doi: 10.13196/j.cims.2023.01.008

    LI Shengyi,MA Yumin,LIU Juan. Smart shop floor scheduling method for equipment load stabilization based on double deep Q-learning network[J]. Computer Integrated Manufacturing Systems,2023,29(1):91-99. doi: 10.13196/j.cims.2023.01.008
    [19]
    李忠信,王大龙,庄佳才,等. 基于遗传模糊控制的风电机组偏航系统疲劳载荷研究[J]. 动力工程学报,2022,42(8):745-752,768. doi: 10.19805/j.cnki.jcspe.2022.08.008

    LI Zhongxin,WANG Dalong,ZHUANG Jiacai,et al. Research on fatigue suppression of wind turbine yaw system based on genetic fuzzy control[J]. Journal of Chinese Society of Power Engineering,2022,42(8):745-752,768. doi: 10.19805/j.cnki.jcspe.2022.08.008
    [20]
    张虎雄,李红卫,马祥. 模糊控制在煤矿智能化开采中的应用[J]. 煤矿机械,2022,43(12):206-210. doi: 10.13436/j.mkjx.202212062

    ZHANG Huxiong,LI Hongwei,MA Xiang. Application of fuzzy control in intelligent mining of coal mine[J]. Coal Mine Machinery,2022,43(12):206-210. doi: 10.13436/j.mkjx.202212062
    [21]
    沈志熙,代东林,赵凯. 基于多特征分步模糊推理的边缘检测算法[J]. 电子科技大学学报,2014,43(3):381-387. doi: 10.3969/j.issn.1001-0548.2014.03.011

    SHEN Zhixi,DAI Donglin,ZHAO Kai. Edge detection based on multi-features and step-by-step fuzzy inference[J]. Journal of University of Electronic Science and Technology of China,2014,43(3):381-387. doi: 10.3969/j.issn.1001-0548.2014.03.011
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(7)  / Tables(2)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (168) PDF downloads(14) Cited by()
    Proportional views
    Related

    苏ICP备 05016349号-2

    Supported by: Beijing Renhe Information Technology Co. Ltd.  Visits：    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return