Intelligent decision-making method for coal caving based on fuzzy deep Q-network

During the coal caving process in the fully mechanized caving face, due to the impact of coal dust and dust water mist on the workers' line of sight, there are problems of over-caving and under-caving in manually controlled coal caving. In order to solve this problem, the tail beam of the hydraulic support is regarded as an intelligent agent, and the coal caving process is abstracted as a Markov optimal decision. A deep Q-network (DQN) is used to make decisions on the action of the coal drawing port. However, there is an overestimation problem in the DQN algorithm. A fuzzy deep Q-network (FDQN) algorithm is proposed and applied to intelligent decision-making of coal caving. The fuzzy control system is constructed by using the fuzzy features of the coal seam status in the coal caving process. The coal quantity and the coal gangue ratio in the coal seam state are taken as the inputs of the fuzzy control system. The output action of the fuzzy control system is replaced with the action of the DQN algorithm using the max operation to select the output Q value of the target network. It improves the online learning rate of the agent and increases the reward value of coal caving action. The coal caving model for the fully mechanized caving face is constructed. The three-dimensional numerical simulation of the coal caving process based on DQN, double depth Q-network (DDQN), and FDQN algorithms is conducted respectively. The results show that the FDQN algorithm has the fastest convergence speed, which is 31.6% faster than the DQN algorithm. It increases the online learning rate of the intelligent agent. The coal caving effect based on the FDQN algorithm is the best from three aspects: the straightness of the coal gangue boundary, the remaining coal above the tail beam, and the amount of gangue in the released body. The extraction rate based on the FDQN algorithm is the highest and the gangue content is the lowest. Compared with the DQN algorithm and DDQN algorithm, the extraction rate of the FDQN algorithm has increased by 2.8% and 0.7% respectively, and the gangue content has decreased by 2.1% and 13.2% respectively. The FDQN-based intelligent decision-making method for coal caving can adjust the action of the hydraulic support tail beam based on the coal seam occurrence status. It effectively solves the problems of over-caving and under-caving during the coal caving process.