Abstract:
Low illumination, dust-induced blur, glare, and target occlusion in underground coal mine drilling videos make it difficult for single-frame detection results to reliably reflect continuous operation states. To address this problem, a method for recognizing coal mine drilling operation states through relation modeling and state evolution was proposed. A Degradation-Aware Relation-Guided Operation State Evolution Network (DROSE-Net) was developed to determine primary operation states and perceive abnormal risks. Operational behavior, environmental visibility, and abnormal risk were defined as underlying attributes that could coexist and were mapped to a single primary operation state selected from five classes: abnormal risk, low-visibility operation, personnel–drill rod interaction, normal drilling, and equipment idleness. Weak labels were constructed from bounding-box annotations, spatial relationships between personnel and drill rods, image degradation scores, and risk rules. Downweighting of boundary samples and manual review were used to improve label reliability. Object semantics, spatial relationships, image degradation, unknown suspicious regions, and detection confidence were encoded as structured state tokens. A state-space model updated the hidden state across consecutive frames, with the primary-state classification head outputting discrete operation states and an independent risk branch outputting a continuous abnormal-risk probability. Experiments on the underground coal mine drilling operation dataset DsDPM 66 showed that DROSE-Net achieved an accuracy of 94.12%, a macro-averaged F1 score of 93.21%, and a macro-averaged Area under the Receiver Operating Characteristic Curve (AUC) of 96.48%. Its Area under the Precision–Recall Curve (AUPRC) for abnormal risk was 0.932. With identical state-token inputs, its accuracy and macro-averaged
F1 score exceeded those of Token-Transformer by 2.86% and 2.90%, respectively. Single-frame inference took 18.6 ms, corresponding to a processing speed of 53.8 frames/s. These results validate the effectiveness of the complete pipeline integrating structured state representation and temporal evolution modeling and provide auxiliary support for video monitoring of underground coal mine drilling operations and abnormal-risk alerts.