Abstract:
Video monitoring of underground drilling operations in coal mines is easily affected by factors such as low illumination, dust blurring, miner lamp glare, target obstruction, and complex personnel-equipment interactions. Traditional single-frame image classification and "target detection + rule-based judgment" methods are difficult to reliably represent continuous operation states, especially in situations where personnel are close to the drill pipe, operations are conducted in low visibility, and anomaly risk identification is prone to false alarms and missed detections. To improve the accuracy, robustness, and real-time performance of drilling operation state identification in complex underground environments, a Degradation-aware Relation-guided Operation State Evolution Network (DROSE-Net) is proposed. The core of this method lies in constructing an integrated model framework of "degradation-aware relation state representation - continuous operation state evolution reasoning," extending underground drilling video from target detection result judgment to operation state understanding and anomaly risk identification. First, addressing the lack of state-level annotations in the publicly available coal mine underground drilling operation dataset DsDPM 66, weak labels for operation states are constructed based on target bounding box annotations, miner-drill rod spatial relationships, image degradation scores, and risk assessment rules. The impact of boundary sample noise is reduced through sample weighting and manual verification. Second, target categories, personnel-equipment spatial relationships, protective equipment visibility, detection confidence, image degradation information, and unknown suspicious regions are uniformly encoded into structured operation state labels, transforming target detection results into semantic evidence at the operation state level. Finally, state space evolution modeling is introduced to constrain changes in operation states within continuous video frames, enabling the identification of five state categories: equipment idle, normal drilling, personnel-drill rod interaction, low visibility operation, and abnormal risk. Experimental results show that DROSE-Net achieves accuracy, macro-average F1 score, and macro-average area under the receiver operating characteristic curve (AUC) of 94.12%, 93.22%, and 96.48%, respectively, all higher than comparable methods such as single-frame classification, detection rules, video recognition, and state labeling modeling. The AUPRC for abnormal risk states reaches 0.932, indicating that the method maintains high precision even with high recall. Degraded scene experiments and ablation experiments show that the degraded perception relation state representation and continuous state evolution reasoning can mitigate the impact of low illumination, dust blurring, glare, and occlusion on state judgment. Real-time performance experiments show that the model's single-frame inference time is 18.6 ms, with an FPS of 53.8. These results demonstrate that DROSE-Net can provide auxiliary support for video monitoring, abnormal risk alerts, and intelligent safety management in coal mine underground drilling operations.