Recognition of violations in belt conveyor area based on multi-feature fusion for time-difference network

The existing methods for recognizing violations in the underground belt conveyor area of coal mines suffer from insufficient feature extraction and difficulty in considering time differences in behavior. It results in low accuracy in recognizing violations (such as climbing, crossing, leaning, etc.) in the area. In order to solve the above problems, a belt conveyor area violation recognition method based on multi-feature fusion for time-difference network (MFFTDN) is proposed using the ResNet50 model. The method combines multi-feature fusion with time difference to perform multi-feature fusion on behaviors in different time periods. Firstly, the short-term multi-feature fusion (STMFF) module is introduced in the second and third stages of the original ResNet50 model, which concatenates the time and features from multiple consecutive frames together. Then the method performs time-difference calculation on the fused features, that is, the feature difference between adjacent frames, to capture local action changes in a short period of time. Secondly, in the fourth stage of the original ResNet50 model, a long- term multi-feature fusion (LTMFF) module is introduced to concatenate short-term multi-features from consecutive frames, and perform time-difference calculations on features from adjacent time points to obtain long-term multi- feature of behavior. Finally, the method classifies the fused features and outputs the recognition results. The experimental results show the following points. ① The average accuracy and precision of the MFFTDN based belt conveyor area violation recognition method have increased by 8.18% and 8.47% respectively compared to the original model ResNet50. It indicates that the simultaneous use of STMFF and LTMFF modules can effectively extract multi-feature information from different time periods. ② The accuracy of this recognition method on the self built dataset of violations in the underground belt conveyor area of coal mines is 89.62%, with an average precision of 89.3% and a model parameter size of 197.2×10⁶. ③ The Grad CAM heatmap shows that this recognition method can more effectively focus on key areas of violations and more accurately capture violations in the underground belt area of the mine.