Abstract:
To address the issues of high false positive rate and lack of zero-shot recognition capability in existing coal gangue object detection models for underground coal gangue sorting scenarios, a Zero-Shot Recognition Distillation Model for Coal Gangue (CGZRDM) is proposed. First, industrial cameras are used to capture and manually annotate simulated underground coal gangue images to construct an object detection dataset. Based on the annotation information, Regions of Interest (ROI) of coal gangue are extracted to build an image classification dataset. Second, the object detection dataset is used to train YOLOv8n, yielding a coal gangue detection model. Then, a Texture Enhancement Module (TEM) is introduced into the image encoder of CLIP-ViT-Base-Patch16 to construct a coal gangue zero-shot recognition model as the teacher model. A CNN-Transformer hybrid network replaces the image encoder of the teacher model to build the lightweight student model CGZRDM. Finally, the teacher model is trained on the image classification dataset, and an improved Feature-based Knowledge Distillation for Vision Transformers (ViTKD) method is employed to transfer the knowledge from the trained teacher model to the student model. In the production environment, the coal gangue detection model performs localization on real-time captured underground coal gangue images, outputting the position information of coal gangue ROI; the ROI are extracted based on the position information; and the distilled student model classifies the coal gangue ROI, outputting the category information. Experimental results show that the distilled student model achieves Top-1 accuracies of 93.2% on the closed-set test set and 86.4% on the zero-shot set, outperforming comparative models such as CBAM-ViT and CLIP-Adapter by an average of 2.7 and 0.2 percentage points, respectively. The inference speed reaches 38.2 FPS on the RK3588 edge device. This work integrates the real-time detection capability of YOLO with the zero-shot recognition capability of CLIP, providing a high-accuracy and strong-generalization solution for underground coal gangue sorting.