Abstract: There are some problems in coal gangue detection, such as small differences of characteristics between samples and dense targets. This leads to low precision and poor real-time performance of the existing coal gangue detection methods. In order to solve this problem, a method of coal gangue detection based on CBA-YOLO model is proposed. The CBA-YOLO model is based on YOLOv5m, which has faster speed and higher precision. The convolutional block attention module (CBAM) is added to the Backbone of YOLOv5m. The spatial attention module and the channel attention module are connected in series to focus on the difference of characteristics and reduce the data dimension. And the detection performance of coal gangue is improved. In the Neck part, the bi-directional feature pyramid network (BiFPN) structure is adopted to improve the calculation efficiency of the model by integrating the features of different scales. Therefore, the detection speed of coal gangue is improved. In the Prediction part, the Alpha-IoU function is used as the loss function. And the weight coefficient is set to accelerate the learning of high confidence targets, so as to further improve the detection precision of coal gangue. The experimental results show that the average detection precision of CBA-YOLO model for coal gangue is 98.2%, which is 3.4% higher than that of YOLOv5 model. The detection speed is increased by 10%. CBA-YOLO model is more robust and can effectively avoid missed detection, false detection and overlap.