Existing coal and gangue sorting method based on image identification has poor real-time performance and low sorting accuracy, the density-based sorting method is suitable for underground preparation but has high cost. In view of above problems, an intelligent coal and gangue sorting system based on EAIDK was designed. Embedded artificial intelligence development platform EAIDK is used to build hardware platform for gangue recognition and sorting control, deep learning algorithm is used to build a convolutional neural network under embedded deep learning framework Tengine, and end-to-end trainable image detection model is established and trained by image data obtained by smart cameras.Relationship between the camera coordinate system and the robot arm coordinate system is obtained through hand-eye calibration, and the gangue is tracked and sorted by robot arm. The experimental results show that the system's gangue recognition accuracy remains stable above 95%, the tracking time of robot arm is less than 30 ms, and the execution error is about 1 mm, which can meet the requirements of coal gangue sorting process.