Coal microscopic image preprocessing mainly includes coal scratch detection and removal. It is difficult to extract spatial shape characteristics accurately and refine edge information effectively for coal scratch detection based on the Hough transform algorithm and it is prone to miss detection and false detection. In order to solve the above problems, a coal scratch detection method based on semantic segmentation is proposed. This method introduces the residual structure to improve the spatial attention model, and embeds the model into U-Net which uses the VGG convolutional layer as the image characteristic encoder to obtain the semantic segmentation of coal scratches. In order to solve the problem that the fast-moving image restoration algorithm makes the texture difference and visual artifacts between the coal scratch removal area and the surrounding area, an image restoration algorithm based on improved area matching is proposed to remove coal scratches. The effective removal of coal scratches is achieved by using k-nearest neighbor image block search, cross-scale and rotation angle search strategies, and an image block offset distance measurement based on Euclidean distance. The experimental results show that the coal scratch detection method based on semantic segmentation can reflect the edge details of coal scratches accurately, has better spatial characteristic analysis performance, and improves the accuracy of coal scratch detection. The method adopts the image restoration algorithm based on improved area matching to remove coal scratches. Therefore, the texture characteristics of the coal scratch removal area and the surrounding area are more consistent, and the overall visual effect of the image is improved.