Abstract: Fault diagnosis methods based on deep learning have high requirements for the quality of the dataset, requiring a large amount of data for good model training to achieve accurate fault diagnosis. However, the fault signals that can be collected in practical applications are usually limited. A method for diagnosing roller faults based on small sample sound signals is proposed to address the problem of limited performance of intelligent fault diagnosis methods due to the difficulty in obtaining sound signals for roller faults and the small sample size. The feature transformation method is used to convert one-dimensional sound signals into two-dimensional time-frequency images, incorporating features from the frequency domain to improve the dataset's capability to express fault features. A dataset expansion method combining multiple types of time-frequency maps has been proposed. The method combines time-frequency maps drawn by three time-frequency analysis methods: short time fourier transform (STFT), continuous wavelet transform (CWT), and Hilbert Huang transform (HHT) to expand the dataset and increase data styles. The concept of deep transfer learning is introduced, using bearing datasets to pre-train the model, and then using roller data to fine-tune the pre-trained model to further improve the recognition accuracy of the model. The experimental results show that the dataset expansion method combining multiple types of time-frequency maps can effectively solve the problem of overfitting when training models with small sample data. After using transfer learning, the testing accuracy of the model reaches 98.81%, an improvement of 7% compared to not using transfer learning. There was no overfitting phenomenon, indicating that the model is well-trained. Compared to the method of generating adversarial networks to expand the STFT time-frequency map dataset and transfer learning, the method of dataset expansion by combining multiple types of time frequency maps and transfer learning has an accuracy improvement of 4%. It is easier to implement, and has stronger interpretability.