In order to accurately predict the sediment discharge of rivers with less data, Ruhe River, a tributary of Jinghe River, was selected as the research object. Based on the measured hydrological data of Kaibian Hydrological Station in the upper reaches of Ruhe River from 1980 to 2020, a BP neural network prediction model based on three activation functions was established. On this basis, a SA-BP neural network prediction model based on simulated annealing ( SA ) algorithm optimization was constructed, and the comparison of six prediction models was carried out. The results show that both BP neural network and SA-BP neural network model can better predict the sediment discharge in Ruhe River Basin, but in the case of only runoff data, the prediction accuracy of BP neural network model is low. The SA algorithm can improve the prediction accuracy of the BP neural network, and the SA-BP neural network based on the ReLU activation function has the best prediction effect, and the prediction accuracy is 0.86. This study provides a new method for accurate prediction of river sediment transport with less data.