In view of the problem of drifting of integrator existed in stator flux observer of traditional direct torque control system of salient pole synchronous motor, the paper proposed a design scheme of direct torque control system of salient pole synchronous motor with stator flux observer based on recurrent neural network. In the scheme, two-phase voltage and current transformed from three-phase voltage and current by 3S/2S are sent to trained stator flux observer based on recurrent neural network whose outputs are α, β components of stator flux, that is Ψsα, Ψsβ, then amplitude and space angle of stator flux are obtained after Ψsα, Ψsβ are processed by vector analyzer, so as to obtain sector location of stator flux accurately. The simulation results showed that the system has good dynamic and static performance comparing with direct torque control system of salient pole synchronous motor based on traditional U-I model.