Abstract: In order to solve problem of temporary loss for tracked moving object when pan-tilt camera rotated, the paper proposed a design scheme of moving object tracking system based on background subtraction algorithm and CAMShift algorithm. The system uses improved background subtraction algorithm to detect moving object when pan-tilt camera is stationary and uses CAMShift algorithm to track moving object when pan-tilt camera is turning, and tracks path of moving object through center coordinates. The simulation result showed that the system can meet with requirements of real-time and stability in certain degree.