Abstract: For the problem that existing mine video monitoring system could only monitor environment and equipment operation status, but could not effectively monitor moving targets behavior such as person, an intelligent video analysis and early warning system for mine was designed. The system composition and architecture were introduced, and system realization principle was expounded taking belt coal-piling detection and person behavior detection and recognition as examples. Some tests were taken out in coal mine underground including belt coal-piling detection, person detection in danger area, system recognition rate and system response time. The tests results show that the system can effectively detect coal amount on belt and danger area range, whose recognition response time is not more than 2 seconds and successful recognition rate is not less than 98%.