A fully mechanized working face inspection system based on SLAM and virtual reality

The reliability of the inspection robot in the fully mechanized working face is low due to the lack of scale information. In order to solve the above problem, virtual reality (VR) technology is introduced into the inspection of fully mechanized working face. A fully mechanized working face inspection system based on simultaneous localization and mapping (SLAM) and VR is designed. The system includes two parts: an inspection robot subsystem located underground and a VR real-time rendering subsystem located on the ground. The inspection robot subsystem utilizes laser SLAM technology to achieve real-time 3D scanning and establish a 3D map. At the same time, a panoramic camera is used to capture the scene of the fully mechanized working face in real-time. The real-time obtained laser point cloud and panoramic video are transmitted to the VR real-time rendering subsystem. The VR real-time rendering subsystem uses GPU acceleration technology to color laser point clouds. By customizing the rendering part of the Unreal 3D engine, real-time rendering of the laser point cloud is achieved, and the laser point cloud is projected onto the VR glasses. Remote operators obtain real-time 3D scenes through VR glasses, and remotely control the movements of the inspection robot through the operating handle. The fully mechanized working face inspection based on the first perspective is achieved. The underground industrial test results show that the system can achieve free switching of perspectives and zoom in on the scene. It enables better observation of details, with higher accuracy and reliability. By using GPU acceleration technology for point cloud coloring, the processing time is significantly shorter than CPU processing time. GPU has higher real-time performance, and the entire system's latency can meet the requirements of inspection tasks.