Method of support posture perception in mining face based on visual-inertial information fusion
-
摘要:
针对目前采场支架姿态感知中惯导方法存在漂移误差大、解算精度低，以及视觉方法存在相机易受粉尘与设备遮挡而位姿识别误差大等问题，提出了一种基于视觉−惯导信息融合的采场支架姿态感知方法。首先将四特征点红外标靶固定于支架底座凸台，将双目相机分别固定于支架顶梁与掩护梁，采用基于Canny−最小二乘法的靶标识别方法和基于四特征点的BA−PnP算法解算顶梁、掩护梁相对底座的俯仰角、横滚角。然后将惯性测量单元固定于液压支架顶梁、掩护梁、底座，通过惯性测量单元中MEMS陀螺仪和加速度计的互补滤波方法解算顶梁、掩护梁、底座在世界坐标系下的俯仰角、横滚角。最后将视觉系统解算的姿态角与惯导解算的姿态角进行扩展卡尔曼滤波多源信息融合，利用视觉信息的低频稳定性抑制惯性测量单元的累计误差，得到精确的采场支架姿态。采用基于视觉、惯导和视觉−惯导信息融合3种支架姿态感知方法进行对比实验，结果表明：① 初始静止状态下，3种方法的精度均较高，但随着支架运行循环次数增加，基于视觉、惯导的感知结果逐渐偏离真值。② 基于视觉、惯导和视觉−惯导信息融合方法的顶梁相对底座的俯仰角感知均方根误差分别为0.201，0.190，0.081°，掩护梁相对底座的俯仰角感知均方根误差分别为0.340，0.297，0.162°。③ 基于视觉−惯导信息融合方法解算的液压支架立柱伸缩长度的均方根误差为13.682 mm，满足现场需求。基于视觉−惯导信息融合的采场支架姿态感知方法可为液压支架智能化控制提供更准确的姿态参数。
Abstract:To address the issues of large drift errors and low calculation accuracy in inertial navigation methods, as well as significant posture recognition errors in visual methods due to camera interference from dust and equipment obstructions, a method of support posture perception in mining face based on visual-inertial information fusion was proposed. First, four feature points of infrared targets were fixed to the base platform of the support, and binocular cameras were fixed to the support top beam and shield beam. A target recognition method based on Canny and least squares, along with a BA-PnP algorithm based on four feature points, was used to solve the pitch and roll angles of the top beam and shield beam relative to the base. Then, an inertial measurement unit (IMU) was fixed to the hydraulic support top beam, shield beam, and base. The complementary filtering method of the MEMS gyroscope and accelerometer in the IMU was used to solve the pitch and roll angles of the top beam, shield beam, and base in the world coordinate system. Finally, the posture angles calculated by the visual system and the inertial navigation system were fused using the extended Kalman filter for multi-source information fusion. The low-frequency stability of the vision information was used to suppress the accumulated errors of the IMU, resulting in accurate posture perception of the mining support. Three methods for support posture perception, based on vision, inertial navigation, and visual-inertial information fusion, were compared in experimental results. The findings showed that: ① In the initial stationary state, all three methods had high accuracy, but as the support operation cycles increased, the vision-based and inertial navigation-based results gradually deviated from the true values. ② The root mean square errors (RMSE) of pitch angle perception for the top beam relative to the base were 0.201°, 0.190°, and 0.081° for the vision-based, inertial navigation-based, and visual-inertial information fusion methods, respectively. For the shield beam relative to the base, the RMSE of pitch angle perception were 0.340°, 0.297°, and 0.162°, respectively. ③ The RMSE of the hydraulic support column extension length calculated by the visual-inertial information fusion method was 13.682 mm, meeting on-site requirements. The visual-inertial information fusion-based support posture perception method could provide more accurate posture parameters for the intelligent control of hydraulic supports.
-
-
![]()
图 2 基于视觉−惯导信息融合的支架姿态感知系统
Figure 2. Vision-inertial information fusion-based support posture perception system
![]()
图 6 不同方法下支架顶梁姿态感知实验结果
Figure 6. Experimental results of support beam posture perception under different methods
![]()
图 7 不同方法下支架掩护梁姿态感知实验结果
Figure 7. Experimental results of support shield beam posture perception under different methods
![]()
图 8 支架前立柱长度测量值和解析值
Figure 8. Measured and analytical values of the support front column length
表 1 IMU传感器参数
Table 1 Inertial measurement unit sensor parameters
参数 陀螺仪 加速度计 量程 ±2 000 （°）/s ±16g 零偏稳定性 <10 （°）/h <0.04 mg 线性度 <0.1% FS <0.1% FS 噪声密度 $ 0.002\;8（\text{° }）/(\mathrm{s}·\sqrt{\mathrm{Hz}}) $ $ 75\text{ }\text{μ}g/\sqrt{\mathrm{Hz}} $ 带宽 256 Hz 260 Hz 正交性误差 ±0.05° ±0.05° 分辨率 <0.02 （°）/s <0.5 mg 注：g为重力加速度。 
下载: 导出CSV
-
[1] REID P B,DUNN M T,REID D C,et al. Real-world automation:new capabilities for underground longwall mining[C]. Australasian Conference on Robotics and Automation,Brisbane,2010:1-8.
[2] 张坤,孙政贤,刘亚,等. 基于信息融合技术的超前液压支架姿态感知方法及实验验证[J]. 煤炭学报,2023,48(增刊1):345-356. ZHANG Kun,SUN Zhengxian,LIU Ya,et al. Research and experimental verification of attitude perception method of advanced hydraulic support based on information fusion technology[J]. Journal of China Coal Society,2023,48(S1):345-356.
[3] 刘相通,李曼,沈思怡,等. 液压支架关键姿态参数测量系统[J]. 工矿自动化,2024,50(4):41-49. LIU Xiangtong,LI Man,SHEN Siyi,et al. Measurement system for key attitude parameters of hydraulic support[J]. Journal of Mine Automation,2024,50(4):41-49.
[4] 李磊,许春雨,宋建成,等. 基于PSO−ELM的综采工作面液压支架姿态监测方法[J]. 工矿自动化,2024,50(8):14-19. LI Lei,XU Chunyu,SONG Jiancheng,et al. Attitude monitoring method for hydraulic support in fully mechanized working face based on PSO-ELM[J]. Journal of Mine Automation,2024,50(8):14-19.
[5] LIANG Minfu,FANG Xinqiu,LI Shuang,et al. A fiber Bragg grating tilt sensor for posture monitoring of hydraulic supports in coal mine working face[J]. Measurement,2019,138:305-313.
[6] 任怀伟,李帅帅,赵国瑞,等. 基于深度视觉原理的工作面液压支架支撑高度与顶梁姿态角测量方法研究[J]. 采矿与安全工程学报,2022,39(1):72-81,93. REN Huaiwei,LI Shuaishuai,ZHAO Guorui,et al. Measurement method of support height and roof beam posture angles for working face hydraulic support based on depth vision[J]. Journal of Mining & Safety Engineering,2022,39(1):72-81,93.
[7] CHEN Hongyue,CHEN Hongyan,XU Yajun,et al. Research on attitude monitoring method of advanced hydraulic support based on multi-sensor fusion[J]. Measurement,2022,187. DOI: 10.1016/j.measurement.2021.110341.
[8] 张洪伟,万志军,程敬义,等. 新型液压支架活柱位移监测仪的研制[J]. 中国煤炭,2015,41(9):69-73. DOI: 10.3969/j.issn.1006-530X.2015.09.017 ZHANG Hongwei,WAN Zhijun,CHENG Jingyi,et al. Development of a new displacement monitor of hydraulic support pillar[J]. China Coal,2015,41(9):69-73. DOI: 10.3969/j.issn.1006-530X.2015.09.017
[9] GAO Kuidong,XU Wenbo,ZHANG Hongyang,et al. Relative position and posture detection of hydraulic support based on particle swarm optimization[J]. IEEE Access,2020,8:200789-200811.
[10] ZHAO Tianyi,AHAMED M J. Pseudo-zero velocity re-detection double threshold zero-velocity update (ZUPT) for inertial sensor-based pedestrian navigation[J]. IEEE Sensors Journal,2021,21(12):13772-13785.
[11] HUANG Guoquan. Visual-inertial navigation:a concise review[C]. International Conference on Robotics and Automation,Montreal,2019. DOI: 10.1109/ICRA.2019.8793604.
[12] QIN Tong,LI Peiliang,SHEN Shaojie. VINS-mono:a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics,2018,34(4):1004-1020.
[13] LI Guangqiang,YU Lei,FEI Shumin. A binocular MSCKF-based visual inertial odometry system using LK optical flow[J]. Journal of Intelligent & Robotic Systems,2020,100(3):1179-1194.
[14] MOURIKIS A I,ROUMELIOTIS S I. A multi-state constraint Kalman filter for vision-aided inertial navigation[C]. IEEE International Conference on Robotics and Automation,Rome,2007:3565-3572.
[15] CARLSSON H,SKOG I,JALDÉN J. Self-calibration of inertial sensor arrays[J]. IEEE Sensors Journal,2021,21(6):8451-8463. DOI: 10.1109/JSEN.2021.3050010
[16] 于永军,徐锦法,张梁,等. 惯导/双目视觉位姿估计算法研究[J]. 仪器仪表学报,2014,35(10):2170-2176. YU Yongjun,XU Jinfa,ZHANG Liang,et al. Research on SINS/binocular vision integrated position and attitude estimation algorithm[J]. Chinese Journal of Scientific Instrument,2014,35(10):2170-2176.
[17] 万继成,张旭辉,杨文娟,等. 基于视觉与惯导的掘进机组合定位方法[J/OL]. 煤炭科学技术:1-12[2025-01-27]. http://kns.cnki.net/kcms/detail/11.2402.td.20240524.1014.003.html. WAN Jicheng,ZHANG Xuhui,YANG Wenjuan,et al. Combined positioning method of roadheader based on vision and inertial navigation[J/OL]. Coal Science and Technology:1-12[2025-01-27]. http://kns.cnki.net/kcms/detail/11.2402.td.20240524.1014.003.html.
[18] 毛清华,周庆,安炎基,等. 惯导与视觉信息融合的掘进机精确定位方法[J]. 煤炭科学技术,2024,52(5):236-248. DOI: 10.12438/cst.2023-1003 MAO Qinghua,ZHOU Qing,AN Yanji,et al. Precise positioning method of tunneling machine for inertial navigation and visual information fusion[J]. Coal Science and Technology,2024,52(5):236-248. DOI: 10.12438/cst.2023-1003
[19] 王平,周雪峰,安爱民,等. 一种鲁棒且线性的PnP问题求解方法[J]. 仪器仪表学报,2020,41(9):271-280. WANG Ping,ZHOU Xuefeng,AN Aimin,et al. Robust and linear solving method for Perspective-n-Point problem[J]. Chinese Journal of Scientific Instrument,2020,41(9):271-280.
[20] VALENTI R G,DRYANOVSKI I,XIAO Jizhong. Keeping a good attitude:a quaternion-based orientation filter for IMUs and MARGs[J]. Sensors,2015,15(8):19302-19330.
[21] 庞义辉,刘新华,王泓博,等. 基于千斤顶行程驱动的液压支架支护姿态与高度解析方法[J]. 采矿与安全工程学报,2023,40(6):1231-1242. PANG Yihui,LIU Xinhua,WANG Hongbo,et al. Support attitude and height analysis method of hydraulic support based on jack stroke drive[J]. Journal of Mining and Safety Engineering,2023,40(6):1231-1242.
计量
- 文章访问数: 60
- HTML全文浏览量: 11
- PDF下载量: 10
