基于图像检测的煤矸分拣机器人实验平台

李三喜; 李亚男; 王梓杰; 侯鹏; 薛光辉

doi:10.13272/j.issn.1671-251x.2022120028

基于图像检测的煤矸分拣机器人实验平台

doi: 10.13272/j.issn.1671-251x.2022120028

李三喜^{1, 2,},
李亚男²,
王梓杰²,
侯鹏²,
薛光辉^2, ,

1.
北京铁路电气化学校信号与电子教研室, 北京　102202
2.
中国矿业大学(北京) 机电与信息工程学院, 北京　100083

基金项目: 国家自然科学基金面上项目（51874308）。

详细信息

作者简介:
李三喜（1965—），男，山西运城人，讲师，主要研究方向为电子产品、机器人和煤矸图像识别，E-mail:976558003@qq.com

通讯作者:
薛光辉（1977—），男，河南汝州人，副教授，博士，主要研究方向为煤矿机器人、煤矿设备自动化与智能化、设备状态检测与健康诊断、无线传感器网络，E-mail：xgh@cumtb.edu.cn

中图分类号: TD67
计量
- 文章访问数: 193
- HTML全文浏览量: 24
- PDF下载量: 36
- 被引次数: 0
出版历程
- 收稿日期: 2022-12-08
- 修回日期: 2023-07-25
- 网络出版日期: 2023-08-03

Experimental platform for coal gangue sorting robot based on image detection

LI Sanxi^{1, 2
,},
LI Ya'nan²,
WANG Zijie²,
HOU Peng²,
XUE Guanghui^{2
, ,}

1.
Signal and Electronics Teaching and Research Office, Beijing Railway Electrification College, Beijing 102202, China
2.
School of Mechanical Electronic and Information Engineering, China University of Mining and Technology-Beijing, Beijing 100083, China

摘要

摘要: 目前煤矸预分选仍多为人工完成，劳动强度大、分拣效率低，且存在安全隐患，利用煤矸分拣机器人代替人工完成煤矸预分选是保障工人健康和安全、提高作业效率的有效途径。然而现有的煤矸分拣机器人在弱光照强度、煤矸表面覆盖煤粉等情况下的效果较差，针对上述问题，提出了基于图像检测的煤矸分拣机器人实验平台。该实验平台通过工业相机采集煤矸图像，利用ResNet18−YOLOv3深度学习算法对图像中的煤矸进行识别，采用TCP通信将矸石的位置信息提供给煤矸分拣模块进行轨迹规划，控制机械臂对矸石进行夹取，完成矸石分拣作业。采用Halcon标定法对实验平台进行手眼标定，从而实现相机像素坐标与机械臂空间坐标的转换；对实验平台进行了定位误差标定，对于尺寸均为50 mm以上的煤矸样本，定位误差不大于9 mm。实验结果表明，该实验平台在强光照条件下的煤矸识别准确率达99%，在弱光照条件下的煤矸识别准确率为95%，在煤粉附着条件下的煤矸识别准确率不低于82%，且煤矸分拣准确率为82%。
- 煤矿机器人 /
- 煤矸分拣 /
- 煤矸识别 /
- 图像检测 /
- 手眼标定
Abstract: Currently, coal gangue pre-sorting is still mostly done manually, with high labor intensity, low sorting efficiency, and safety hazards. Using coal gangue sorting robots to replace manual coal gangue pre-sorting is an effective way to ensure the health and safety of workers and improve work efficiency. However, the existing coal gangue sorting robots have poor performance in situations such as low light intensity and coal gangue surface covered with coal powder. To solve the above problems, an experimental platform for coal gangue sorting robot based on image detection is proposed. This experimental platform collects coal gangue images through industrial cameras. The platform uses ResNet18-YOLOv3 deep learning algorithm to identify the coal gangue in the images. The platform uses TCP communication to provide the position information of the gangue to the coal gangue sorting module for trajectory planning, then controls the manipulator to clamp the gangue and completes the gangue sorting operation. The platform uses the Halcon calibration method for hand-eye calibration of the experimental platform, in order to achieve the conversion of camera pixel coordinates and manipulator spatial coordinates. The positioning error of the experimental platform is calibrated. For coal gangue samples with sizes above 50 mm, the positioning error should not exceed 9 mm. The experimental results show that the recognition accuracy of the experimental platform for coal gangue under strong lighting conditions is 99%. The recognition accuracy of coal gangue under weak lighting conditions is 95%. The recognition accuracy of coal gangue under pulverized coal adhesion conditions is not less than 82%. The accuracy of coal gangue sorting is 82%.
- coal mining robot /
- coal gangue sorting /
- coal gangue recognition /
- image detection /
- hand-eye calibration

HTML全文

图 1 煤矸分拣机器人实验平台组成

Figure 1. Construction of experimental platform for coal-gangue sorting robot

下载: 全尺寸图片幻灯片

图 2 图像采集模块

Figure 2. Image acquisition module

下载: 全尺寸图片幻灯片

图 3 煤矸分拣机器人实验平台

Figure 3. Experimental platform for coal-gangue sorting robot

下载: 全尺寸图片幻灯片

图 4 基于ResNet18−YOLOv3的煤矸识别模型结构

Figure 4. Structure of coal-gangue recognition model based on ResNet18-YOLOv3

下载: 全尺寸图片幻灯片

图 5 标定盘

Figure 5. Calibration plate

下载: 全尺寸图片幻灯片

图 6 煤矸分拣机器人实验平台分拣实验流程

Figure 6. Sorting flow of experimental platform for coal-gangue sorting robot

下载: 全尺寸图片幻灯片

图 7 不同光照和煤粉附着情况下煤矸正确检测结果

Figure 7. Correct coal-gangue detection results under different illumination and pulverized coal adhesion

下载: 全尺寸图片幻灯片

图 8 煤矸错漏检结果

Figure 8. Error and omission coal-gangue detection results

下载: 全尺寸图片幻灯片

表 1 AUBO−i5协作机械臂的技术指标及参数

Table 1. Technical indexes and parameters of AUBO-i5 cooperative manipulator

技术指标	参数
自由度	6
最大工作半径/mm	886.5
负载/kg	5
重复定位精度/mm	±0.02
工作速度/（m·s⁻¹）	≤2.8

下载: 导出CSV

表 2 带式输送机技术指标及参数

Table 2. Technical indexes and parameters of belt conveyor

技术指标	参数
长度/m	1.5
宽度/m	0.4
高度/m	0.8～0.9
运行速度/（m·s^-1）	0.3～0.6

下载: 导出CSV

表 3 标定盘9个点的空间坐标及对应的像素坐标

Table 3. The space coordinates of 9 points of the calibration plate and the corresponding pixel coordinates

序号	像素坐标/（mm，mm）	空间坐标/（mm，mm）
1	（725，1 849）	（743，−506）
2	（717，1113）	（738，−577）
3	（729，385）	（737，−640）
4	（1 329，1841）	（682，−505）
5	（1 333，1109）	（680，−570）
6	（1 341，377）	（675，−640）
7	（2 069，1849）	（623，−498）
8	（2 081，1113）	（614，−560）
9	（2 077，385）	（620，−636）

下载: 导出CSV

表 4 煤矸分拣机器人实验平台煤矸定位误差

Table 4. Coal-gangue positioning error of experimental platform for coal-gangue sorting robot

样本号	定位坐标/ （mm，mm）	实际坐标/ （mm，mm）	X轴误差/mm	Y轴误差/mm
1	（663，−243）	（668，−237）	5	6
2	（725，−145）	（728，−136）	3	9
3	（612，224）	（610，217）	2	7
4	（684，148）	（689，140）	5	8
5	（652，−60）	（653，−57）	1	3
6	（605，164）	（609，158）	4	6
7	（534，63）	（528，58）	6	5
8	（476，25）	（469，24）	7	1
9	（588，97）	（581，102）	7	5
10	（523，−56）	（527，−53）	4	3

下载: 导出CSV

表 5 煤矸分拣结果

Table 5. Coal-gangue sorting results

矸石数量/个	正确分拣数量/个	错漏拣数量/个	准确率/%	错漏拣率/%
50	41	9	82	18

下载: 导出CSV

参考文献(26)

[1]	康红普,王国法,王双明,等. 煤炭行业高质量发展研究[J]. 中国工程科学,2021,23(5):130-138. doi: 10.15302/J-SSCAE-2021.05.016 KANG Hongpu,WANG Guofa,WANG Shuangming,et al. High-quality development of China's coal industry[J]. Strategic Study of CAE,2021,23(5):130-138. doi: 10.15302/J-SSCAE-2021.05.016
[2]	王学民,许光前,杨硕. 原煤预排矸技术与装备发展历程综述[J]. 选煤技术,2019(1):62-66,71. WANG Xuemin,XU Guangqian,YANG Shuo. A general overview of the course of development of raw coal preliminary dirt removal technology and equipment[J]. Coal Preparation Technology,2019(1):62-66,71.
[3]	SUN Zhiyuan,LU Wenhao,XUAN Pengcheng,et al. Separation of gangue from coal based on supplementary texture by morphology[J]. International Journal of Coal Preparation and Utilization,2019,42(3):221-237.
[4]	王鹏,曹现刚,夏晶,等. 基于机器视觉的多机械臂煤矸石分拣机器人系统研究[J]. 工矿自动化,2019,45(9):47-53. WANG Peng,CAO Xiangang,XIA Jing,et al. Research on multi-manipulator coal and gangue sorting robot system based on machine vision[J]. Industry and Mine Automation,2019,45(9):47-53.
[5]	李曼,段雍,曹现刚,等. 煤矸分选机器人图像识别方法和系统[J]. 煤炭学报,2020,45(10):3636-3644. LI Man,DUAN Yong,CAO Xiangang,et al. Image identification method and system for coal and gangue sorting robot[J]. Journal of China Coal Society,2020,45(10):3636-3644.
[6]	程学珍,王伟,卫阿盈. 煤与矸石在线分选系统设计[J]. 工矿自动化,2013,39(8):95-98. CHENG Xuezhen,WANG Wei,WEI Aying. Design of online separation system for coal and gangue[J]. Industry and Mine Automation,2013,39(8):95-98.
[7]	孙稚媛. 基于视觉信息的煤矸石自动分拣机器人系统研究[D]. 北京: 中国矿业大学（北京）, 2021. SUN Zhiyuan. Research on automatic separating robot system of coal and gangue based on visual information[D]. Beijing: China University of Mining and Technology-Beijing, 2021.
[8]	曹现刚,费佳浩,王鹏,等. 基于多机械臂协同的煤矸分拣方法研究[J]. 煤炭科学技术,2019,47(4):7-12. CAO Xiangang,FEI Jiahao,WANG Peng,et al. Study on coal-gangue sorting method based on multi-manipulator collaboration[J]. Coal Science and Technology,2019,47(4):7-12.
[9]	王冠军,苏婷婷,刘文博,等. 基于EAIDK的智能煤矸分拣系统设计[J]. 工矿自动化,2020,46(1):105-108. WANG Guanjun,SU Tingting,LIU Wenbo,et al. Design of intelligent coal and gangue sorting system based on EAIDK[J]. Industry and Mine Automation,2020,46(1):105-108.
[10]	胡平,李新,吕晨辉,等. 基于机器视觉的矿井煤矸分拣装置研制[J]. 煤矿机械,2022,43(12):211-213. HU Ping,LI Xin,LYU Chenhui,et al. Development of coal gangue separation device based on machine vision for mines[J]. Coal Mine Machinery,2022,43(12):211-213.
[11]	徐志强,吕子奇,王卫东,等. 煤矸智能分选的机器视觉识别方法与优化[J]. 煤炭学报,2020,45(6):2207-2216. XU Zhiqiang,LYU Ziqi,WANG Weidong,et al. Machine vision recognition method and optimization for intelligent separation of coal and gangue[J]. Journal of China Coal Society,2020,45(6):2207-2216.
[12]	HE Lei, WANG Shuang, GUO Yongcun, et al. Study of raw coal identification method by dual-energy X-ray and dual-view visible light imaging[J]. International Journal of Coal Preparation and Utilization, 2022. DOI: 10.1080/19392699.2022.2051013.
[13]	ESHAQ R M A,HU Eryi,LI Menggang,et al. Separation between coal and gangue based on infrared radiation and visual extraction of the YCbCr color space[J]. IEEE Access,2022,8:55204-55220.
[14]	LI Na, GONG Xingyu. An image preprocessing model of coal and gangue in high dust and low light conditions based on the joint enhancement algorithm[J]. Computational Intelligence and Neuroscience, 2021. DOI: 10.1155/2021/2436486.
[15]	DOU Dongyang,WU Wenze,YANG Jianguo,et al. Classification of coal and gangue under multiple surface conditions via machine vision and relief-SVM[J]. Powder Technology,2019,356:1024-1028. doi: 10.1016/j.powtec.2019.09.007
[16]	GUO Yongcun,WANG Xinquan,WANG Shuang,et al. Identification method of coal and coal gangue based on dielectric characteristics[J]. IEEE Access,2021,9:9845-9854. doi: 10.1109/ACCESS.2021.3050196
[17]	WANG Weidong,LYU Ziqi,LU Hengrun. Research on methods to differentiate coal and gangue using image processing and a support vector machine[J]. International Journal of Coal Preparation and Utilization,2018,41(8):603-616.
[18]	WANG Dongxing,NI Jingxiu,DU Tingyu. An image recognition method for coal gangue based on ASGS-CWOA and BP Neural Network[J]. Symmetry,2022,14(5):880. DOI: 10.3390/sym14050880.
[19]	LAI Wenhao,ZHOU Mengran,HU Feng,et al. A study of multispectral technology and two-dimension autoencoder for coal and gangue recognition[J]. IEEE Access,2020,8:61834-61843. doi: 10.1109/ACCESS.2020.2983740
[20]	ZHANG Qiang, GU Jieying, LIU Junming. Research on coal and rock type recognition based on mechanical vision[J]. Shock and Vibration, 2021. DOI: 10.1155/2021/6617717.
[21]	ESHAQ R M A,HU Eryi,QAID H A A M,et al. Using deep convolutional neural networks and infrared thermography to identify coal quality and gangue[J]. IEEE Access,2021,9:147315-147327. doi: 10.1109/ACCESS.2021.3121270
[22]	HU Feng,BIAN Kai. Accurate identification strategy of coal and gangue using infrared imaging technology combined with convolutional neural network[J]. IEEE Access,2022,10:8758-8766. doi: 10.1109/ACCESS.2022.3144386
[23]	JIANG Junhao,HAN Yanfeng,ZHAO Huijun,et al. Recognition and sorting of coal and gangue based on image process and multilayer perceptron[J]. International Journal of Coal Preparation and Utilization,2023,43(1):54-72. doi: 10.1080/19392699.2021.2002852
[24]	XING Jichuan,ZHAO Zimo,WANG Yaozhi,et al. Coal and gangue identification method based on the intensity image of lidar and DenseNet[J]. Applied Optics,2021,60(22):6566-6572. doi: 10.1364/AO.422498
[25]	BAI Feiyan,FAN Minqiang,YANG Hongli,et al. Fast recognition using convolutional neural network for the coal particle density range based on images captured under multiple light sources[J]. International Journal of Mining Science and Technology,2021,31(6):1053-1061. doi: 10.1016/j.ijmst.2021.09.004
[26]	杨伟姣,杨先海,薛鹏,等. 基于Halcon的固定视点手眼标定方法[J]. 机床与液压,2021,49(8):35-37,71. doi: 10.3969/j.issn.1001-3881.2021.08.007 YANG Weijiao,YANG Xianhai,XUE Peng,et al. Hand-eye calibration method of fixed viewpoint based on Halcon[J]. Machine Tool & Hydraulics,2021,49(8):35-37,71. doi: 10.3969/j.issn.1001-3881.2021.08.007