基于AR虚实图像注意力机制的电缆装配质量检测方法

doi:10.11959/j.issn.2096-3750.2021.00236

Abstract

Abstract:

Cable assembly in spacecraft occupies a large proportion in the final assembly process, and the quality of cable assembly directly affects the working performance of the whole product.The existing augmented reality-assisted cable assembly system lacks the ability to make real-time judgment and feedback on the assembly results, and cannot identify whether the currently assembled cable meets the quality requirements.In order to solve the above problems, a cable assembly quality inspection method based on AR virtual-real image attention mechanism was proposed.Critical areas in the captured images were located and detected by constructing virtual assist models to filter redundant image information.Based on YOLOv4-Tiny image detection model, the key node location of assembled cable was judged.The overlap of the laying path was obtained based on the method of calculating the average overlap of the neighborhood, and the bending radius of the cable was derived based on the method of camera inverse projection.Finally, an augmented reality-based cable assembly quality inspection system was developed using HoloLens2.The quality inspection was performed on the four designed cable assembly cases, and the feasibility and effectiveness of the method and system were experimentally verified.

Key words: cable assembly, quality inspection, augmented reality assisted assembly, AR virtual-real image, attention mechanism

CLC Number:

TP391.9
V465

Ganlin ZHAO, Chang YU, Jianfu ZHANG, Jianxin YANG, Pingfa FENG, Qun SHEN. Inspection method for cable assembly quality based on AR virtual-real image attention mechanism[J]. Chinese Journal on Internet of Things, 2021, 5(3): 27-38.

Figures/Tables 17

References 20

[1]	GOODDING J C , ARDELEAN E V , BABUSKA V ,et al. Experimental techniques and structural parameter estimation studies of spacecraft cables[J]. Journal of Spacecraft and Rockets, 2011,48(6): 942-957.
[2]	ARDELEAN E V , BABU?KA V , GOODDING J C ,et al. Cable effects study:tangents,rabbit holes,dead ends,and valuable results[J]. Journal of Spacecraft and Rockets, 2014,52(2): 569-583.
[3]	WANG X , ONG S K , NEE A Y C . A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016,4(1): 1-22.
[4]	CAUDELL T P , MIZELL D W . Augmented reality:an application of heads-up display technology to manual manufacturing processes[C]// Proceedings of the 25th Hawaii International Conference on System Sciences.[S.l.:s.n.], 1992,2: 659-669.
[5]	MIZELL D . Boeing's wire bundle assembly project[M]. Fundamentals of Wearable Computers and Augmented Reality. Boca Raton: CRC Press, 2001: 462-482.
[6]	SERVáN J , MAS F , MENéNDEZ J L ,et al. Using augmented reality in AIRBUS A400M shop floor assembly work instructions[C]// AIP Conference Proceedings. Melville:American Institute of Physics,2012, 1431（1）： 633-640.
[7]	ZHENG L Y , LIU X , AN Z W ,et al. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection[J]. Virtual Reality ＆ Intelligent Hardware, 2020,2(1): 12-27.
[8]	ZHANG W , CHEN C Y , CHEN H J ,et al. Application of augmented reality in satellite cable network assembly[C]// IOP Conference Series:Materials Science and Engineering. Bristol:IOP Publishing, 2019,54(1): 012034.
[9]	陈华俊, 陈畅宇, 孙刚 ,等. 增强现实在航天器电缆装配中的应用技术研究[J]. 科技与创新, 2019(8): 40-44.
	CHEN H J , CHEN C Y , SUN G ,et al. Research on the application technology of augmented reality in spacecraft cable assembly[J]. Science and Technology ＆ Innovation, 2019(8): 40-44.
[10]	PARMAR P , . Use of computer vision to detect tangles in tangled objects[C]// Proceedings of 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013). Piscataway:IEEE Press, 2013: 39-44.
[11]	SUN F J , WANG H , FAN J Q . 2D otsu segmentation algorithm based on simulated annealing genetic algorithm for iced-cable images[C]// Proceedings of 2009 International Forum on Information Technology and Applications. Piscataway:IEEE Press, 2009: 600-602.
[12]	温沛涵, 吴梦杰, 易树平 ,等. 基于目标识别的软管装配视觉检测系统[J]. 计算机集成制造系统, 2015,21(6): 1486-1496.
	WEN P H , WU M J , YI S P ,et al. Visual inspection system for hose assembly based on object recognition[J]. Computer Integrated Manufacturing Systems, 2015,21(6): 1486-1496.
[13]	焦晨航 . 约束空间中电缆线安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2019.
	JIAO C H . Research on cable installation method in constrained space[D]. Harbin:Harbin Institute of Technology, 2019.
[14]	刘睿, 尹旭悦, 范秀敏 ,等. 面向 AR 辅助引导的电缆敷设质量一致性视觉检测方法[J]. 计算机集成制造系统, 2021,27(1): 53-62.
	LIU R , YIN X Y , FAN X M ,et al. AR guidance-based visual inspection method for cable laying quality consistency check[J]. Computer Integrated Manufacturing Systems, 2021,27(1): 53-62.
[15]	魏煌涛 . 基于视觉的电缆安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2020.
	WEI H T . Research on installation method of cable based on vision[D]. Harbin:Harbin Institute of Technology, 2020.
[16]	冒文彦 . 基于卷积神经网络的高铁电缆线夹装配质量检测技术研究[D]. 镇江:江苏大学, 2020.
	MAO W Y . Research on assembly quality inspection technology of high-speed rail cable clamps based on convolutional neural network[D]. Zhenjiang,China:Jiangsu University, 2020.
[17]	LI B , CHEN C . Transmission line detection based on a hierarchical and contextual model for aerial images[J]. Journal of Electronic Imaging, 2018,27(4): 1:043054.
[18]	LI S F , ZHENG P , ZHENG L Y . An AR-assisted deep learning-based approach for automatic inspection of aviation connectors[J]. IEEE Transactions on Industrial Informatics, 2021,17(3): 1721-1731.
[19]	BOCHKOVSKIY A , WANG C Y , LIAO H Y MARK . YOLOv4:optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[20]	EVERINGHAM M , WINN J . The PASCAL visual object classes challenge 2007 (voc2007) development kit[J]. International Journal of Computer Vision, 2006,111(1): 98-136.

Metrics

Recommended 0

No Suggested Reading articles found!

标准类型	评价内容	量化指标
装配位置标准	电缆关键节点装配位置与设计的误差	≤25 mm
装配几何标准	电缆敷设路径与设计路径重合度	≥85%
	电缆最小弯曲半径（电缆外径为d）	≥10 d

编号					检测结果
	节点	E1.1	M1.2	M1.3	M1.4	M1.5	E1.6
电缆1	检测物体	connector	clip	clip	clip	clip	connector
	置信度	1.00	0.68	1.00	0.81	0.98	1.00
	节点	M2.1	M2.2	E2.3
电缆2	检测物体	未检出	clip	未检出
	置信度	无	1.00	无
	节点	M3.1	M3.2	M3.3	M3.4	M3.5	M3.6	E3.7
电缆3	检测物体	clip	clip	clip	未检出	clip	clip	connector
	置信度	0.99	1.00	0.98	无	0.94	1.00	1.00
	节点	E4.1	M4.2	M4.3	M4.4
电缆4	检测物体	connector	clip	clip	clip
	置信度	1.00	0.96	1.00	0.91

电缆编号	扩倍模型像素/个	重叠区域像素/个	邻域平均重合度	实际重合度
电缆1	162 259	47 111	87.10%	91.14%
电缆2	100 861	20 045	59.62%	54.43%
电缆3	168 389	27 395	48.81%	44.76%
电缆4	147 183	45 735	93.22%	86.74%

区域编号	电缆1-1	电缆1-2	电缆2-1	电缆3-1	电缆4-1	电缆4-2
区域图像
检测半径/mm	104.79	88.32	124.30	40.65	139.11	94.55
实际半径/mm	106.65	90.53	122.17	42.64	133.99	93.06
检测误差率	1.74%	2.44%	1.74%	4.66%	3.82%	1.60%

电缆编号	关键节点检测	敷设路径检测	弯曲半径检测	质量检测结果
电缆1	合格	合格	合格	合格
电缆2	不合格	不合格	合格	不合格
电缆3	不合格	不合格	不合格	不合格
电缆4	合格	合格	合格	合格

Inspection method for cable assembly quality based on AR virtual-real image attention mechanism

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 20

Related Articles 2

Metrics

Recommended 0

[1]	Zhifei ZHANG, Feng LIU, Yiyang GE, Shuo LI, Yu ZHANG, Ke XIONG. An intrusion detection method based on depthwise separable convolution and attention mechanism [J]. Chinese Journal on Internet of Things, 2023, 7(1): 49-59.
[2]	Chi JIN,Zhijun LI,Dayang SUN,Fengye HU. Human activity recognition algorithm based on the spatial feature for WBAN [J]. Chinese Journal on Internet of Things, 2019, 3(3): 70-75.