ToF相机的有效深度数据提取与校正算法研究

doi:10.11959/j.issn.2096-6652.202008

智能科学与技术学报 ›› 2020, Vol. 2 ›› Issue (1): 72-79.doi: 10.11959/j.issn.2096-6652.202008

ToF相机的有效深度数据提取与校正算法研究

乔欣,葛晨阳(),邓鹏超,周艳辉,姚慧敏

西安交通大学人工智能与机器人研究所，陕西西安 710049

修回日期:2020-02-28 出版日期:2020-03-20 发布日期:2020-04-10
作者简介:乔欣（1990– ），男，西安交通大学博士生，主要研究方向为计算机视觉、3D 感知和多传感融合|葛晨阳（1977– ），男，博士，西安交通大学副教授，主要研究方向为计算视觉、三维感知、SoC设计|邓鹏超（1990– ），男，西安交通大学硕士生，主要研究方向为计算视觉和深度学习|周艳辉（1981– ），女，博士，西安交通大学副教授，主要研究方向为智能信号处理、反问题方法研究及应用|姚慧敏（1986– ），女，博士，西安交通大学讲师，主要研究方向为计算机视觉、结构光成像和图像处理
基金资助:
国家重大科研仪器研制基金资助项目(61627811);国家自然科学基金资助项目(61571358)

Research on valid depth data extraction and correction for ToF camera

Xin QIAO,Chenyang GE(),Pengchao DENG,Yanhui Zhou,Huimin Yao

Institute of Artificial Intelligence and Robotics,Xi’an Jiaotong University,Xi’an 710049,China

Revised:2020-02-28 Online:2020-03-20 Published:2020-04-10
Supported by:
The National Important Development Program on Scientific Research Instruments(61627811);The National Natural Science Foundation of China(61571358)

摘要/Abstract

摘要：

提出一种针对 ToF 相机的有效深度数据提取与校正算法，利用深度图和置信度图对深度信息进行校正。首先，基于核密度估计和连通域标记对测得的深度图进行自适应分割；然后使用一种改进的结构张量进行边缘检测，从而探测深度图中的无效像素和飞行像素；最后用双三次方插值或投票操作纠正或剔除这些像素，同时使用增强置信度剔除错误像素。实验结果证明了该算法的有效性，对比传统方法，本文所提算法可以剔除更多的无效像素，且保留更多的有效深度数据，对于噪声的鲁棒性也更好。

关键词: ToF相机, 有效深度数据, 飞行像素, 自适应分割, 边缘检测, 增强置信度

Abstract:

An algorithm was proposed to extract the valid depth information and correct the values of flying pixels with depth map and confidence map for time-of-flight camera.Firstly,the depth map was segmented adaptively based on kernel density estimation and connected component labeling.Then edge detection was performed by using a modified structure tensor to recognize the invalid pixels and the flying pixels.At last,the values of flying pixels were corrected with the bi-cubic interpolation and that of the invalid pixels were deleted by voting.Meanwhile,using augmented confidence,the pixels with wrong depth values were removed.Experimental results show the effectiveness of the proposed algorithm.Comparing with the conventional methods,the proposed algorithm can remove more invalid pixels and remain more valid depth data.Also,the proposed method is more robust to noise.

Key words: ToF camera, valid depth data, flying pixels, adaptive segmentation, edge detection, augmented confidence

中图分类号:

TP751

乔欣, 葛晨阳, 邓鹏超, 等. ToF相机的有效深度数据提取与校正算法研究[J]. 智能科学与技术学报, 2020, 2(1): 72-79.

Xin QIAO, Chenyang GE, Pengchao DENG, et al. Research on valid depth data extraction and correction for ToF camera[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(1): 72-79.

图/表 10

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

参考文献 17

[1]	HE Y , CHE N S . Recent advancees in 3D data acquiisition and processin ng by time-of-fliight camera[J]. IEEE Access, 2019,7: 112495-12510.
[2]	HAASE J F , BEER M , RUSKOWSKI J ,et al. Multi object detectio on in direct time-of-flight measurrements with SPADs[C]// IEEE 20 18 14th Confereence on Ph.D.Reseearch in Microelecttronics and Electro onics(PRIME). Piscataway:IEEE Press, 2018: 237-2239.
[3]	PINTO A M , COSTA P , MOREIRA A P ,et al. Evaluation of deppth sensors for rrobotic application ns[C]// 2015 IEEE International Conf ference on Auttonomous Robot Systems and Comppetitions. Piscatawaay:IEEE Press, 22015: 139-143.
[4]	NGUYEN T , HUYNH H , MEUNIER J . 3D reconstruction wi ith time-of-flighht depth camera an nd multiple mirroors[J]. IEEE Acces ss, 2018,6: 381006-38114.
[5]	INOUE Y , NISHIDE S , REN F J . Facial expression recognition ada aptive to face ppose using RGB-D camera[C]// 2019 International Confference on Indu ustrial,Engineering g and Other Appliccations Applied Inteelligent Systems. Heidelberg:Sprringer, 2016: 422-427.
[6]	SCH?NLIEB A , HANNES P , STEGER C ,et al. Hybrid Sensinng approach forr coded modulation n Time-of-Flight ccameras[C]// Desig gn,Automation ＆ Test in Europe Conference ＆ Exhhibition(DATE). Pi iscataway:IE EE Press, 2019: 1076-1081.
[7]	ZANUTTIGH P , MARIN G , MUTTO C D ，et al. Time-of-flight annd structured ligght depth cameras[M]. Heidelberg: SppringerPress, 2016.
[8]	SABOV A , KRüGER J , . Identiification and correc ction of flying pixe els in range camera data[C]// The 24th Spring Confference on Comput ter Graphics. Ne ew York:ACM Presss, 2008: 135-142.
[9]	ZHANG H , RONG X , LI Y ,et al. Calibration aand gray-level imag ge generation ffoor the SR4500 TooF camera[J]. Interrnational Journal of Control and Automation, 2016,9(6): 277-288.
[10]	JANN P , BI RK A . A novel ap pproach to efficien tt error correction f for the Swiss Raanger time-of-flightt 3D camera[C]// RoboCup 2008:Rob bot Soccer Worldd Cup XII. Heidelberg:Springer, 2008,8: 247-258.
[11]	MUTTO C D , ZANUTTIGH P , CORTELAZZ O G M . Probabilist tic ToF and ster eo data fusion baseed on mixed pixelss measurement mo odels[J]. IEEE Transactions on Pattern Analysis aand Machine Intellligence, 2015,37(11): 2260-2272
[12]	ZHANG Z Y . A flexible new tec chnique for cameraa calibration[J]. IEEE Transactions on Pattern Analysis and Maachine Intelligenc ce, 2000,22(11): 1330-1334.
[13]	PARVIZI E , WU Q M J . Multtiple object trackinng based on adaptiv ve depth segmenntation[C]// 2008 Canadian Conferennce on Computer an nd Robot Visionn. Piscataway:IEEE Press, 2008: 273-277.
[14]	RUBIO D O , LENSKIY A , RYU J . Connected co omponents for a fa ast and robust 2D LiDAR data seggmentation[C]// Thee 7th Asia Modellin ng Symposium. Piscataway:IEEE Press, 2013: 160-165.
[15]	F?RSTNER W , GüLCH E , . A fast operator for d detection and preci ise location of d istinct points,cornners and centers of circular features[C]// Proceedings of ISPRS Inter r Commission Conference on Fa ast Processing off Photogrammetric Data.[S.l.:s.n]. 1998: 281-305.
[16]	ZHONG Y T , WANG Y , PIAO Y . Depth image interpolation n algorithm base ed on confidence map[C]// Chinese Society for Optical l Engineering Conference.[S.l.:s.n]. 2017:102550D.
[17]	VIOLA P , WELLS W . Alignment by max ximization of mutuual informationn[J]. International Journal of Computer Vision, 1997,24(2): 137-154.

算法	最小相似度	平均相似度	最大相似度
AAT	0.000 6	0.05	0.13
FDT	0.04	0.12	0.27
ADT	0.06	0.2	0.3
本文所提算法	0.11	0.35	0.54

ToF相机的有效深度数据提取与校正算法研究

Research on valid depth data extraction and correction for ToF camera

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 17

相关文章 0

Metrics

推荐阅读 0