智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (2): 186-199.doi: 10.11959/j.issn.2096-6652.202222
崔少伟1,2, 王硕1,3,4, 胡静怡1,2, 张超凡1,3
出版日期:
2022-06-15
发布日期:
2022-06-01
作者简介:
崔少伟(1996− ),男,中国科学院自动化研究所博士生,主要研究方向为机器人触觉感知与灵巧操作基金资助:
Shaowei CUI1,2, Shuo WANG1,3,4, Jingyi HU1,2, Chaofan ZHANG1,3
Online:
2022-06-15
Published:
2022-06-01
Supported by:
摘要:
得益于高空间分辨率、多触觉模式的感知,视触觉传感技术已被广泛应用到机器人主动感知、位姿估计及掌内操作等各类机器人操作任务中。首先根据传感原理分类总结了目前主流的视触觉传感技术,主要可分为GelSight类视触觉传感器、双(多)目视触觉传感器以及其他类型视触觉传感器3类。在此基础上,进一步对主要触觉传感信息建模方法进行了介绍,包括接触表面三维几何形状、力/力矩、滑动等不同触觉模式。此外,聚焦机器人操作领域,对视触觉传感器的具体应用场景进行了探讨。最后,讨论了视触觉传感器的下一步发展方向以及如何将其进一步应用到机器人灵巧操作任务中。
中图分类号:
崔少伟, 王硕, 胡静怡, 等. 面向机器人操作任务的视触觉传感技术综述[J]. 智能科学与技术学报, 2022, 4(2): 186-199.
Shaowei CUI, Shuo WANG, Jingyi HU, et al. A survey of visuotactile sensing technologies for robotic manipulation[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 186-199.
表1
主流视触觉传感器结构设计、触觉传感功能及触觉模式"
传感器 | 结构形态 | 接触胶体层 | 光源 | 相机系统 | 几何重建 | 力估计 | 滑动感知 |
GelSight[ | 指尖 | 阵列标记点 | RGB | 单目 | 光度立体 | 神经网络 | 接触模型 |
GelSlim[ | 指尖 | 阵列标记点 | RGB | 单目 | 光度立体 | 有限元 | 接触模型 |
DIGIT[ | 指尖 | 无标记点 | RGB | 单目 | 待开发 | 无 | 无 |
GelTip[ | 手指 | 无标记点 | RGB | 单目 | 光度立体 | 无 | 无 |
InSight[ | 手指 | 无标记点 | RGB | 单目 | 光度立体 | 神经网络 | 无 |
DenseTact[ | 指尖 | 无标记点 | RGB | 单目 | 神经网络 | 无 | 无 |
Soft-bubble[ | 手掌 | 稠密标记点 | 单色 | 单目+深度 | 深度相机 | 无 | 无 |
GelStereo[ | 指尖 | 阵列标记点 | 单色 | 双目 | 双目立体 | 神经网络 | 神经网络 |
OmniTact[ | 指尖 | 无标记点 | RGB | 多目 | 光度立体 | 无 | 无 |
Tac3D[ | 手掌 | 阵列标记点 | 单色 | 虚拟双目 | 双目立体 | 有限元 | 接触模型 |
TacTip[ | 指尖 | 阵列标记探针 | 单色 | 单目 | 无 | 无 | 神经网络 |
GelForce[ | 指尖 | 双层标记点 | 单色 | 单目 | 无 | 有限元 | 无 |
Sf.& D’ A.[ | 手掌 | 稠密标记点 | 单色 | 单目 | 无 | 有限元 | 无 |
FingerVision[ | 指尖 | 阵列标记点 | 无 | 单目 | 无 | 无 | 接触模型 |
DelTact[ | 指尖 | 稠密像素图层 | 单色 | 单目 | 光流估计 | 有限元 | 神经网络 |
[6] | LI Q , KROEMER O , SU Z ,et al. A review of tactile information:perception and action through touch[J]. IEEE Transactions on Robotics, 2020,36(6): 1619-1634. |
[7] | CHI C , SUN X G , XUE N ,et al. Recent progress in technologies for tactile sensors[J]. Sensors (Basel,Switzerland), 2018,18(4): 948. |
[8] | ABAD A C , RANASINGHE A . Visuotactile sensors with emphasis on GelSight sensor:a review[J]. IEEE Sensors Journal, 2020,20(14): 7628-7638. |
[9] | YUAN W Z , DONG S Y , ADELSON E H . GelSight:high-resolution robot tactile sensors for estimating geometry and force[J]. Sensors (Basel,Switzerland), 2017,17(12): 2762. |
[10] | DONG S Y , YUAN W Z , ADELSON E H . Improved GelSight tactile sensor for measuring geometry and slip[C]// Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2017: 137-144. |
[11] | DONLON E , DONG S Y , LIU M ,et al. GelSlim:a high-resolution,compact,robust,and calibrated tactile-sensing finger[C]// Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2018: 1927-1934. |
[12] | TAYLOR I , DONG S Y , RODRIGUEZ A . GelSlim3.0:high-resolution measurement of shape,force and slip in a compact tactile-sensing finger[J]. arXiv preprint,2021,arXiv:2103.12269. |
[13] | WARD-CHERRIER B , PESTELL N , CRAMPHORN L ,et al. The TacTip family:soft optical tactile sensors with 3D-printed biomimetic morphologies[J]. Soft Robotics, 2018,5(2): 216-227. |
[1] | MYERS D . Sensation and perception.Psychology[M]. New York: Worth, 2010. |
[2] | ANSHEL J R . Visual ergonomics in the workplace[J]. AAOHN Journal, 2007,55(10): 414-420. |
[14] | ALSPACH A , HASHIMOTO K , KUPPUSWAMY N ,et al. Soft-bubble:a highly compliant dense geometry tactile sensor for robot manipulation[C]// Proceedings of 2019 2nd IEEE International Conference on Soft Robotics. Piscataway:IEEE Press, 2019: 597-604. |
[15] | YAMAGUCHI A , ATKESON C G . Combining finger vision and optical tactile sensing:reducing and handling errors while cutting vegetables[C]// Proceedings of 2016 IEEE-RAS 16th International Conference on Humanoid Robots. Piscataway:IEEE Press, 2016: 1045-1051. |
[3] | RANASINGHE A , SORNKARN N , DASGUPTA P ,et al. Salient feature of haptic-based guidance of people in low visibility environments using hard reins[J]. IEEE Transactions on Cybernetics, 2016,46(2): 568-579. |
[4] | AKHTAR N , MIAN A . Threat of adversarial attacks on deep learning in computer vision:a survey[J]. IEEE Access, 2018,6: 14410-14430. |
[16] | YAMAGUCHI A , ATKESON C G . Implementing tactile behaviors using FingerVision[C]// Proceedings of 2017 IEEE-RAS 17th International Conference on Humanoid Robotics. Piscataway:IEEE Press, 2017: 241-248. |
[17] | LAMBETA M , CHOU P W , TIAN S ,et al. DIGIT:a novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation[J]. IEEE Robotics and Automation Letters, 2020,5(3): 3838-3845. |
[18] | WANG S X , SHE Y , ROMERO B ,et al. GelSight Wedge:measuring high-resolution 3D contact geometry with a compact robot finger[C]// Proceedings of 2021 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2021: 6468-6475. |
[19] | DO W K , KENNEDY III M . DenseTact:optical tactile sensor for dense shape reconstruction[J]. arXiv preprint,2022,arXiv:2201.01367. |
[20] | GOMES D F , LIN Z L , LUO S . GelTip:a finger-shaped optical tactile sensor for robotic manipulation[C]// Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2021: 9903-9909. |
[21] | SUN H , KUCHENBECKER K J , MARTIUS G . A soft thumb-sized vision-based sensor with accurate all-round force perception[J]. Nature Machine Intelligence, 2022,4(2): 135-145. |
[22] | KUPPUSWAMY N , ALSPACH A , UTTAMCHANDANI A ,et al. Soft-bubble grippers for robust and perceptive manipulation[C]// Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2020: 9917-9924. |
[23] | PADMANABHA A , EBERT F , TIAN S ,et al. OmniTact:a multi-directional high-resolution touch sensor[C]// Proceedings of 2020 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2020: 618-624. |
[24] | ZHANG T , CONG Y , LI X M ,et al. Robot tactile sensing:vision based tactile sensor for force perception[C]// Proceedings of 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation,Control,and Intelligent Systems. Piscataway:IEEE Press, 2018: 1360-1365. |
[25] | CUI S W , WANG R , HU J Y ,et al. In-hand object localization using a novel high-resolution visuotactile sensor[J]. IEEE Transactions on Industrial Electronics, 2022,69(6): 6015-6025. |
[26] | KAKANI V , CUI X N , MA M J ,et al. Vision-based tactile sensor mechanism for the estimation of contact position and force distribution using deep learning[J]. Sensors (Basel,Switzerland), 2021,21(5): 1920. |
[27] | ZHANG L W , WANG Y , JIANG Y . Tac3D:a novel vision-based tactile sensor for measuring forces distribution and estimating friction coefficient distribution[J]. arXiv preprint,2022,arXiv:2202.06211. |
[28] | LEPORA N F . Soft biomimetic optical tactile sensing with the TacTip:a review[J]. IEEE Sensors Journal, 2021,21(19): 21131-21143. |
[29] | VLACK K , MIZOTA T , KAWAKAMI N ,et al. GelForce:a vision-based traction field computer interface[C]// Proceedings of CHI’05 Extended Abstracts on Human Factors in Computing Systems. New York:ACM Press, 2005: 1154-1155. |
[30] | SATO K , KAMIYAMA K , KAWAKAMI N ,et al. Finger-shaped GelForce:sensor for measuring surface traction fields for robotic hand[J]. IEEE Transactions on Haptics, 2010,3(1): 37-47. |
[31] | SFERRAZZA C , D'ANDREA R , . Design,motivation and evaluation of a full-resolution optical tactile sensor[J]. Sensors (Basel,Switzerland), 2019,19(4): 928. |
[32] | SFERRAZZA C , WAHLSTEN A , TRUEEB C ,et al. Ground truth force distribution for learning-based tactile sensing:a finite element approach[J]. IEEE Access, 2019,7: 173438-173449. |
[33] | SFERRAZZA C , D’ANDREA R , . Sim-to-real for high-resolution optical tactile sensing:from images to 3D contact force distributions[J]. arXiv preprint,2020,arXiv:2012.11295. |
[34] | ZHANG Y Z , YUAN W H , KAN Z C ,et al. Towards learning to detect and predict contact events on vision-based tactile sensors[J]. arXiv preprint,2019,arXiv:1910.03973. |
[35] | DU Y P , ZHANG G L , ZHANG Y Z ,et al. High-resolution 3-dimensional contact deformation tracking for FingerVision sensor with dense random color pattern[J]. IEEE Robotics and Automation Letters, 2021,6(2): 2147-2154. |
[36] | ZHANG G L , DU Y P , YU H Y ,et al. DelTact:a vision-based tactile sensor using dense color pattern[J]. arXiv preprint,2022,arXiv:2202.02179. |
[37] | JOHNSON M K , ADELSON E H . Retrographic sensing for the measurement of surface texture and shape[C]// Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009: 1070-1077. |
[38] | ROMERO B , VEIGA F , ADELSON E . Soft,round,high resolution tactile fingertip sensors for dexterous robotic manipulation[C]// Proceedings of 2020 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2020: 4796-4802. |
[39] | CUI S W , WANG R , HU J Y ,et al. Self-supervised contact geometry learning by GelStereo visuotactile sensing[J]. IEEE Transactions on Instrumentation and Measurement, 2022,71: 1-9. |
[40] | KAMIYAMA K , KAJIMOTO H , KAWAKAMI N ,et al. Evaluation of a vision-based tactile sensor[C]// Proceedings of IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2004: 1542-1547. |
[41] | MA D L , DONLON E , DONG S Y ,et al. Dense tactile force estimation using GelSlim and inverse FEM[C]// Proceedings of 2019 International Conference on Robotics and Automation. Piscataway:IEEE Press, 2019: 5418-5424. |
[42] | ZHANG Y Z , KAN Z C , YANG Y ,et al. Effective estimation of contact force and torque for vision-based tactile sensors with Helmholtz-Hodge decomposition[J]. IEEE Robotics and Automation Letters, 2019,4(4): 4094-4101. |
[43] | DONG S Y , MA D L , DONLON E ,et al. Maintaining grasps within slipping bounds by monitoring incipient slip[C]// Proceedings of 2019 International Conference on Robotics and Automation. Piscataway:IEEE Press, 2019: 3818-3824. |
[44] | SUI R M , ZHANG L W , LI T M ,et al. Incipient slip detection method with vision-based tactile sensor based on distribution force and deformation[J]. IEEE Sensors Journal, 2021,21(22): 25973-25985. |
[45] | LI J H , DONG S Y , ADELSON E . Slip detection with combined tactile and visual information[C]// Proceedings of 2018 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2018: 7772-7777. |
[46] | ZHANG Y Z , KAN Z C , TSE Y A ,et al. Fingervision tactile sensor design and slip detection using convolutional LSTM network[J]. arXiv preprint,2018,arXiv:1810.02653. |
[47] | JAMES J W , LEPORA N F . Slip detection for grasp stabilization with a multifingered tactile robot hand[J]. IEEE Transactions on Robotics, 2021,37(2): 506-519. |
[48] | ABAD A C , REID D , RANASINGHE A . HaptiTemp:a next-generation thermosensitive GelSight-like visuotactile sensor[J]. IEEE Sensors Journal, 2022,22(3): 2722-2734. |
[49] | YUAN W Z , SRINIVASAN M A , ADELSON E H . Estimating object hardness with a GelSight touch sensor[C]// Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2016: 208-215. |
[50] | CALANDRA R , OWENS A , UPADHYAYA M ,et al. The feeling of success:does touch sensing help predict grasp outcomes?[J]. arXiv preprint,2017,arXiv:1710.05512. |
[51] | CALANDRA R , OWENS A , JAYARAMAN D ,et al. More than a feeling:learning to grasp and regrasp using vision and touch[J]. IEEE Robotics and Automation Letters, 2018,3(4): 3300-3307. |
[52] | HOGAN F R , BAUZA M , CANAL O ,et al. Tactile regrasp:grasp adjustments via simulated tactile transformations[C]// Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2018: 2963-2970. |
[53] | DONG S Y , JHA D K , ROMERES D ,et al. Tactile-RL for insertion:generalization to objects of unknown geometry[C]// Proceedings of 2021 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2021: 6437-6443. |
[54] | WANG C , WANG S X , ROMERO B ,et al. SwingBot:learning physical features from in-hand tactile exploration for dynamic swing-up manipulation[C]// Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2021: 5633-5640. |
[55] | ANZAI T , TAKAHASHI K . Deep gated multi-modal learning:in-hand object pose changes estimation using tactile and image data[C]// Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2021: 9361-9368. |
[56] | LI R , PLATT R , YUAN W Z ,et al. Localization and manipulation of small parts using GelSight tactile sensing[C]// Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2014: 3988-3993. |
[57] | BAUZA M , VALLS E , LIM B ,et al. Tactile object pose estimation from the first touch with geometric contact rendering[J]. arXiv preprint,2020,arXiv:2012.05205. |
[58] | BAUZA M , CANAL O , RODRIGUEZ A . Tactile mapping and localization from high-resolution tactile imprints[C]// Proceedings of 2019 International Conference on Robotics and Automation. Piscataway:IEEE Press, 2019: 3811-3817. |
[59] | LEPORA N F , LLOYD J . Optimal deep learning for robot touch:training accurate pose models of 3D surfaces and edges[J]. IEEE Robotics & Automation Magazine, 2020,27(2): 66-77. |
[60] | SHE Y , WANG S X , DONG S Y ,et al. Cable manipulation with a tactile-reactive gripper[J]. The International Journal of Robotics Research, 2021,40(12/13/14): 1385-1401. |
[61] | KIM S , RODRIGUEZ A . Active extrinsic contact sensing:application to general peg-in-hole insertion[J]. arXiv preprint,2021,arXiv:2110.03555. |
[62] | LAMBETA M , XU H Z , XU J W ,et al. PyTouch:a machine learning library for touch processing[C]// Proceedings of 2021 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2021: 13208-13214. |
[63] | SORENSEN S , KOLAGUNDA A , SAPONARO P ,et al. Refractive stereo ray tracing for reconstructing underwater structures[C]// Proceedings of 2015 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2015: 1712-1716. |
[5] | BILLARD A , KRAGIC D . Trends and challenges in robot manipulation[J]. Science, 2019,364(6446): eaat8414. |
[64] | FISHEL J A , LOEB G E . Sensing tactile microvibrations with the BioTac—comparison with human sensitivity[C]// Proceedings of 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics. Piscataway:IEEE Press, 2012: 1122-1127. |
[1] | 曹翔, 孙长银. 基于不确定事件威胁度评估的UUV任务重规划[J]. 智能科学与技术学报, 2022, 4(4): 493-502. |
[2] | 张佳欣, 张森林, 刘妹琴, 董山玲, 郑荣濠. 面向海洋环境自适应采样的多AUV协同定位[J]. 智能科学与技术学报, 2022, 4(4): 503-512. |
[3] | 蔡莹皓, 杨华, 安璇, 王文硕, 杜沂东, 张嘉韬, 王志刚. 神经符号学及其应用研究[J]. 智能科学与技术学报, 2022, 4(4): 560-570. |
[4] | 赵超, 许杰, 陈星宇, 梅魁志, 兰旭光. 机器人持续学习进展与展望[J]. 智能科学与技术学报, 2022, 4(3): 308-323. |
[5] | 徐德, 秦方博. 机器人自动轴孔装配研究进展[J]. 智能科学与技术学报, 2022, 4(2): 200-211. |
[6] | 葛悦光, 张少林, 蔡莹皓, 鲁涛, 温大勇, 王海涛, 王硕. 本体知识表示方法在机器人领域的应用研究综述[J]. 智能科学与技术学报, 2022, 4(2): 212-222. |
[7] | 胡静怡, 崔少伟, 张超凡, 张伯约, 王硕. 基于触觉感知和伺服的物体三维边缘重建方法[J]. 智能科学与技术学报, 2022, 4(2): 233-245. |
[8] | 董璐, 熊爱玲. 基于改进RRT*-Smart的复杂动态环境下的无人艇路径规划[J]. 智能科学与技术学报, 2022, 4(2): 264-276. |
[9] | 王腾, 潘晶, 董璐, 孙长银. 面向防疫的智能导诊机器人关键技术及应用[J]. 智能科学与技术学报, 2021, 3(2): 187-194. |
[10] | 王日中, 李慧平, 崔迪, 徐德民. 基于深度强化学习算法的自主式水下航行器深度控制[J]. 智能科学与技术学报, 2020, 2(4): 354-360. |
[11] | 傅汇乔, 唐开强, 邓归洲, 王鑫鹏, 陈春林. 基于深度强化学习的六足机器人运动规划[J]. 智能科学与技术学报, 2020, 2(4): 361-371. |
[12] | 王大伟,王卓,王鹏,王志刚,王海涛. 基于边缘计算的云原生机器人系统[J]. 智能科学与技术学报, 2020, 2(3): 275-283. |
[13] | 付海军,陈世超,林懿伦,熊刚,胡斌. 人在回路的混合增强智能在Sawyer的研究与验证[J]. 智能科学与技术学报, 2019, 1(3): 280-286. |
[14] | 张翔,刘红卫,刘卓群,闫振国,陈小前,黄奕勇. 空间智能软体机械臂动力学建模与控制[J]. 智能科学与技术学报, 2019, 1(1): 52-61. |
[15] | 郑亚君,薛磊,董璐,王庆领. 基于UWB定位技术的多移动机器人编队控制[J]. 智能科学与技术学报, 2019, 1(1): 83-87. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|