智能科学与技术学报 ›› 2019, Vol. 1 ›› Issue (1): 70-82.doi: 10.11959/j.issn.2096-6652.201909

• 学术论文 • 上一篇    下一篇

基于半监督生成对抗网络的三维重建云工作室

余翀()   

  1. 英伟达半导体科技(上海)有限公司,上海 201210
  • 修回日期:2019-03-01 出版日期:2019-03-20 发布日期:2019-05-28
  • 作者简介:余翀(1989- ),男,复旦大学硕士研究生,英伟达半导体科技有限公司高级架构师,英特尔亚太研发有限公司高级工程师,主要研究领域为人工智能与机器学习、机器视觉与图像处理、无人机与导航系统、智能控制理论与机器人系统、物联网技术与可穿戴设备等。

Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks

Chong YU()   

  1. NVIDIA Semiconductor Technology Co.,Ltd.,Shanghai 201210,China
  • Revised:2019-03-01 Online:2019-03-20 Published:2019-05-28

摘要:

由于固有的问题复杂性和计算复杂度,三维重建是计算机视觉研究和应用领域非常重要且富有挑战性的课题。目前已有的三维重建算法往往会导致重建的三维模型上存在着明显的空洞、扭曲失真或者模糊不清的部分,而基于机器学习的三维重建算法往往又只能重建简单的分离物体,并表示成三维体元形式。所以这些算法框架对于实际应用来说都还远远不够。从 2014 年起,生成对抗网络被广泛应用于半监督学习,以及产生非真实数据集的应用中。所以本文的重点是采用生产对抗网络原理,来获得高质量的三维重建效果。提出了一种新颖的半监督三维重建算法架构,命名为 SS-GAN-3D。该算法通过训练生成对抗网络模型,使其达到收敛状态,以此来迭代式地提高原始三维重建模型的质量。SS-GAN-3D 只需要将事先观测的二维图像作为弱监督样本,对于三维结构外形的先验知识或者参考观测都没有任何依赖。最终通过定性和定量实验,以及对实验结果的分析,该算法框架在 Tanks & Temples 和 ETH3D 标准三维重建测试集上,比目前最先进的三维重建方法有明显优势。基于SS-GAN-3D算法,又提出了三维重建云工作室解决方案。

关键词: 三维重建, 生成对抗网络, 半监督学习, 云工作室

Abstract:

Because of the intrinsic complexity in computation,three-dimensional (3D) reconstruction is an essential and challenging topic in computer vision research and applications.The existing methods for 3D reconstruction often produce holes,distortions and obscure parts in the reconstructed 3D models.While the 3D reconstruction algorithms based on machine learning can only reconstruct voxelized 3D models for simple isolated objects,they are not adequate for real usage.From 2014,the generative adversarial network (GAN) is widely used in generating unreal dataset and semi-supervised learning.So the focus of this paper is to achieve high quality 3D reconstruction performance by adopting GAN principle.A novel semi-supervised 3D reconstruction framework,namely SS-GAN-3D was proposed,which can iteratively improve any raw 3D reconstruction models by training the GAN models to converge.This new model only takes 2D observation images as the weak supervision,and doesn’t rely on prior knowledge of shape models or any referenced observations.Finally,through qualitative and quantitative experiments and analysis,this new method shows compelling advantages over the current state-of-the-art methods on Tanks & Temples and ETH3D reconstruction benchmark datasets.Based on SS-GAN-3D,the 3D reconstruction studio solution was proposed.

Key words: three-dimensional reconstruction, generative adversarial network, semi-supervised learning, cloud studio

中图分类号: 

No Suggested Reading articles found!