物联网学报 ›› 2019, Vol. 3 ›› Issue (4): 63-71.doi: 10.11959/j.issn.2096-3750.2019.00133
修回日期:
2019-10-31
出版日期:
2019-12-30
发布日期:
2020-02-05
作者简介:
桑永胜(1974- ),男,四川绵阳人,四川大学计算机学院副教授,主要研究方向为类脑智能、机器视觉以及深度学习等|李仁昊(1997- ),男,北京人,四川大学计算金融专业学生,主要研究方向为计算机视觉、数据挖掘等|李耀仟(1997- ),男,广西崇左人,四川大学计算金融专业学生,主要研究方向计算机视觉、大数据分析。|王蔷薇(1981- ),男,四川绵阳人,四川旅游学院讲师,主要研究方向为智能数据分析。|毛耀(1978- ),男,四川眉山人,中国科学院光电技术研究所研究员,主要研究方向为光束控制及计算机管理技术,包括预测跟踪、惯性稳定、自主控制、机器视觉、强化学习等。
基金资助:
Yongsheng SANG1,Renhao LI1,Yaoqian LI1,Qiangwei WANG2,Yao MAO3()
Revised:
2019-10-31
Online:
2019-12-30
Published:
2020-02-05
Supported by:
摘要:
神经形态视觉传感器是一种模拟生物视觉系统工作机理的传感器,具有高时间分辨率、低时延、低功耗以及高动态范围等特点。首先,介绍了神经形态工程的研究背景、神经形态芯片以及神经形态视觉传感器的工作机理和主要优点。然后,详细介绍了神经形态视觉的主要计算方法,包括概率统计方法、脉冲神经网络方法以及深度神经网络方法等。最后,综述了神经形态视觉传感器在同步定位与地图构建、图像重建以及特征检测与跟踪等方面的应用研究。对神经形态视觉传感器从硬件、计算方法和应用等方面作了系统概述,为相关研究者提供了全面的参考。
中图分类号:
桑永胜,李仁昊,李耀仟,王蔷薇,毛耀. 神经形态视觉传感器及其应用研究[J]. 物联网学报, 2019, 3(4): 63-71.
Yongsheng SANG,Renhao LI,Yaoqian LI,Qiangwei WANG,Yao MAO. Research on neuromorphic vision sensor and its applications[J]. Chinese Journal on Internet of Things, 2019, 3(4): 63-71.
[1] | 陶建华, 陈云霁 . 类脑计算芯片与类脑智能机器人发展现状与思考[J]. 中国科学院院刊, 2016,31(7): 803-811. |
TAO J H , CHEN Y J . Current status and consideration on brain-like computing chip and brain-like intelligent robot[J]. Bulletin of Chinese Academy of Sciences, 2016,31(7): 803-811. | |
[2] | MEAD C . Neuromorphic electronic systems[J]. Proceedings of the IEEE, 1990,78(10): 1629-1636. |
[3] | NAWROCKI R A , VOYLES R M , SHAHEEN S E . A mini review of neuromorphic architectures and implementations[J]. IEEE Transactions on Electron Devices, 2016,63(10): 3819-3829. |
[4] | BOAHEN K . Neuromorphic microchips[J]. Scientific American, 2005,292(5): 56-63. |
[5] | HASLER J , MARR B . Finding a roadmap to achieve large neuromorphic hardware systems[J]. Frontiers in Neuroscience, 2013,7:118. |
[6] | GAO P , BENJAMIN B V , BOAHEN K . Dynamical system guided mapping of quantitative neuronal models onto neuromorphic hardware[J]. IEEE Transactions on Circuits and Systems I:Regular Papers, 2012,59(10): 2383-2394. |
[7] | MEROLLA P A , ARTHUR J V , ALVAREZ-ICAZA R ,et al. A million spiking-neuron integrated circuit with a scalable communication network and interface[J]. Science, 2014,345(6197): 668-673. |
[8] | YU S , CHEN P Y , CAO Y ,et al. Scaling-up resistive synaptic arrays for neuro-inspired architecture:challenges and prospect[C]// Electron Devices Meeting. IEEE, 2016. |
[9] | MA L , XU Q , YANG Y . Organic nonvolatile memory by controlling the dynamic copper—ion concentration within organic layer[J]. Applied Physics Letters, 2004,84(24): 4908-4910. |
[10] | BLOM P W , SMIT C W , HAVERKORT J E ,et al. Carrier capture into a semiconductor quantum well[J]. Physical Review B,Condense Matter, 1993,47(4): 2072-2081. |
[11] | SON D I , YOU C H , KIM W T ,et al. Electrical bistabilities and memory mechanisms of organic bistable devices based on colloidal ZnO quantum dot-polymethylmethacrylate polymer nanocomposites[J]. Applied Physics Letters, 2009,94(13):>132103. |
[12] | HA H , KIM O . Electrode-material-dependent switching characteristics of organic nonvolatile memory devices based on Poly(3,4ethylenedioxythiophene):poly (styrenesulfonate) film[J]. IEEE Electron Device Letters, 2010,31(4): 368-370. |
[13] | MüLLER S , PERLOV C , JACKSON W ,et al. A polymer/ semiconductor write-once read-many-times memory[J]. Nature, 2003,426(6963): 166-169. |
[14] | NAWROCKI R A , VOYLES R M , SHAHEEN S E . Neurons in polymer:hardware neural units based on polymer memristive devices and polymer transistors[J]. IEEE Transactions on Electron Devices, 2014,61(10): 3513-3519. |
[15] | VANARSE A , OSSEIRAN A , RASSAU A . A review of current neuromorphic approaches for vision,auditory,and olfactory sensors[J]. Frontiers in Neuroscience, 2016,10(275):115. |
[16] | SARPESHKAR R , LYON R F , MEAD C . A low-power wide-dynamicrange analog VLSI cochlea[J]. Analog Integrated Circuits and Signal Processing, 1998,16(3): 245-274. |
[17] | WATTS L , KERNS D A , LYON R F ,et al. Improved implementation of the silicon cochlea[J]. IEEE Journal of Solid-State Circuits, 1992,27(5): 692-700. |
[18] | SARPESHKAR R , SALTHOUSE C , SIT J J ,et al. An ultra-low-power programmable analog bionic ear processor[J]. IEEE Transactions on Biomedical Engineering, 2005,52(4): 711-727. |
[19] | CHAN V , LIU S C , VAN SCHAIK A . AER EAR:a matched silicon cochlea pair with address event representation interface[J]. IEEE Transactions on Circuits and Systems I:Regular Papers, 2007,54(1): 48-59. |
[20] | LIU S C , VAN SCHAIK A , MINCH B A ,et al. Asynchronous binaural spatial audition sensor with 2×64×4 channel output[J]. IEEE transactions on biomedical circuits and systems, 2013,8(4): 453-464. |
[21] | POSCH C , SERRANO-GOTARREDONA T , LINARES- BARRANCO B . Retinomorphic event-based vision sensors:bioinspired cameras with spiking output[J]. Proceedings of the IEEE, 2014,102(10): 1470-1484. |
[22] | LICHTSTEINER P , POSCH C , DELBRUCK T . A 128×128 120dB 15μs latency asynchronous temporal contrast vision sensor[J]. IEEE Journal of Solid-State Circuits, 2008,43(2): 566-576. |
[23] | BERNER R , BRANDLI C , YANG M ,et al. A 240×180 10mW 12us latency sparse-output vision sensor for mobile applications[C]// VLSI Circuits (VLSIC),2013 Symposium on IEEE. IEEE, 2013. |
[24] | TOBI D , MEAD C A . Time-derivative adaptive silicon photoreceptor array[J]. Proceedings of SPIE-the International Society for Optical Engineering, 1993,1541: 92-99. |
[25] | GALLEGO G , DELBRUCK T , ORCHARD G ,et al. Event-based vision:a survey[J]. arXiv:Computer Vision and Pattern Recognition, 2019. |
[26] | POSCH C , MATOLIN D , WOHLGENANNT R . An asynchronous time-based image sensor[C]// IEEE International Symposium on Circuits & Systems. IEEE, 2008. |
[27] | CHEN S , AKSELROD P , ZHAO B ,et al. Efficient feedforward categorization of objects and human postures with address-event image sensors[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2012,34(2):302. |
[28] | ZHAO B , DING R , CHEN S ,et al. Feedforward categorization on AER motion events using cortex-like features in a spiking neural network[J]. IEEE Transactions on Neural Networks, 2015,26(9): 1963-1978. |
[29] | BICHLER O , QUERLIOZ D , THORPE S J ,et al. Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity[J]. Neural Networks, 2012,32(2): 339-348. |
[30] | LI H , LI G , SHI L . Classification of spatiotemporal events based on random forest[C]// International Conference on Brain Inspired Cognitive Systems. 2016: 138-148. |
[31] | SHAISTA H , ARINDAM B . Multiclass classification by adaptive network of dendritic neurons with binary synapses using structural plasticity[J]. Frontiers in Neuroscience, 2016,10:113. |
[32] | PENG X , ZHAO B , YAN R ,et al. Bag of events:an efficient probability-based feature extraction method for AER image sensors[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016: 1-13. |
[33] | ZHANG D , LU G . Review of shape representation and description techniques[J]. Pattern recognition, 2004,37(1): 1-19. |
[34] | LI H , LI G , JI X ,et al. Deep representation via convolutional neural network for classification of spatiotemporal event streams[J]. Neurocomputing, 2018:S0925231218301486. |
[35] | 吴金建 . 基于人类视觉系统的图像信息感知和图像质量评价[D]. 西安:西安电子科技大学, 2014. |
WU J J . Image information perception and quality assessment based on the human visual system[D]. Xi’an:Xidian University, 2014. | |
[36] | QU X , ZHAO F , ZHOU M ,et al. A novel fast and robust binary affine invariant descriptor for image matching[J]. Mathematical Problems in Engineering, 2014,2014(1): 1-7. |
[37] | LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553):436. |
[38] | SCHMIDHUBE R JüRGE N . Deep learning in neural networks:an overview[J]. Neural Networks, 2015,61: 85-117. |
[39] | CIRE?AN D , MEIER U , SCHMIDHUBER J . Multi-column deep neural networks for image classification[J].,2012:1202.2745. arXiv:Computer Vision and Pattern Recognition, 2012:1202.2745. |
[40] | HE K , ZHANG X , REN S ,et al. Delving deep into rectifiers:surpassing human-level performance on image net classification[C]// IEEE International Conference on Computer Vision. IEEE, 2015: 1026-1034. |
[41] | SCHWARTZ E L . Spatial mapping in the primate sensory projection:analytic structure and relevance to perception[J]. Biological Cybernetics, 1977,25(4): 181-194. |
[42] | LAGORCE X , ORCHARD G , GALLUPPI F ,et al. Hots:a hierarchy of event-based time-surfaces for pattern recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(7): 1346-1359. |
[43] | SAEED A , COHEN G K , WANG R M ,et al. The ripple pond:enabling spiking networks to see[J]. Frontiers in Neuroscience, 2013,7:212. |
[44] | RAMESH B , YANG H , ORCHARD G M ,et al. DART:distribution aware retinal transform for event-based cameras[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. |
[45] | GERSTNER W , KISTLER W . Spiking neuron models:an introduction[M]. Oxford: Cambridge University PressPress, 2002. |
[46] | IZHIKEVICH E M , EDELMAN G M . Large-scale model of mammalian thalamocortical systems[J]. Proceedings of the National Academy of Sciences, 2008,105(9): 3593-3598. |
[47] | ANANTHANARAYANAN R , ESSER S K , SIMON H D ,et al. The cat is out of the bag:cortical simulations with 109neurons,1013synapses[C]// Proceedings of the Conference on High Performance Computing Networking,Storage and Analysis. IEEE, 2009. |
[48] | ELIASMITH C , STEWART T C , CHOO X ,et al. A large-scale model of the functioning brain[J]. Science, 2012,338(6111): 1202-1205. |
[49] | SERRANO-GOTARREDONA T , LINARES-BARRANCO B , GALLUPPI F ,et al. ConvNets experiments on SpiNNaker[C]// IEEE International Symposium on Circuits and Systems. IEEE, 2015. |
[50] | FOLOWOSELE F , VOGELSTEIN R J , ETIENNE-CUMMINGS R . Towards a cortical prosthesis:implementing a spike-based HMAX model of visual object recognition in silico[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2011,1(4): 516-525. |
[51] | AMIR A , TABA B , BERG D ,et al. A low power,fully event-based gesture recognition system[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017. |
[52] | EVANGELOS S , DANIEL N , MICHAEL P ,et al. Robustness of spiking deep belief networks to noise and reduced bit precision of neuro-inspired hardware platforms[J]. Frontiers in Neuroscience, 2015,9(00222):222. |
[53] | FARABET C , RAFAEL P , PéREZ-CARRASCO PJ ,et al. Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel ConvNets for visual processing[J]. Frontiers in Neuroscience, 2012,6: 32-32. |
[54] | NGUYEN A , DO T T , CALDWELL D G ,et al. Real-time 6DOF pose relocalization for event cameras with stacked spatial LSTM networks[J].,2017. arXiv:Computer Vision and Pattern Recognition, 2017. |
[55] | PARK P K J , CHO B H , PARK J M ,et al. Performance improvement of deep learning based gesture recognition using spatiotemporal demosaicing technique[C]// IEEE International Conference on Image Processing. IEEE, 2016. |
[56] | ZHU A Z , YUAN L , CHANEY K ,et al. EV-FlowNet:self-supervised optical flow estimation for event-based cameras[J]. arXiv:Computer Vision and Pattern Recognition, 2018. |
[57] | VIDAL A R , REBECQ H , HORSTSCHAEFER T ,et al. Ultimate SLAM? Combining events,images,and IMU for robust visual SLAM in HDR and high-speed scenarios[J]. IEEE Robotics and Automation Letters, 2018,3(2): 994-1001. |
[58] | REBECQ H , HORSTSCH?FER T , GALLEGO G ,et al. EVO:a geometric approach to event-based 6-DOF parallel tracking and mapping in real time[J]. IEEE Robotics and Automation Letters, 2016,2(2): 593-600. |
[59] | NI Z , PACORET C , BENOSMAN R ,et al. Asynchronous event-based high speed vision for microparticle tracking[J]. Journal of Microscopy, 2012,245(3): 236-244. |
[1] | 苏麟, 党小超, 郝占军, 汝春瑞, 尚旭. 基于WPT-MEC的动态自适应卸载方法[J]. 物联网学报, 2022, 6(4): 128-138. |
[2] | 郅佳琳, 滕颖蕾, 张新阳, 牛涛, 宋梅. 基于DNN卷积核分割的边缘协作推理性能分析[J]. 物联网学报, 2022, 6(4): 72-81. |
[3] | 李国权, 徐永海, 林金朝, 黄正文. 基于Res-DNN的端到端MIMO系统信号检测算法[J]. 物联网学报, 2022, 6(1): 65-72. |
[4] | 周毅, 胡姝婷, 李伟, 承楠, 路宁, 沈学民. 图神经网络驱动的交通预测技术:探索与挑战[J]. 物联网学报, 2021, 5(4): 1-16. |
[5] | 王家君, 曹薇, 张贵龙, 张淮智, 凌子兴, 赵小强. 基于多预测模型与非线性组合的神经网络风速预测[J]. 物联网学报, 2021, 5(4): 81-89. |
[6] | 魏泽发,崔华. 基于SqueezeNet卷积神经网络的车辆检测[J]. 物联网学报, 2020, 4(3): 120-125. |
[7] | 金驰,李志军,孙大洋,胡封晔. 基于空间特征的无线体域网人体姿态识别算法[J]. 物联网学报, 2019, 3(3): 70-75. |
[8] | 于天琪,朱咏絮,王现斌. 基于边缘计算的物联网监测系统中利用自编码神经网络实现的异常检测[J]. 物联网学报, 2018, 2(4): 14-21. |
[9] | 张雪,石志国,刘璇. 面向不平衡数据的多层神经网络模型[J]. 物联网学报, 2018, 2(2): 65-72. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|