大脑建模的理论框架及热点问题

doi:10.11959/j.issn.2096-6652.202141

摘要/Abstract

摘要：

理解大脑的工作原理是实现“终极”人工智能的重要途径。建模并验证模型的正确性是理解大脑工作原理的重要手段。首先概述大脑的实验研究方法和建模研究方法；然后给出大脑的框图模型，并以强化学习为核心介绍大脑建模的理论框架；最后给出目前亟待解决的几个热点问题，并讨论大脑建模与相关学科之间的关系。

关键词: 大脑, 模型, 人工智能, 机器学习, 前额皮层, 海马, 杏仁核

Abstract:

Understanding how the brain works is an important way to constructing the final artificial general intelligence.Building models, and testing the correctness of these models play a pivotal role on understanding how the brain works.Firstly, the experimental techniques and mathematical modelling methods for the study of brain were reviewed, and then the block diagram model and theoretical framework were shown, with an emphasis on reinforcement learning.Finally, some highlighted problems in this area, and the relation with closely related disciplines were addressed.

Key words: brain, model, artificial intelligence, machine learning, prefrontal cortex, hippocampus, amygdale

中图分类号:

TP18

胡东伟,冯晓璐. 大脑建模的理论框架及热点问题[J]. 智能科学与技术学报, 2021, 3(4): 412-434.

Dongwei HU,Xiaolu FENG. Theoretical framework of brain modelling and highlighted problems[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 412-434.

图/表 18

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

图16

图17

图18

参考文献 114

[1]	HINTON G E , SALAKHUTDINOV R R . Reducing the dimensionality of data with neural networks[J]. Science, 2006,313(5786): 504-507.
[2]	钟义信 . 机制主义人工智能理论:一种通用的人工智能理论[J]. 智能系统学报, 2018,13(1): 2-18.
	ZHONG Y X . Mechanism-based artificial intelligence theory:a universal theory of artifical intelligence[J]. CAAI Transactions on Intelligent Systems, 2018,13(1): 2-18.
[3]	何华灿 . 泛逻辑学理论:机制主义人工智能理论的逻辑基础[J]. 智能系统学报, 2018,13(1): 19-36.
	HE H C . Universal logic theory:logical foundation of mechanism-based artificial intelligence theory[J]. CAAI Transactions on Intelligent Systems, 2018,13(1): 19-36.
[4]	汪培庄 . 因素空间理论:机制主义人工智能理论的数学基础[J]. 智能系统学报, 2018,13(1): 37-54.
	WANG P Z . Factor space-mathematical basis of mechanism based artificial intelligence theory[J]. CAAI Transactions on Intelligent Systems, 2018,13(1): 37-54.
[5]	汪培庄, 周红军, 何华灿 ,等. 因素表示的信息空间与广义概率逻辑[J]. 智能系统学报, 2019,14(5): 843-852.
	WANG P Z , ZHOU H J , HE H C ,et al. Factorial information space and generalized probability logic[J]. CAAI Transactions on Intelligent Systems, 2019,14(5): 843-852.
[6]	刘经纬, 赵辉, 周瑞 ,等. 规则推理与神经计算智能控制系统改进及比较[J]. 智能系统学报, 2017,12(6): 823-832.
	LIU J W , ZHAO H , ZHOU R ,et al. Improvement and comparison research between intelligent control systems based on rule based reasoning and neural computation AI methods[J]. CAAI Transactions on Intelligent Systems, 2017,12(6): 823-832.
[7]	李德毅 . AI:人类社会发展的加速器[J]. 智能系统学报, 2017,12(5): 583-589.
	LI D Y . Artificial intelligence:an accelerator for the development of human society[J]. CAAI Transactions on Intelligent Systems, 2017,12(5): 583-589.
[8]	曾毅, 刘成林, 谭铁牛 . 类脑智能研究的回顾与展望[J]. 计算机学报, 2016,39(1): 212-222.
	ZENG Y , LIU C L , TAN T N . Retrospect and outlook of brain-inspired intelligence research[J]. Chinese Journal of Computers, 2016,39(1): 212-222.
[9]	FASANO A , DANIELE A , ALBANESE A . Treatment of motor and non-motor features of Parkinson’s disease with deep brain stimulation[J]. The Lancet Neurology, 2012,11(5): 429-442.
[10]	KUO C C J . Understanding convolutional neural networks with a mathematical model[J]. Journal of Visual Communication and Image Representation, 2016,41: 406-413.
[11]	寿天德 . 神经生物学[M]. 3版.北京: 高等教育出版社, 2013.
	SHOU T D . Neurobiology[M]. 3rd edition. Beijing: Higher Education Press, 2013.
[12]	KANDEL E , SCHWARTZ J , JESSELL T ,et al. Principles of neural science[M]. 5th edition.[S.l.:s.n.], 2013.
[13]	DAYAN P , ABBOTT L F . Theoretical neuroscience[M]. Cambridge: The MIT Press, 2001.
[14]	ECKARDT M . The hippocampus as a cognitive map[J]. The Journal of Nervous and Mental Disease, 1980,168(3): 191-192.
[15]	LI Z P . Understanding vision:theory,models,and data[M]. Oxford: Oxford University Press, 2014.
[16]	GEORGIEVA P , SILVA F , MILANOVA M ,et al. EEG signal processing for brain–computer interfaces[M]// Springer handbook of bio-/neuroinformatics. Heidelberg: Springer, 2014.
[17]	LIU Z Q . Causation,Bayesian networks,and cognitive MapsFull-text in English[J]. Acta Automatica Sinica, 2001,27(4): 552-566.
[18]	BARBER D . Machine learning[M]// Bayesian reasoning and machine learning. Cambridge: Cambridge University Press, 2012: 303-304.
[19]	AXELROD R . Structure of decision:the cognitive maps of political elites[M]. Princeton: Princeton University Press, 2016.
[20]	FREDRIC M H , IVICA K . 神经计算原理[M]. 叶世伟,王海娟,译.北京: 机械工业出版社, 2007.
	FREDRIC M H , IVIC K . Principles of neurocomputing for science ＆engingeering[M]. Translated by YE S W,WANG H J. Beijing: China Machine Press, 2007.
[21]	DE KAMPS M , VAN DER VELDE F . From artificial neural networks to spiking neuron populations and back again[J]. Neural Networks, 2001,14(6/7): 941-953.
[22]	FURBER S . I origins[M]// SpiNNaker:a spiking neural network architecture. Boston-Delft: Now Publishers, 2020.
[23]	GERSTNER W , KISTLER W M , NAUD R ,et al. Neuronal dynamics[M]. Cambridge: Cambridge University Press, 2009.
[24]	BRETTE R . Computing with neural synchrony[J]. PLoS Computational Biology, 2012,8(6): e1002561.
[25]	VERSCHURE P F M J , PENNARTZ C M A , PEZZULO G . The why,what,where,when and how of goal-directed choice:neuronal and computational principles[J]. Philosophical Transactions of the Royal Society B:Biological Sciences, 2014,369(1655): 20130483.
[26]	VERSCHURE PAUL F M J . Distributed adaptive control:a theory of the mind,brain,body nexus[J]. Biologically Inspired Cognitive Architectures, 2012: 55-72.
[27]	BOLDT A , BLUNDELL C , DE MARTINO B . Confidence modulates exploration and exploitation in value-based learning[J]. Neuroscience of Consciousness, 2019,2019(1): niz004.
[28]	VEENMAN M V J , VAN HOUT-WOLTERS B H A M , AFFLERBACH P . Metacognition and learning:conceptual and methodological considerations[J]. Metacognition and Learning, 2006,1(1): 3-14.
[29]	COX M T . Metacognition in computation:a selected research review[J]. Artificial Intelligence, 2005,169(2): 104-141.
[30]	MAASS W . On the computational power of winner-take-all[J]. Neural Computation, 2000,12(11): 2519-2535.
[31]	FARZANEH Y , AKBARZADEH A , AKBARI A A . New automated learning CPG for rhythmic patterns[J]. Intelligent Service Robotics, 2012,5(3): 169-177.
[32]	GUERTIN P A . Central pattern generator for locomotion:anatomical,physiological,and pathophysiological considerations[J]. Frontiers in Neurology, 2012,3: 183.
[33]	KATZ P S . Evolution of central pattern generators and rhythmic behaviours[J]. Philosophical Transactions of the Royal Society of London Series B,Biological Sciences, 2016,371(1685): 20150057.
[34]	KARL F . The free-energy principle:a rough guide to the brain?[J]. Trends in Cognitive Sciences, 2009,13(7): 293-301.
[35]	BOGACZ R . A tutorial on the free-energy framework for modelling perception and learning[J]. Journal of Mathematical Psychology, 2017,76: 198-211.
[36]	KINGMA D P , WELLING M . An introduction to variational autoencoders[J]. Foundations and Trends? in Machine Learning, 2019,12(4): 307-392.
[37]	HINTON G . Boltzmann machine[J]. Scholarpedia, 2007,2(5): 1668.
[38]	SALAKHUTDINOV R , HINTON G E . Deep Boltzmann machines[J]. Journal of Machine Learning Research, 2009,5(2): 448-455.
[39]	BACHMAN P , SORDONI A , TRISCHLER A . Learning algorithms for active learning[C]// Proceedings of the 34th International Conference on Machine Learning.[S.l.:s.n.], 2017.
[40]	DOUGLAS R J , MARTIN K A C . Recurrent neuronal circuits in the neocortex[J]. Current Biology:CB, 2007,17(13): 496-500.
[41]	WANG X J . Decision making in recurrent neuronal circuits[J]. Neuron, 2008,60(2): 215-234.
[42]	MORSE A F , DE GREEFF J , BELPEAME T ,et al. Epigenetic robotics architecture (ERA)[J]. IEEE Transactions on Autonomous Mental Development, 2010,2(4): 325-339.
[43]	DAYAN P , DAW N D . Decision theory,reinforcement learning,and the brain[J]. Cognitive,Affective ＆ Behavioral Neuroscience, 2008,8(4): 429-453.
[44]	DABNEY W , KURTH-NELSON Z , UCHIDA N ,et al. A distributional code for value in dopamine-based reinforcement learning[J]. Nature, 2020,577(7792): 671-675.
[45]	MILLER K J , SHENHAV A , LUDVIG E A . Habits without values[J]. Psychological Review, 2019,126(2): 292-311.
[46]	DOLAN R J , DAYAN P . Goals and habits in the brain[J]. Neuron, 2013,80(2): 312-325.
[47]	FRIEDEL E , KOCH S P , WENDT J ,et al. Devaluation and sequential decisions:linking goal-directed and model-based behavior[J]. Fron tiers in Human Neuroscience, 2014,8: 587.
[48]	PEZZULO G , VERSCHURE P F M J , BALKENIUS C ,et al. The principles of goal-directed decision-making:from neural mechanisms to computation and robotics[J]. Philosophical Transactions of the Royal Society B:Biological Sciences, 2014,369(1655): 20130470.
[49]	DAW N D , DAYAN P . The algorithmic anatomy of model-based evaluation[J]. Philosophical Transactions of the Royal Society of London Series B,Biological Sciences, 2014,369(1655): 20130478.
[50]	BUSCHMAN T J , MILLER E K . Goal-direction and top-down control[J]. Philosophical Transactions of the Royal Society B:Biological Sciences, 2014,369(1655): 20130471.
[51]	BOTVINICK M , WEINSTEIN A . Model-based hierarchical reinforcement learning and human action control[J]. Philosophical Transactions of the Royal Society B:Biological Sciences, 2014,369(1655): 20130480.
[52]	BOUREAU Y L , SOKOL-HESSNER P , DAW N D . Deciding how to decide:self-control and meta-decision making[J]. Trends in Cognitive Sciences, 2015,19(11): 700-710.
[53]	GLIMCHER P W , FEHR E . Neuroeconomics:decision making and the brain[M]. 2nd edition.[S.l:s.n.], 2014.
[54]	HA D , SCHMIDHUBER J . World models[J]. arXiv preprint,2018,arXiv:1803.10122.
[55]	FU M C , . Monte Carlo tree search:a tutorial[C]// Proceedings of 2018 Winter Simulation Conference (WSC). Piscataway:IEEE Press, 2018: 222-236.
[56]	SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489.
[57]	SILVER D , SCHRITTWIESER J , SIMONYAN K ,et al. Mastering the game of go without human knowledge[J]. Nature, 2017,550(7676): 354-359.
[58]	MORAN R , KERAMATI M , DAYAN P ,et al. Retrospective model-based inference guides model-free credit assignment[J]. Nature Communications, 2019,10: 750.
[59]	GERSHMAN S J , MARKMAN A B , OTTO A R . Retrospective revaluation in sequential decision making:a tale of two systems[J]. Journal of Experimental Psychology General, 2014,143(1): 182-194.
[60]	KR?MER W . Kahneman,D(2011):thinking,fast and slow[J]. Statistical Papers, 2014,55(3): 915.
[61]	BANSAL S , CALANDRA R , CHUA K ,et al. MBMF:model-based priors for model-free reinforcement learning[J]. arXiv preprint,2017,arXiv:1709.03153.
[62]	DEISENROTH M P , RASMUSSEN C E . PILCO:a model-based and data-efficient approach to policy search[C]// Proceedings of the 28th International Conference on Machine Learning.[S.l.:s.n.], 2011: 465-472.
[63]	SMITTENAAR P , FITZGERALD T H B , ROMEI V ,et al. Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans[J]. Neuron, 2013,80(4): 914-919.
[64]	KERAMATI M , DEZFOULI A , PIRAY P . Speed/accuracy trade-off between the habitual and the goal-directed processes[J]. PLoS Computational Biology, 2011,7(5): e1002055.
[65]	KOECHLIN E , SUMMERFIELD C . An information theoretical approach to prefrontal executive function[J]. Trends in Cognitive Sciences, 2007,11(6): 229-235.
[66]	TYE K M , JANAK P H . Amygdala neurons differentially encode motivation and reinforcement[J]. The Journal of Neuroscience:the Official Journal of the Society for Neuroscience, 2007,27(15): 3937-3945.
[67]	SCH?NFELD L M , ZECH M P , SCH?BLE S , ,et al. Lesions of the rat basolateral amygdala reduce the behavioral response to ultrasonic vocalizations[J]. Behavioural Brain Research, 2020,378: 112274.
[68]	PANKSEPP J . Affective neuroscience:the foundations of human and animal emotions[M]. Oxford: Oxford University Press, 2005.
[69]	SMITH R , ALKOZEI A , KILLGORE W D S . How do emotions work?[J]. Frontiers for Young Minds, 2017,5: 69.
[70]	于冬梅 . 情感计算关键技术研究[D]. 上海:东华大学, 2009.
	YU D M . Research on key technology of affective computing[D]. Shanghai:Donghua University, 2009.
[71]	VALIKHANI M , SOURKOUNIS C . A brain emotional learning-based intelligent controller (BELBIC) for DFIG system[C]// Proceedings of 2014 International Symposium on Power Electronics,Electrical Drives,Automation and Motion. Piscataway:IEEE Press, 2014: 713-718.
[72]	MCDUFF D , KAPOOR A . Visceral machines:risk-aversion in reinforcement learning with intrinsic physiological rewards[C]// Proceedings of 2019 Conference Blind Submission.[S.l.:s.n.], 2019.
[73]	NAVAJAS J , HINDOCHA C , FODA H ,et al. The idiosyncratic nature of confidence[J]. Nature Human Behaviour, 2017,1(11): 810-818.
[74]	BISHOP S J , GAGNE C . Anxiety,depression,and decision making:a computational perspective[J]. Annual Review of Neuroscience, 2018,41: 371-388.
[75]	LERNER J S , LI Y , VALDESOLO P ,et al. Emotion and decision making[J]. Annual Review of Psychology, 2015,66(1): 799-823.
[76]	OTTER D W , MEDINA J R , KALITA J K . A survey of the usages of deep learning for natural language processing[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021,32(2): 604-624.
[77]	朱张莉, 饶元, 吴渊 ,等. 注意力机制在深度学习中的研究进展[J]. 中文信息学报, 2019,33(6): 1-11.
	ZHU Z L , RAO Y , WU Y ,et al. Research progress of attention mechanism in deep learning[J]. Journal of Chinese Information Processing, 2019,33(6): 1-11.
[78]	ANTUNES A , LAFLAQUIERE A , OGATA T ,et al. A Bi-directional multiple timescales LSTM model for grounding of actions and verbs[C]// Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway:IEEE Press, 2019: 2614-2621.
[79]	?TEPáNOVá K , KLEIN F B , CANGELOSI A ,et al. Mapping language to vision in a real-world robotic scenario[J]. IEEE Transactions on Cognitive and Developmental Systems, 2018,10(3): 784-794.
[80]	FELDMAN H , FRISTON K J . Attention,uncertainty,and free-energy[J]. Frontiers in Human Neuroscience, 2010,4: 215.
[81]	STACHENFELD K L , BOTVINICK M M , GERSHMAN S J . The hippocampus as a predictive map[J]. Nature Neuroscience, 2017,20(11): 1643-1653.
[82]	LAND M F . Do we have an internal model of the outside world?[J]. Philosophical Transactions of the Royal Society of London Series B,Biological Sciences, 2014,369(1636): 20130045.
[83]	WINTER S S , TAUBE J S . Head direction cells:from generation to integration[M]// Space,time and memory in the hippocampal formation. Vienna: Springer Vienna, 2014: 83-106.
[84]	ROWLAND D C , ROUDI Y , MOSER M B ,et al. Ten years of grid cells[J]. Annual Review of Neuroscience, 2016,39: 19-40.
[85]	BARRY C , LEVER C , HAYMAN R ,et al. The boundary vector cell model of place cell firing and spatial memory[J]. Reviews in the Neurosciences, 2006,17(1/2): 71-97.
[86]	KROPFF E , CARMICHAEL J E , MOSER M B ,et al. Speed cells in the medial entorhinal cortex[J]. Nature, 2015,523(7561): 419-424.
[87]	CULLEN K E , TAUBE J S . Our sense of direction:progress,controversies and challenges[J]. Nature Neuroscience, 2017,20(11): 1465-1473.
[88]	PARK E H , KEELEY S , SAVIN C ,et al. How the internally organized direction sense is used to navigate[J]. Neuron, 2019,101(2): 285-293.
[89]	BEHRENS T E J , MULLER T H , WHITTINGTON J C R ,et al. What is a cognitive map? Organizing knowledge for flexible behavior[J]. Neuron, 2018,100(2): 490-509.
[90]	KOBER J , BAGNELL J A , PETERS J . Reinforcement learning in robotics:a survey[J]. The International Journal of Robotics Research, 2013,32(11): 1238-1274.
[91]	DU Z X , ZHOU C , DING M ,et al. Cognitive knowledge graph reasoning for one-shot relational learning[J]. arXiv preprint,2019,arXiv:1906.05489.
[92]	RITTER S , WANG J X , KURTH-NELSON Z ,et al. Been there,done that:meta-learning with episodic recall[C]// Proceedings of the 35th International Conference on Machine Learning.[S.l.:s.n.], 2018.
[93]	DURRANT-WHYTE H , BAILEY T . Simultaneous localization and mapping:part I[J]. IEEE Robotics ＆ Automation Magazine, 2006,13(2): 99-110.
[94]	BAILEY T , DURRANT-WHYTE H , . Simultaneous localization and mapping (SLAM):part II[J]. IEEE Robotics ＆ Automation Magazine, 2006,13(3): 108-117.
[95]	BOWLING M , BURCH N , JOHANSON M ,et al. Heads-up limit hold’em poker is solved[J]. Science, 2015,347(6218): 145-149.
[96]	FAULKNER R , PRECUP D . Dyna planning using a feature based generative model[J]. arXiv preprint,2018,arXiv:1805.10129.
[97]	SOLWAY A , BOTVINICK M M . Goal-directed decision making as probabilistic inference:a computational framework and potential neural correlates[J]. Psychological Review, 2012,119(1): 120-154.
[98]	BOTVINICK M M , NIV Y , BARTO A G . Hierarchically organized behavior and its neural foundations:a reinforcement learning perspective[J]. Cognition, 2009,113(3): 262-280.
[99]	PONG V , GU S X , DALAL M ,et al. Temporal difference models:model-free deep RL for model-based control[J]. arXiv preprint,2018:arXiv:1802.09081.
[100]	MAISTO D , DONNARUMMA F , PEZZULO G . Divide et impera:subgoaling reduces the complexity of probabilistic inference and problem solving[J]. Journal of the Royal Society Interface, 2015,12(104): 20141335.
[101]	BERKMAN E T . The neuroscience of goals and behavior change[J]. Consulting Psychology Journal:Practice and Research, 2018,70(1): 28-44.
[102]	KOOL W , BOTVINICK M . Mental labour[J]. Nature Human Behaviour, 2018,2(12): 899-908.
[103]	FLEMING S M , DAW N D . Self-evaluation of decision-making:a general Bayesian framework for metacognitive computation[J]. Psychological Review, 2017,124(1): 91-114.
[104]	NAVAJAS J , BAHRAMI B , LATHAM P E . Post-decisional accounts of biases in confidence[J]. Current Opinion in Behavioral Sciences, 2016,11: 55-60.
[105]	ATIYA N A A , RA?ó I , PRASAD G ,et al. A neural circuit model of decision uncertainty and change-of-mind[J]. Nature Communications, 2019,10: 2287.
[106]	MURPHY R R . Introduction to AI robotics[J]. Industrial Robot:an International Journal, 2001,28(3): 266-267.
[107]	DE SILVA L , EKANAYAKE H . Behavior-based robotics and the reactive paradigm a survey[C]// Proceedings of 2008 11th International Conference on Computer and Information Technology. Piscataway:IEEE Press, 2008: 36-43.
[108]	HASSABIS D , KUMARAN D , SUMMERFIELD C ,et al. Neuroscience-inspired artificial intelligence[J]. Neuron, 2017,95(2): 245-258.
[109]	KRALIK J D , MAO T , CHENG Z ,et al. Modeling incubation and restructuring for creative problem solving in robots[J]. Robotics and Autonomous Systems, 2016,86: 162-173.
[110]	WINFIELD A F T . Experiments in artificial theory of mind:from safety to story-telling[J]. Frontiers in Robotics and AI, 2018,5: 75.
[111]	GROSS R . Psychology:the science of mind and behavior (6E)[M]. UK: Hodder Education, 2010.
[112]	SHAMS L , BEIERHOLM U R . Causal inference in perception[J]. Trends in Cognitive Sciences, 2010,14(9): 425-432.
[113]	MICHEL M , BECK D , BLOCK N ,et al. Opportunities and challenges for a maturing science of consciousness[J]. Nature Human Behaviour, 2019,3(2): 104-107.
[114]	DAYAN P , DURSTEWITZ D . Editorial overview:computational modeling:after the middle ages of computational neuroscience[J]. Current Opinion in Behavioral Sciences, 2016,11(9): v-vi.