Chinese Journal of Intelligent Science and Technology ›› 2021, Vol. 3 ›› Issue (1): 65-75.doi: 10.11959/j.issn.2096-6652.202107

• Special topic:emotional brain computer interface • Previous Articles     Next Articles

Emotion recognition based on brain and machine collaborative intelligence

Dongjun LIU1,2, Yuhan WANG1,2, Wenfen LING1,2, Yong PENG1,2, Wanzeng KONG1,2   

  1. 1 College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China
    2 Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
  • Revised:2021-02-08 Online:2021-03-15 Published:2021-03-01
  • Supported by:
    The National Key Research and Development Program of China(2017YFE0116800);The National Natural Science Foundation of China(U20B2074);Science and Technology Program of Zhejiang Province(2018C04012);Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province(20200E10010)


Emotion recognition is a direct and effective mode of emotion recognition.Machine learning relies on the formal representation of image expressions, lacks the cognitive representation ability of the brain, and has poor recognition performance on small sample data sets or complex expression (camouflage) data sets.To this end, the formal representation of machine artificial intelligence was combined with the emotional cognitive ability of human brain general intelligence, and a brain-machine collaborative intelligence emotion recognition method was proposed.Firstly, electroencephalogram (EEG) emotional features were extracted from EEG to obtain the brain’s cognitive representation of emotions.Secondly, the visual features of the image were extracted from the emotional image to obtain the machine’s formal representation of the emotion.In order to enhance the generalization ability of the machine model, the transfer adaptation between samples was introduced in the feature learning.After obtaining image visual features and EEG emotional features, the random forest regression model was trained to obtain the brain-machine mapping relationship between image visual features and EEG emotional features.The visual features of the test image were generated through the brain-machine mapping relationship to generate virtual EEG emotional features, and then the virtual EEG emotional features and image visual features were fused for emotion recognition.This method has been verified on the Chinese facial affective picture system (CFAPS) and found that the average recognition accuracy of the seven emotions is 88.51%, which is 3%~5% higher than the image-based method.

Key words: emotion recognition, EEG signal, brain-machine collaborative intelligence, deep learning

CLC Number: 

No Suggested Reading articles found!