基于AE和Transformer的运动想象脑电信号分类研究

doi:10.11959/j.issn.2096-3750.2023.00310

Abstract

Abstract:

The motor imagery brain-computer interface has always been the focus of scholars.But traditional system cannot accurately extract significant signals and has low classification accuracy.To overcome such difficulty, a new Transformer model was proposed based on the auto-encoder (AE).The filter bank common spatial pattern (FBCSP) was used to extract the features of multiple frequency bands, and the AE was exploited to obtain the dimensionality-reduced feature matrix.Finally, it considered the influence of the global signal features by the position encoding of the Transformer model and considered the internal correlation of the feature matrix by using the multi-head self-attention mechanism.By comparison with the traditional K-nearest neighbors (KNN) system based on linear discriminant analysis (LDA), the experimental results validates that the classification effect of AE+Transformer model is better than that of LDA+KNN system.It shows that the improved algorithm is suitable for the binary classification of motor imagery.

Key words: motor imagery, deep learning, auto-encoder, attention module, Transformer model

CLC Number:

TN911.73

Rui JIANG, Liuting SUN, Xiaoming WANG, Dapeng LI, Youyun XU. Research on EEG signal classification of motor imagery based on AE and Transformer[J]. Chinese Journal on Internet of Things, 2023, 7(1): 118-128.

Figures/Tables 12

References 30

[1]	WANG S F , LEE Y H , SHIAH Y J ,et al. Time-frequency analysis of EEGs recorded during meditation[C]// Proceedings of 2011 1st International Conference on Robot,Vision and Signal Processing. Piscataway:IEEE Press, 2011,11(10): 73-76.
[2]	RUSNAC A L , GRIGORE O . EEG preprocessing methods for BCI imagined speech signals[C]// Proceedings of 2021 International Conference on e-Health and Bioengineering (EHB). Piscataway:IEEE Press, 2021: 1-4.
[3]	MüHL C , CHANEL G , ALLISON B ,et al. Third workshop on affective brain-computer interfaces (ABCI 2013):introduction[C]// Proceedings of 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Piscataway:IEEE Press, 2013:821.
[4]	WEN D , LIANG B B , ZHOU Y H ,et al. The current research of combining multi-modal brain-computer interfaces with virtual reality[J]. IEEE Journal of Biomedical and Health Informatics, 2021,25(9): 3278-3287.
[5]	KAUATI-SAITOé SILVEIRA G F M , DA-SILVA P J G ,et al. Classification of motor tasks from EEG signals comparing preprocessing techniques[C]// XXVI Brazilian Congress on Biomedical Engineering, 2019: 109-113.
[6]	WANG K , ZHAI D H , XIA Y Q . Motor imagination EEG recognition algorithm based on DWT,CSP and extreme learning machine[C]// Proceedings of 2019 Chinese Control Conference (CCC). Piscataway:IEEE Press, 2019: 4590-4595.
[7]	王行愚, 金晶, 张宇 ,等. 脑控:基于脑-机接口的人机融合控制[J]. 自动化学报, 2013,39(3): 208-221.
	WANG X Y , JIN J , ZHANG Y ,et al. Brain control:human-computer integration control based on brain-computer interface[J]. Acta Automatica Sinica, 2013,39(3): 208-221.
[8]	SHI T W , REN L , CUI W H . Feature extraction of brain–computer interface electroencephalogram based on motor imagery[J]. IEEE Sensors Journal, 2020,20(20): 11787-11794.
[9]	FEDOSOV N , LEVADNIY I , DMITRIEV A ,et al. Independent component analysis for different movements detection in BCI application based on sensorimotor rhythms[C]// Proceedings of 2020 Ural Symposium on Biomedical Engineering,Radio electronics and Information Technology (USBEREIT). Piscataway:IEEE Press, 2020: 69-72.
[10]	SELIM S , TANTAWI M M , SHEDEED H A ,et al. A CSP\AM-BA-SVM approach for motor imagery BCI system[J]. IEEE Access, 2018,6(10): 49192-49208.
[11]	JIA X W , LI K , LI X Y ,et al. A novel semi-supervised deep learning framework for affective state recognition on EEG signals[C]// Procee dings of 2014 IEEE International Conference on Bioinformatics and Bioengineering. Piscataway:IEEE Press, 2014: 30-37.
[12]	LI X Y , JIA X W , XUN G X ,et al. Improving EEG feature learning via synchronized facial video[C]// Proceedings of 2015 IEEE International Conference on Big Data(Big Data). Piscataway:IEEE Press, 2015: 843-848.
[13]	孙嘉瑶 . 深度学习注意力机制在脑电信号分类中的应用[D]. 深圳:中国科学院大学(中国科学院深圳先进技术研究院), 2021.
	SUN J Y . Application of deep learning attention mechanism in EEG classification[D]. Shenzhen:Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences, 2021.
[14]	TAO Y Z , SUN T , MUHAMED A ,et al. Gated transformer for decoding human brain EEG signals[J]. Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual International Conference, 2021: 125-130.
[15]	李畅, 蔡国超, 黄晓阳 ,等. 基于Transformer引导卷积神经网络的脑电信号分类方法:CN114564991A[P]. 2022-05-31.
	LI C , CAI G C , HUANG X Y ,et al. EEG (electroencephalogram) signal classification method based on Transform guided convolutional neural network:CN114564991A[P]. 2022-05-31.
[16]	LU N , LI T F , REN X D ,et al. A deep learning scheme for motor imagery classification based on restricted boltzmann machines[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering:a Publication of the IEEE Engineering in Medicine and Biology Society, 2017,25(6): 566-576.
[17]	ZHANG R , LI X P , WANG Y W ,et al. Using brain network features to increase the classification accuracy of MI-BCI inefficiency subject[J]. IEEE Access, 2019,7: 74490-74499.
[18]	DAS R , LOPEZ P S , AHMED KHAN M ,et al. FBCSP and adaptive boosting for multiclass motor imagery BCI data classification:a machine learning approach[C]// Proceedings of 2020 IEEE International Conference on Systems,Man,and Cybernetics. Piscataway:IEEE Press, 2020: 1275-1279.
[19]	CHRISTENSEN S M , HOLM N S , PUTHUSSERYPADY S . An improved five class MI based BCI scheme for drone control using filter bank CSP[C]// Proceedings of 2019 7th International Winter Conference on Brain-Computer Interface (BCI). Piscataway:IEEE Press, 2019: 1-6.
[20]	HUNG J W , LIN J R , ZHUANG L Y . The evaluation study of the deep learning model transformer in speech translation[C]// Proceedings of 2021 7th International Conference on Applied System Innovation (ICASI). Piscataway:IEEE Press, 2021: 30-33.
[21]	SALTZ P , LIN S Y , CHENG S C ,et al. Dementia detection using transformer-based deep learning and natural language processing models[C]// Proceedings of 2021 IEEE 9th International Conference on Healthcare Informatics. Piscataway:IEEE Press, 2021: 509-510.
[22]	徐嘉杰, 卢兆军, 袁飞 ,等. 基于深度神经网络和 SoftMax 分类器的台区负荷分类识别方法[J]. 电气自动化, 2021,43(6): 102-104,114.
	XU J J , LU Z J , YUAN F ,et al. Classification and identification method of station load based on deep neural network and SoftMax classifier[J]. Electrical Automation, 2021,43(6): 102-104,114.
[23]	邓建国 . 有监督学习中的损失函数及图像标注应用[D]. 太原:太原科技大学, 2020.
	DENG J G . Loss function in supervised earning and its application in image annotation[D]. Taiyuan:Taiyuan University of Science and Technology, 2020.
[24]	ZHANG Z J , . Improved Adam optimizer for deep neural networks[C]// Proceedings of 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS). Piscataway:IEEE Press, 2018: 1-2.
[25]	CHENG L W , LI D L , YU G J ,et al. A motor imagery EEG feature extraction method based on energy principal component analysis and deep belief networks[J]. IEEE Access, 2020,8(8): 21453-21472.
[26]	HOSSEINI M P , POMPILI D , ELISEVICH K ,et al. Optimized deep learning for EEG big data and seizure prediction BCI via Internet of Things[J]. IEEE Transactions on Big Data, 2017,3(4): 392-404.
[27]	SHAHLAEI F , BAGH N , SHALIGRAM A D ,et al. Classification of motor imagery tasks using inter trial variance in the brain computer interface[C]// Proceedings of 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA). Piscataway:IEEE Press, 2018: 1-6.
[28]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[EB]. 2017.
[29]	李嘉莹, 赵丽, 边琰 ,等. 基于LDA和KNN的下肢运动想象脑电信号分类研究[J]. 国外电子测量技术, 2021,40(1): 9-14.
	LI J Y , ZHAO L , BIAN Y ,et al. Classification of lower limb motor imagination signals based on LDA and KNN[J]. Foreign Electronic Measurement Technology, 2021,40(1): 9-14.
[30]	兰成辉, 李江天, 李敏 ,等. 基于皮电和肌电的驾驶疲劳判别阈值研究[J]. 科技风, 2021,1(5): 171-172.
	LAN C H , LI J T , LI M ,et al. Research on driving fatigue discrimination threshold based on skin and emg[J]. Technology Wind, 2021,1(5): 171-172.

Metrics

Recommended 0

No Suggested Reading articles found!

真实情况	预测结果		含义
真实情况	正例	反例	含义
正例	TP（将正例正确预测为正例）	FN（将正例错误预测为负例）	TP+FN 表示实际数据集中正样本的数量
反例	FP（将负例错误预测为正例）	TN（将负例正确预测为负例）	FP+TN 表示实际数据集中负样本的数量
含义	TP+FP表示预测的正类样本数	FN+TN表示预测的负类样本数	TP+FN+FP+TN 表示样本总数

	ACC	Kappa	F1-score	运行时间/s
CNN+Transformer	88.57%	0.771 4	0.887 3	32.110 2
Transformer	89.29%	0.785 7	0.888 9	21.244 1

Research on EEG signal classification of motor imagery based on AE and Transformer

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 30

Related Articles 9

Metrics

Recommended 0

对比项	TP	FN	FP	TN
KNN	59	11	7	63
LDA+KNN	59	11	5	65
Transformer	60	10	5	65
AE+Transformer	60	10	2	68

[1]	Zhifei ZHANG, Feng LIU, Yiyang GE, Shuo LI, Yu ZHANG, Ke XIONG. An intrusion detection method based on depthwise separable convolution and attention mechanism [J]. Chinese Journal on Internet of Things, 2023, 7(1): 49-59.
[2]	Biao ZHANG, Ximing WANG, Yifan XU, Wen LI, Hao HAN, Songyi LIU, Xueqiang CHEN. Multi-domain collaborative anti-jamming based on multi-agent deep reinforcement learning [J]. Chinese Journal on Internet of Things, 2022, 6(4): 104-116.
[3]	Dan LUO, Ruzhi XU, Zhitao GUAN. Differential privacy budget optimization based on deep learning in IoT [J]. Chinese Journal on Internet of Things, 2022, 6(2): 65-76.
[4]	Xuanzhe XU, Ke NING, Xuemin ZHENG, Mingxin ZHAO, Mengmeng XU, Nanjian WU, Liyuan LIU. Verification of an artificial intelligence vision chip design for edge computing based on hardware simulation system [J]. Chinese Journal on Internet of Things, 2022, 6(1): 20-28.
[5]	Guoquan LI, Yonghai XU, Jinzhao LIN, Zhengwen HUANG. Res-DNN based signal detection algorithm for end-to-end MIMO systems [J]. Chinese Journal on Internet of Things, 2022, 6(1): 65-72.
[6]	Ling TAN, Shanshan RONG, Jingming XIA, Sarker SAJIB, Wenjie MA. Real-time diagnosis of multi-category skin diseases based on IR-VGG [J]. Chinese Journal on Internet of Things, 2021, 5(3): 115-125.
[7]	Chunmin LIN, Liekang ZENG, Xu CHEN. Research on power efficient autonomous UAV navigation algorithm: an edge intelligence driven approach [J]. Chinese Journal on Internet of Things, 2021, 5(2): 87-96.
[8]	Muhan CHEN,Jiajia GUO,Xiao LI,Shi JIN. An overview of the CSI feedback based on deep learning for massive MIMO systems [J]. Chinese Journal on Internet of Things, 2020, 4(1): 33-44.
[9]	Yong LIAO,Haimei YAO,Yuanxiao HUA. Channel state information acquisition algorithm based on deep learning for IoT [J]. Chinese Journal on Internet of Things, 2019, 3(1): 8-13.