基于AE和Transformer的运动想象脑电信号分类研究

doi:10.11959/j.issn.2096-3750.2023.00310

物联网学报 ›› 2023, Vol. 7 ›› Issue (1): 118-128.doi: 10.11959/j.issn.2096-3750.2023.00310

基于AE和Transformer的运动想象脑电信号分类研究

蒋锐¹, 孙刘婷¹, 王小明¹, 李大鹏¹, 徐友云¹^,²

¹ 南京邮电大学通信与信息工程学院，江苏南京 210003
² 南京邮电大学通信与网络技术国家工程研究中心，江苏南京 210003

修回日期:2022-11-06 出版日期:2023-03-30 发布日期:2023-03-01
作者简介:蒋锐（1985− ），男，博士，南京邮电大学副教授，主要研究方向为雷达信号处理、移动通信系统、脑波信号处理等
孙刘婷（1997− ），女，南京邮电大学通信与信息工程学院硕士生，主要研究方向为脑波信号处理等
王小明（1986− ），男，博士，南京邮电大学副教授，主要研究方向为通信资源协调分配、深度学习等
李大鹏（1982− ），男，博士，南京邮电大学教授，主要研究方向为移动通信系统等
徐友云（1966− ），男，博士，南京邮电大学教授，主要研究方向为移动通信系统、5G通信等
基金资助:
国家自然科学基金资助项目(61971241);国家自然科学基金资助项目(62071245)

Research on EEG signal classification of motor imagery based on AE and Transformer

Rui JIANG¹, Liuting SUN¹, Xiaoming WANG¹, Dapeng LI¹, Youyun XU¹^,²

¹ School of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
² National Engineering Research Center for Communication and Network Technology, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

Revised:2022-11-06 Online:2023-03-30 Published:2023-03-01
Supported by:
The National Natural Science Foundation of China(61971241);The National Natural Science Foundation of China(62071245)

摘要/Abstract

摘要：

基于运动想象的脑机接口系统一直是海内外研究学者的关注对象。针对传统运动想象脑电识别系统不能精准提取显著特征、分类识别准确率低等问题，提出一种新的基于自编码器（AE, auto-encoder）降维的 Transformer分类识别模型。该方法使用滤波器组共空间模式（FBCSP, filter bank common spatial pattern）对数据进行多个频段的特征提取，并利用AE获得降维后的特征矩阵。同时借助Transformer模型的位置编码考虑全局信号特征影响并利用多头自注意力机制考虑特征矩阵的内部关联性，提升系统分类识别效果。与传统的基于线性判别分析（LDA, linear discriminant analysis）的K-近邻（KNN, K-nearest neighbors）法分类识别系统进行对比，实验表明AE+Transformer模型的分类识别效果优于LDA+KNN系统，说明这种改进后的算法适用于运动想象的二分类。

关键词: 运动想象, 深度学习, 自编码器, 注意力模块, Transformer模型

Abstract:

The motor imagery brain-computer interface has always been the focus of scholars.But traditional system cannot accurately extract significant signals and has low classification accuracy.To overcome such difficulty, a new Transformer model was proposed based on the auto-encoder (AE).The filter bank common spatial pattern (FBCSP) was used to extract the features of multiple frequency bands, and the AE was exploited to obtain the dimensionality-reduced feature matrix.Finally, it considered the influence of the global signal features by the position encoding of the Transformer model and considered the internal correlation of the feature matrix by using the multi-head self-attention mechanism.By comparison with the traditional K-nearest neighbors (KNN) system based on linear discriminant analysis (LDA), the experimental results validates that the classification effect of AE+Transformer model is better than that of LDA+KNN system.It shows that the improved algorithm is suitable for the binary classification of motor imagery.

Key words: motor imagery, deep learning, auto-encoder, attention module, Transformer model

中图分类号:

TN911.73

蒋锐, 孙刘婷, 王小明, 李大鹏, 徐友云. 基于AE和Transformer的运动想象脑电信号分类研究[J]. 物联网学报, 2023, 7(1): 118-128.

Rui JIANG, Liuting SUN, Xiaoming WANG, Dapeng LI, Youyun XU. Research on EEG signal classification of motor imagery based on AE and Transformer[J]. Chinese Journal on Internet of Things, 2023, 7(1): 118-128.

图/表 12

图1

图2

图3

图4

图5

表1

表2

图6

图7

图8

表3

图9

参考文献 30

[1]	WANG S F , LEE Y H , SHIAH Y J ,et al. Time-frequency analysis of EEGs recorded during meditation[C]// Proceedings of 2011 1st International Conference on Robot,Vision and Signal Processing. Piscataway:IEEE Press, 2011,11(10): 73-76.
[2]	RUSNAC A L , GRIGORE O . EEG preprocessing methods for BCI imagined speech signals[C]// Proceedings of 2021 International Conference on e-Health and Bioengineering (EHB). Piscataway:IEEE Press, 2021: 1-4.
[3]	MüHL C , CHANEL G , ALLISON B ,et al. Third workshop on affective brain-computer interfaces (ABCI 2013):introduction[C]// Proceedings of 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. Piscataway:IEEE Press, 2013:821.
[4]	WEN D , LIANG B B , ZHOU Y H ,et al. The current research of combining multi-modal brain-computer interfaces with virtual reality[J]. IEEE Journal of Biomedical and Health Informatics, 2021,25(9): 3278-3287.
[5]	KAUATI-SAITOé SILVEIRA G F M , DA-SILVA P J G ,et al. Classification of motor tasks from EEG signals comparing preprocessing techniques[C]// XXVI Brazilian Congress on Biomedical Engineering, 2019: 109-113.
[6]	WANG K , ZHAI D H , XIA Y Q . Motor imagination EEG recognition algorithm based on DWT,CSP and extreme learning machine[C]// Proceedings of 2019 Chinese Control Conference (CCC). Piscataway:IEEE Press, 2019: 4590-4595.
[7]	王行愚, 金晶, 张宇 ,等. 脑控:基于脑-机接口的人机融合控制[J]. 自动化学报, 2013,39(3): 208-221.
	WANG X Y , JIN J , ZHANG Y ,et al. Brain control:human-computer integration control based on brain-computer interface[J]. Acta Automatica Sinica, 2013,39(3): 208-221.
[8]	SHI T W , REN L , CUI W H . Feature extraction of brain–computer interface electroencephalogram based on motor imagery[J]. IEEE Sensors Journal, 2020,20(20): 11787-11794.
[9]	FEDOSOV N , LEVADNIY I , DMITRIEV A ,et al. Independent component analysis for different movements detection in BCI application based on sensorimotor rhythms[C]// Proceedings of 2020 Ural Symposium on Biomedical Engineering,Radio electronics and Information Technology (USBEREIT). Piscataway:IEEE Press, 2020: 69-72.
[10]	SELIM S , TANTAWI M M , SHEDEED H A ,et al. A CSP\AM-BA-SVM approach for motor imagery BCI system[J]. IEEE Access, 2018,6(10): 49192-49208.
[11]	JIA X W , LI K , LI X Y ,et al. A novel semi-supervised deep learning framework for affective state recognition on EEG signals[C]// Procee dings of 2014 IEEE International Conference on Bioinformatics and Bioengineering. Piscataway:IEEE Press, 2014: 30-37.
[12]	LI X Y , JIA X W , XUN G X ,et al. Improving EEG feature learning via synchronized facial video[C]// Proceedings of 2015 IEEE International Conference on Big Data(Big Data). Piscataway:IEEE Press, 2015: 843-848.
[13]	孙嘉瑶 . 深度学习注意力机制在脑电信号分类中的应用[D]. 深圳:中国科学院大学(中国科学院深圳先进技术研究院), 2021.
	SUN J Y . Application of deep learning attention mechanism in EEG classification[D]. Shenzhen:Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences, 2021.
[14]	TAO Y Z , SUN T , MUHAMED A ,et al. Gated transformer for decoding human brain EEG signals[J]. Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual International Conference, 2021: 125-130.
[15]	李畅, 蔡国超, 黄晓阳 ,等. 基于Transformer引导卷积神经网络的脑电信号分类方法:CN114564991A[P]. 2022-05-31.
	LI C , CAI G C , HUANG X Y ,et al. EEG (electroencephalogram) signal classification method based on Transform guided convolutional neural network:CN114564991A[P]. 2022-05-31.
[16]	LU N , LI T F , REN X D ,et al. A deep learning scheme for motor imagery classification based on restricted boltzmann machines[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering:a Publication of the IEEE Engineering in Medicine and Biology Society, 2017,25(6): 566-576.
[17]	ZHANG R , LI X P , WANG Y W ,et al. Using brain network features to increase the classification accuracy of MI-BCI inefficiency subject[J]. IEEE Access, 2019,7: 74490-74499.
[18]	DAS R , LOPEZ P S , AHMED KHAN M ,et al. FBCSP and adaptive boosting for multiclass motor imagery BCI data classification:a machine learning approach[C]// Proceedings of 2020 IEEE International Conference on Systems,Man,and Cybernetics. Piscataway:IEEE Press, 2020: 1275-1279.
[19]	CHRISTENSEN S M , HOLM N S , PUTHUSSERYPADY S . An improved five class MI based BCI scheme for drone control using filter bank CSP[C]// Proceedings of 2019 7th International Winter Conference on Brain-Computer Interface (BCI). Piscataway:IEEE Press, 2019: 1-6.
[20]	HUNG J W , LIN J R , ZHUANG L Y . The evaluation study of the deep learning model transformer in speech translation[C]// Proceedings of 2021 7th International Conference on Applied System Innovation (ICASI). Piscataway:IEEE Press, 2021: 30-33.
[21]	SALTZ P , LIN S Y , CHENG S C ,et al. Dementia detection using transformer-based deep learning and natural language processing models[C]// Proceedings of 2021 IEEE 9th International Conference on Healthcare Informatics. Piscataway:IEEE Press, 2021: 509-510.
[22]	徐嘉杰, 卢兆军, 袁飞 ,等. 基于深度神经网络和 SoftMax 分类器的台区负荷分类识别方法[J]. 电气自动化, 2021,43(6): 102-104,114.
	XU J J , LU Z J , YUAN F ,et al. Classification and identification method of station load based on deep neural network and SoftMax classifier[J]. Electrical Automation, 2021,43(6): 102-104,114.
[23]	邓建国 . 有监督学习中的损失函数及图像标注应用[D]. 太原:太原科技大学, 2020.
	DENG J G . Loss function in supervised earning and its application in image annotation[D]. Taiyuan:Taiyuan University of Science and Technology, 2020.
[24]	ZHANG Z J , . Improved Adam optimizer for deep neural networks[C]// Proceedings of 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS). Piscataway:IEEE Press, 2018: 1-2.
[25]	CHENG L W , LI D L , YU G J ,et al. A motor imagery EEG feature extraction method based on energy principal component analysis and deep belief networks[J]. IEEE Access, 2020,8(8): 21453-21472.
[26]	HOSSEINI M P , POMPILI D , ELISEVICH K ,et al. Optimized deep learning for EEG big data and seizure prediction BCI via Internet of Things[J]. IEEE Transactions on Big Data, 2017,3(4): 392-404.
[27]	SHAHLAEI F , BAGH N , SHALIGRAM A D ,et al. Classification of motor imagery tasks using inter trial variance in the brain computer interface[C]// Proceedings of 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA). Piscataway:IEEE Press, 2018: 1-6.
[28]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[EB]. 2017.
[29]	李嘉莹, 赵丽, 边琰 ,等. 基于LDA和KNN的下肢运动想象脑电信号分类研究[J]. 国外电子测量技术, 2021,40(1): 9-14.
	LI J Y , ZHAO L , BIAN Y ,et al. Classification of lower limb motor imagination signals based on LDA and KNN[J]. Foreign Electronic Measurement Technology, 2021,40(1): 9-14.
[30]	兰成辉, 李江天, 李敏 ,等. 基于皮电和肌电的驾驶疲劳判别阈值研究[J]. 科技风, 2021,1(5): 171-172.
	LAN C H , LI J T , LI M ,et al. Research on driving fatigue discrimination threshold based on skin and emg[J]. Technology Wind, 2021,1(5): 171-172.

真实情况	预测结果		含义
真实情况	正例	反例	含义
正例	TP（将正例正确预测为正例）	FN（将正例错误预测为负例）	TP+FN 表示实际数据集中正样本的数量
反例	FP（将负例错误预测为正例）	TN（将负例正确预测为负例）	FP+TN 表示实际数据集中负样本的数量
含义	TP+FP表示预测的正类样本数	FN+TN表示预测的负类样本数	TP+FN+FP+TN 表示样本总数

	ACC	Kappa	F1-score	运行时间/s
CNN+Transformer	88.57%	0.771 4	0.887 3	32.110 2
Transformer	89.29%	0.785 7	0.888 9	21.244 1

基于AE和Transformer的运动想象脑电信号分类研究

Research on EEG signal classification of motor imagery based on AE and Transformer

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 30

相关文章 10

Metrics

推荐阅读 0

对比项	TP	FN	FP	TN
KNN	59	11	7	63
LDA+KNN	59	11	5	65
Transformer	60	10	5	65
AE+Transformer	60	10	2	68

[1]	张志飞, 刘峰, 葛祎阳, 李烁, 张煜, 熊轲. 一种基于深度可分离卷积和注意力机制的入侵检测方法[J]. 物联网学报, 2023, 7(1): 49-59.
[2]	张彪, 汪西明, 徐逸凡, 李文, 韩昊, 刘松仪, 陈学强. 基于多智能体深度强化学习的多域协同抗干扰方法研究[J]. 物联网学报, 2022, 6(4): 104-116.
[3]	李贤, 毕宿志, 曾泓儒, 林彬, 林晓辉. 基于智能化用户协作的边缘计算任务卸载与资源分配优化[J]. 物联网学报, 2022, 6(4): 41-52.
[4]	罗丹, 徐茹枝, 关志涛. 物联网环境中基于深度学习的差分隐私预算优化方法[J]. 物联网学报, 2022, 6(2): 65-76.
[5]	徐宣哲, 宁珂, 郑学敏, 赵明心, 徐萌萌, 吴南健, 刘力源. 基于硬件仿真系统的边缘计算人工智能视觉芯片设计验证[J]. 物联网学报, 2022, 6(1): 20-28.
[6]	李国权, 徐永海, 林金朝, 黄正文. 基于Res-DNN的端到端MIMO系统信号检测算法[J]. 物联网学报, 2022, 6(1): 65-72.
[7]	谈玲, 荣杉山, 夏景明, SajibSarker, 马雯杰. 基于IR-VGG的多分类皮肤病实时诊断[J]. 物联网学报, 2021, 5(3): 115-125.
[8]	林椿珉, 曾烈康, 陈旭. 边缘智能驱动的高能效无人机自主导航算法研究[J]. 物联网学报, 2021, 5(2): 87-96.
[9]	陈慕涵,郭佳佳,李潇,金石. 基于深度学习的大规模MIMO信道状态信息反馈[J]. 物联网学报, 2020, 4(1): 33-44.
[10]	廖勇,姚海梅,花远肖. 一种基于深度学习的物联网信道状态信息获取算法[J]. 物联网学报, 2019, 3(1): 8-13.