基于连续图像深度学习的Wi-Fi人体行为识别方法

doi:10.11959/j.issn.1000-436x.2020141

Abstract

Abstract:

For the problems existing in most of the researches,such as weak anti-noise ability,incompatible signal size and insufficient feature extraction of deep-learning-based Wi-Fi human activity recognition,a kind of sequential image deep learning-based recognition method was proposed.Based on the idea of sequential image deep learning,a series of image frames were reconstructed from time-varied Wi-Fi signal to ensure the consistency of input size.In addition,a low-rank decomposition method was innovatively designed to separate low-rank activity information merged in noises.Finally,a deep model combining temporal stream and spatial stream was proposed to automatically capture the spatiotemporal features from length-varied image sequences.The proposed method was extensively tested in WiAR dataset and self collected dataset.The experimental results show the proposed method could achieve the accuracy of 0.94 and 0.96,which indicate its high-accuracy performance and robustness in pervasive environments.

Key words: activity recognition, Wi-Fi signal, deep learning, image recognition, low-rank decomposition

CLC Number:

TN391

Qizhen ZHOU,Jianchun XING,Qiliang YANG,Deshuai HAN. Sequential image deep learning-based Wi-Fi human activity recognition method[J]. Journal on Communications, 2020, 41(8): 43-54.

Figures/Tables 14

References 30

[1]	YANG Q , . Activity recognition:linking low-level sensors to high-level intelligence[C]// Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. New York:ACM Press, 2009: 20-25.
[2]	罗会兰, 王婵娟, 卢飞 . 视频行为识别综述[J]. 通信学报, 2018,39(6): 169-180.
	LUO H L , WANG C J , LU F . Survey of video behavior recognition[J]. Journal on Communications, 2018,39(6): 169-180.
[3]	王钰翔, 李晟洁, 王皓 ,等. 基于 Wi-Fi 的非接触式行为识别研究综述[J]. 浙江大学学报(工学版), 2017,51(4): 648-654.
	WANG Y X , LI S J , WANG H ,et al. Survey on Wi-Fi based contact-less activity recognition[J]. Journal of Zhejiang University (Engineer-ing Science), 2017,51(4): 648-654.
[4]	LARA O D , LABRADOR M A . A survey on human activity recognition using wearable sensors[J]. IEEE Communications Surveys ＆ Tutorials, 2012,15(3): 1192-1209.
[5]	殷晓玲, 陈晓江, 夏启寿 ,等. 基于智能手机内置传感器的人体运动状态识别[J]. 通信学报, 2019,40(3): 157-169.
	YIN X L , CHEN X J , XIA Q S ,et al. Human motion state recognition based on smart phone built-in sensor[J]. Journal on Communications, 2019,40(3): 157-169.
[6]	HERATH S , HARANDI M , PORIKLI F . Going deeper into action recognition:a survey[J]. Image and Vision Computing, 2017,60: 4-21.
[7]	BAHL P , PADMANABHAN V N , BAHL V ,et al. RADAR:an in-building RF-based user location and tracking system[C]// The 19th Annual Joint Conferences of the IEEE Computer and Communications Societies. Piscataway:IEEE Press, 2000: 775-784.
[8]	CISCO. Global mobile data traffic forecast update,2016–2021 white paper[R]. Cisco,(2017-02-07)[2020-02-26].
[9]	鲁勇, 吕绍和, 王晓东 ,等. 基于 Wi-Fi 信号的人体行为感知技术研究综述[J]. 计算机学报, 2019,42(2): 3-23.
	LU Y , LYU S H , WANG X D ,et al. A survey on Wi-Fi based human behavior analysis technology[J]. Chinese Journal of Computers, 2019,42(2): 3-23.
[10]	SEIFELDIN M , SAEED A , KOSBA A E ,et al. Nuzzer:a large-scale device-free passive localization system for wireless environments[J]. IEEE Transactions on Mobile Computing, 2012,12(7): 1321-1334.
[11]	HALPERIN D , HU W , SHETH A ,et al. Tool release:gathering 802.11 n traces with channel state information[J]. ACM SIGCOMM Computer Communication Review, 2011,41(1): 53-53.
[12]	党小超, 黄亚宁, 郝占军 ,等. 基于信道状态信息的无源室内人员日常行为检测方法[J]. 通信学报, 2019,40(4): 160-170.
	DANG X C , HUANG Y N , HAO Z J ,et al. Passive indoor human daily behavior detection method based on channel state information[J]. Journal on Communications, 2019,40(4): 160-170.
[13]	PALIPANA S , ROJAS D , AGRAWAL P ,et al. FallDeFi:ubiquitous fall detection using commodity Wi-Fi devices[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2018,1(4): 1-25.
[14]	ZENG Y , PATHAK P H , MOHAPATRA P . WiWho:Wi-Fi-based person identification in smart spaces[C]// 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks. Piscataway:IEEE Press, 2016: 1-12.
[15]	TAN S , YANG J . WiFinger:leveraging commodity Wi-Fi for fine-grained finger gesture recognition[C]// Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computin. New York:ACM Press, 2016: 201-210.
[16]	WANG W , LIU A X , SHAHZAD M ,et al. Understanding and modeling of Wi-Fi signal based human activity recognition[C]// Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. New York:ACM Press, 2015: 65-76.
[17]	YUE S , HE H , WANG H ,et al. Extracting multi-person respiration from entangled RF signals[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2018,2(2):86.
[18]	ZOU H , ZHOU Y , YANG J ,et al. Device-free occupancy detection and crowd counting in smart buildings with Wi-Fi-enabled IoT[J]. Energy and Buildings, 2018,174: 309-322.
[19]	MA Y , ZHOU G , WANG S . Wi-Fi sensing with channel state information:a survey[J]. ACM Computing Surveys (CSUR), 2019,52(3): 1-36.
[20]	周鑫, 何晓新, 郑昌文 . 基于图像深度学习的无线电信号识别[J]. 通信学报, 2019,40(7): 114-125.
	ZHOU X , HE X X , ZHENG C W . Radio signal recognition based on image deep learning[J]. Journal on Communications, 2019,40(7): 114-125.
[21]	MA Y , ZHOU G , WANG S ,et al. SignFi:sign language recognition using Wi-Fi[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2018,2(1): 1-21.
[22]	ZHOU Q , XING J , CHEN W ,et al. From signal to image:enabling fine-grained gesture recognition with commercial Wi-Fi devices[J]. Sensors, 2018,18(9): 3142-3163.
[23]	ZHOU T , TAO D . Godec:randomized low-rank ＆ sparse matrix decomposition in noisy case[C]// Proceedings of the 28th International Conference on Machine Learning.[S.n.:s.l. ], 2011: 33-40.
[24]	WANG X , GAO L , SONG J ,et al. Beyond frame-level CNN:saliency-aware 3-D CNN with LSTM for video action recognition[J]. IEEE Signal Processing Letters, 2016,24(4): 510-514.
[25]	GUO L , WANG L , LIN C ,et al. WiAR:a public dataset for Wi-Fi-based activity recognition[J]. IEEE Access, 2019,7(1): 154935-154945.
[26]	QIAN K , WU C , ZHANG Y ,et al. Widar2.0:passive human tracking with a single Wi-Fi link[C]// Proceedings of the 16th Annual International Conference on Mobile Systems,Applications,and Services. New York:ACM Press, 2018: 350-361.
[27]	YOSINSKI J , CLUNE J , BENGIO Y ,et al. How transferable are features in deep neural networks?[J]. In Advances in Neural Information Processing Systems, 2014,1(1): 3320-3328.
[28]	WANG F , GONG W , LIU J . On spatial diversity in Wi-Fi-based human activity recognition:a deep learning-based approach[J]. IEEE Internet of Things Journal, 2018,6(2): 2035-2047.
[29]	OHARA K , MAEKAWA T , MATSUSHITA Y . Detecting state changes of indoor everyday objects using Wi-Fi channel state information[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2017,1(3): 1-28.
[30]	YOUSEFI S , NARUI H , DAYAL S ,et al. A survey on behavior recognition using Wi-Fi channel state information[J]. IEEE Communications Magazine, 2017,55(10): 98-104.

Metrics

Recommended 0

No Suggested Reading articles found!

方法	信号预处理	模型输入	模型主体结构	活动种类	精度
文献[28]	低通滤波+PCA+有效区域选择	幅值STFT图	一层Conv+2层LSTM+一层fc+Softmax	8种	0.96
文献[29]	BSS	原始+分解后幅值/相位	3层Conv+2层LSTM+Softmax+HMM	4种	0.85
SIL-Fi	RPCA	幅值连续帧	3层Conv+2层LSTM+一层fc+Softmax	16种(WiAR)	0.94
				8种(自主采集)	0.96

方法	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	平均精度
文献[28]	0.92	0.89	0.84	0.94	0.96	0.85	0.92	0.90	0.96	0.93	0.86	0.95	0.90	0.85	0.95	0.97	0.911 9
文献[29]	0.90	0.94	0.93	0.93	0.92	0.83	0.92	0.88	0.97	0.94	0.93	0.92	0.89	0.85	0.98	0.98	0.919 4
SIL-Fi	0.92	0.98	0.93	0.96	0.94	0.85	0.94	0.86	0.99	0.97	0.99	0.94	0.90	0.99	1.00	0.93	0.942 7

方法	1	2	3	4	5	6	7	8	平均精度
文献[28]	0.92	0.95	0.95	0.96	0.96	0.91	0.93	0.92	0.938 3
文献[29]	0.94	0.95	0.95	0.96	0.92	0.92	0.94	0.94	0.940 6
SIL-Fi	0.95	0.95	0.97	0.97	0.95	0.95	0.96	0.94	0.956 4

序号	深度模型组合方式	WiAR数据集
序号	深度模型组合方式	训练时间/s	平均精度
1	一层Conv+一层LSTM+一层fc+Softmax	18 846	0.54
2	2层Conv+一层LSTM+一层fc+Softmax	10 108	0.77
3	2层Conv+2层LSTM+一层fc+Softmax	10 407	0.71
4	3层Conv+一层LSTM+一层fc+Softmax	4 923	0.82
5	3层Conv+2层LSTM+一层fc+Softmax	6 387	0.94

测试人员序号		会议室					办公室
测试人员序号		女性1	女性2	男性1	男性2	男性3	女性1	女性2	男性1	男性2	男性3
手动特征提取	完备特征^[14]	0.78	0.80	0.81	0.77	0.77	0.72	0.75	0.81	0.76	0.80
	特征挑选^[18]	0.85	0.88	0.87	0.84	0.84	0.83	0.86	0.86	0.81	0.83
自动特征提取	CNN^[22]	0.93	0.97	0.97	0.94	0.93	0.89	0.92	0.95	0.93	0.92
	LSTM^[30]	0.92	0.95	0.94	0.95	0.94	0.91	0.93	0.93	0.94	0.93
	SIL-Fi	0.95	0.97	0.99	0.95	0.94	0.94	0.96	0.97	0.95	0.93

Sequential image deep learning-based Wi-Fi human activity recognition method

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 30

Related Articles 15

Metrics

Recommended 0

[1]	Dongyu CHEN, Hua CHEN, Limin FAN, Yifang FU, Jian WANG. Research on test strategy for randomness based on deep learning [J]. Journal on Communications, 2023, 44(6): 23-33.
[2]	Rongpeng LI, Bingyan WANG, Honggang ZHANG, Zhifeng ZHAO. Design of knowledge enhanced semantic communication receiver [J]. Journal on Communications, 2023, 44(6): 70-76.
[3]	Shuai MA, Ke PEI, Huayan QI, Hang LI, Wen CAO, Hongmei WANG, Hailiang XIONG, Shiyin LI. Research on geomagnetic indoor high-precision positioning algorithm based on generative model [J]. Journal on Communications, 2023, 44(6): 211-222.
[4]	Feibo JIANG, Yubo PENG, Li DONG. Deep image semantic communication model for 6G [J]. Journal on Communications, 2023, 44(3): 198-208.
[5]	Jie YANG, Biao DONG, Xue FU, Yu WANG, Guan GUI. Lightweight decentralized learning-based automatic modulation classification method [J]. Journal on Communications, 2022, 43(7): 134-142.
[6]	Xiuzhang YANG, Guojun PENG, Zichuan LI, Yangqi LYU, Side LIU, Chenguang LI. Research on entity recognition and alignment of APT attack based on Bert and BiLSTM-CRF [J]. Journal on Communications, 2022, 43(6): 58-70.
[7]	Yong LIAO, Shiyi WANG. CSI feedback algorithm based on RM-Net for massive MIMO systems in high-speed mobile environment [J]. Journal on Communications, 2022, 43(5): 166-176.
[8]	Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image [J]. Journal on Communications, 2022, 43(5): 190-203.
[9]	Zenghua ZHAO, Yuefan TONG, Jiayang CUI. Device-independent Wi-Fi fingerprinting indoor localization model based on domain adaptation [J]. Journal on Communications, 2022, 43(4): 143-153.
[10]	Yong LIAO, Gang CHENG, Yujie LI. CSI feedback algorithm based on deep unfolding for massive MIMO systems [J]. Journal on Communications, 2022, 43(12): 77-88.
[11]	Xueyuan DUAN, Yu FU, Kun WANG, Bin LI. LDoS attack detection method based on simple statistical features [J]. Journal on Communications, 2022, 43(11): 53-64.
[12]	Junyan HUO, Ruipeng QIU, Yanzhuo MA, Fuzheng YANG. Reference frame list optimization algorithm in video coding by quality enhancement of the nearest picture [J]. Journal on Communications, 2022, 43(11): 136-147.
[13]	Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy [J]. Journal on Communications, 2022, 43(10): 94-105.
[14]	Hongxia ZHANG, Qi WANG, Dengyue WANG, Ben WANG. Honeypot contract detection of blockchain based on deep learning [J]. Journal on Communications, 2022, 43(1): 194-202.
[15]	Yan YAN, Yiming CONG, Mahmood Adnan, Quanzheng SHENG. Statistics release and privacy protection method of location big data based on deep learning [J]. Journal on Communications, 2022, 43(1): 203-216.