基于低成本物联网芯片ESP32的人体行为识别系统

doi:10.11959/j.issn.2096-3750.2023.00330

物联网学报 ›› 2023, Vol. 7 ›› Issue (2): 133-142.doi: 10.11959/j.issn.2096-3750.2023.00330

基于低成本物联网芯片ESP32的人体行为识别系统

胡超¹, 鲁邦彦¹, 杨彦兵¹^,², 陈哲³, 张磊¹, 陈良银¹^,²

¹ 四川大学计算机学院，四川成都 610065
² 四川大学工业互联网研究院，四川成都 610065
³ 广州爱闻思人工智能有限公司，广东广州 510555

修回日期:2023-03-09 出版日期:2023-06-30 发布日期:2023-06-01
作者简介:胡超（2000- ），男，四川大学计算机学院硕士生，主要研究方向为无线网络、Wi-Fi感知、可见光通信
鲁邦彦（2000- ），男，四川大学计算机学院硕士生，主要研究方向为可见光通信、嵌入式Linux、无线网络
杨彦兵（1987- ），男，博士，四川大学计算机学院副研究员，主要研究方向为智能感知与通信、可见光通信与传感
陈哲（1988- ），男，博士，广州爱闻思人工智能有限公司研究员，主要研究方向为AIoT、人工智能
张磊（1978- ），男，博士，四川大学计算机学院副教授，主要研究方向为数据挖掘、移动计算
陈良银（1968- ），男，博士，四川大学计算机学院教授，主要研究方向为物联网系统及安全、工业互联网、智能识别等
基金资助:
国家自然科学基金资助项目(61902267);国家自然科学基金资助项目(62072319);四川省科技厅重点研发项目(2020YFS0575);四川大学-宜宾市战略合作专项项目(2020CDYB-29)

Human activity recognition system based on low-cost IoT chip ESP32

Chao HU¹, Bangyan LU¹, Yanbing YANG¹^,², Zhe CHEN³, Lei ZHANG¹, Liangyin CHEN¹^,²

¹ College of Computer Science, Sichuan University, Chengdu 610065, China
² Industrial Internet Research Institute, Sichuan University, Chengdu 610065, China
³ AIWise Co., Ltd., Guangzhou 510555, China

Revised:2023-03-09 Online:2023-06-30 Published:2023-06-01
Supported by:
The National Natural Science Foundation of China(61902267);The National Natural Science Foundation of China(62072319);The Key Research and Development Program of Science and Technology Department of Sichuan Province(2020YFS0575);The Sichuan University-Yibin Municipal People’s Government University and City Strategic Cooperation Special Fund Project(2020CDYB-29)

摘要/Abstract

摘要：

人体行为识别广泛存在于运动管理、行为分类等应用中，当前的人体行为识别应用主要分为基于摄像机、基于可穿戴设备和基于Wi-Fi感知3类。其中，基于摄像机的人体行为识别应用存在隐私泄露的风险，基于可穿戴设备的人体行为识别应用存在续航短、精度差等问题。基于Wi-Fi感知的人体行为识别一般通过Wi-Fi网卡或软件无线电设备识别信道状态信息变化的规律，从而推测用户行为，不存在隐私泄露和续航短的问题，但Wi-Fi网卡需要依靠计算机且软件无线电平台价格昂贵，极大地限制了Wi-Fi感知的应用场景。针对上述问题，提出了一种基于低成本物联网芯片 ESP32 的人体行为识别系统。具体地，所提系统首先使用 Hampel 滤波器和高斯滤波器对ESP32获得的信道状态信息进行预处理，然后使用主成分分析和离散小波变换降低数据的维度，最后通过K最近邻（KNN, K-nearest neighbor）算法对数据进行分类。实验结果表明该系统在仅使用两个ESP32节点的情况下，可以达到与当前主流Wi-Fi感知系统（Intel 5300网卡）相近的识别准确率，6种行为的平均准确率为98.6%。

关键词: 人体行为识别, 信道状态信息, KNN, 离散小波变换, 动态时间规整

Abstract:

Human activity recognition widely exists in applications such as sports management and activity classification.The current human activity recognition applications are mainly divided into three types: camera-based, wearable device-based, and Wi-Fi awareness-based.Among them, the camera-based human activity recognition application has the risk of privacy leakage, and the wearable device-based human activity recognition application has problems such as short battery life and poor accuracy.Human activity recognition based on Wi-Fi sensing generally uses Wi-Fi network cards or software-defined radio devices to identify the rules of channel state information changes, so as to infer user activity.It does not have the problems of privacy leakage and short battery life.But Wi-Fi network cards need to rely on computers and software-defined radio platforms are expensive, which greatly limit the application scenarios of Wi-Fi sensing.Aiming at the above problems, a human activity recognition system based on the low-cost IoT chip ESP32 was proposed.Specifically, the Hampel filter and Gaussian filter were used to preprocess the channel state information obtained by ESP32.Then, the principal component analysis and discrete wavelet transform were utilized to reduce the dimension of the data.Finally, the K-nearest neighbor (KNN) algorithm was applied to classify data.The experimental results show that the system can achieve a recognition accuracy which close to the current mainstream Wi-Fi perception system (Intel 5300 network card) when only two ESP32 nodes are deployed, and the average accuracy rate for the six activities is 98.6%.

Key words: human activity recognition, channel state information, KNN, discrete wavelet transform, dynamic time warping

中图分类号:

TP393

胡超, 鲁邦彦, 杨彦兵, 陈哲, 张磊, 陈良银. 基于低成本物联网芯片ESP32的人体行为识别系统[J]. 物联网学报, 2023, 7(2): 133-142.

Chao HU, Bangyan LU, Yanbing YANG, Zhe CHEN, Lei ZHANG, Liangyin CHEN. Human activity recognition system based on low-cost IoT chip ESP32[J]. Chinese Journal on Internet of Things, 2023, 7(2): 133-142.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

图11

图12

参考文献 30

[1]	WANG B H , YU J , WANG K ,et al. Fall detection based on dual-channel feature integration[J]. IEEE Access, 2020(8): 103443-103453.
[2]	VRSKOVA R , HUDEC R , KAMENCAY P ,et al. A new approach for abnormal human activities recognition based on ConvLSTM architecture[J]. Sensors (Basel,Switzerland), 2022,22(8): 2946.
[3]	HORN B K P , SCHUNCK B G . Determining optical flow[J]. Artificial Intelligence, 1981,17(1/2/3): 185-203.
[4]	LERTNIPHONPHAN K , ARAMVITH S , CHALIDABHONGSE T H . Human action recognition using direction histograms of optical flow[C]// Proceedings of 2011 11th International Symposium on Communications ＆ Information Technologies (ISCIT). Piscataway:IEEE Press, 2011: 574-579.
[5]	WANG H , KL?SER A , SCHMID C ,et al. Action recognition by dense trajectories[C]// Proceedings of CVPR. Piscataway:IEEE Press, 2011: 3169-3176.
[6]	WANG H , KL?SER A , SCHMID C ,et al. Dense trajectories and motion boundary descriptors for action recognition[J]. International Journal of Computer Vision, 2013,103(1): 60-79.
[7]	WANG H , SCHMID C . Action recognition with improved trajectories[C]// Proceedings of 2013 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2014: 3551-3558.
[8]	PEREZ-GAMBOA S , SUN Q Q , ZHANG Y . Improved sensor based human activity recognition via hybrid convolutional and recurrent neural networks[C]// Proceedings of 2021 IEEE International Symposium on Inertial Sensors and Systems (INERTIAL). Piscataway:IEEE Press, 2021: 1-4.
[9]	YURTMAN A , BARSHAN B . Activity recognition invariant to sensor orientation with wearable motion sensors[J]. Sensors (Basel,Switzerland), 2017,17(8): 1838.
[10]	JETHANANDANI M , PERUMAL T , SHARMA A . Random k-labelsets method for human activity recognition with multi-sensor data in smart home[C]// Proceedings of 2019 IEEE 16th India Council International Conference (INDICON). Piscataway:IEEE Press, 2020: 1-4.
[11]	HOODA D , RANI R . Ontology driven human activity recognition in heterogeneous sensor measurements[J]. Journal of Ambient Intelligence and Humanized Computing, 2020(11): 5947-5960.
[12]	NWEKE H F , TEH Y W , MUJTABA G ,et al. Multi-sensor fusion based on multiple classifier systems for human activity identification[J]. Human-centric Computing and Information Sciences, 2019(9): 1-44.
[13]	ESPINOSA H G , THIEL D V , SORELL M ,et al. Can we trust inertial and heart rate sensor data from an APPLE watch device?[C]// Proceedings of 13th Conference of the International Sports Engineering Association. Basel Switzerland:MDPI, 2020,49(1): 128.
[14]	LIANG Y R , ZHAO C Z , YUAN H ,et al. A review of rechargeable batteries for portable electronic devices[J]. InfoMat, 2019,1(1): 6-32.
[15]	ZHANG Y H , ZHANG L . WiFi-based contactless activity recognition on smartphones[C]// Proceedings of 2017 IEEE/CIC International Conference on Communications in China (ICCC). Piscataway:IEEE Press, 2018: 1-6.
[16]	CHEN J , HUANG X Y , JIANG H ,et al. Low-cost and device-free human activity recognition based on hierarchical learning model[J]. Sensors (Basel,Switzerland), 2021,21(7): 2359.
[17]	HSIEH C F , CHEN Y C , HSIEH C Y ,et al. Device-free indoor human activity recognition using Wi-Fi RSSI:machine learning approaches[C]// Proceedings of 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan). Piscataway:IEEE Press, 2020: 1-2.
[18]	GU Y , REN F J , LI J . Paws:passive human activity recognition based on Wi-Fi ambient signals[J]. IEEE Internet of Things Journal, 2015,3(5): 796-805.
[19]	SIGG S , HOCK M , SCHOLZ M ,et al. Passive,device-free recognition on your mobile phone:tools,features and a case study[C]// International Conference on Mobile and Ubiquitous Systems:Computing,Networking,and Services. Cham:Springer, 2014: 435-446.
[20]	SIGG S , BLANKE U , TR?STER G . The telepathic phone:frictionless activity recognition from WiFi-RSSI[C]// Proceedings of 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom). Piscataway:IEEE Press, 2014: 148-155.
[21]	HALPERIN D , HU W J , SHETH A ,et al. Tool release:gathering 802.11n traces with channel state information[J]. ACM SIGCOMM Computer Communication Review, 2011,41(1): 53.
[22]	GUO L L , WANG L , LIU J L ,et al. HuAc:human activity recognition using crowdsourced WiFi signals and skeleton data[J]. Wireless Communications and Mobile Computing, 2018,2018: 1-15.
[23]	JIANG W J , XUE H F , MIAO C L ,et al. Towards 3D human pose construction using WiFi[C]// Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. New York:ACM Press, 2020: 1-14.
[24]	FEI H , XIAO F , HAN J S ,et al. Multi-variations activity based gaits recognition using commodity WiFi[J]. IEEE Transactions on Vehicular Technology, 2020,69(2): 2263-2273.
[25]	CHEN Z H , ZHANG L , JIANG C Y ,et al. WiFi CSI based passive human activity recognition using attention based BLSTM[J]. IEEE Transactions on Mobile Computing, 2019,18(11): 2714-2724.
[26]	杨蓉 . 全球物联网设备数量增长 9%达到 123 亿[J]. 计算机与网络, 2021,47(22): 9-10.
	YANG R . The global number of iot devices grew 9 percent to 12.3 billion[J]. Computer and network, 2021,47(22): 9-10.
[27]	MA Y S , ZHOU G , WANG S Q . WiFi sensing with channel state information:a survey[J]. ACM Computing Surveys, 2020,52(3): 1-36.
[28]	SAINI R , CHERNIAKOV M , LENIVE V . Direct path interference suppression in bistatic system:DTV based radar[C]// Proceedings of the International Conference on Radar (IEEE Cat.No.03EX695). Piscataway:IEEE Press, 2004: 309-314.
[29]	PEARSON R K , NEUVO Y , ASTOLA J ,et al. Generalized hampel filters[J]. EURASIP Journal on Advances in Signal Processing, 2016(1): 1-18.
[30]	COVER T , HART P . Nearest neighbor pattern classification[J]. IEEE Transactions on Information Theory, 1967,13(1): 21-27.

滤波器	平均识别准确率	处理时延/ms
Hampel滤波器	98.3%	2.1
中值滤波器	97.5%	20.4

基于低成本物联网芯片ESP32的人体行为识别系统

Human activity recognition system based on low-cost IoT chip ESP32

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 30

相关文章 8

Metrics

推荐阅读 0

[1]	牛超, 杨卫东, 胡鹏明, 高向上, 沈二波. Wi-freshness：基于CSI的猪肉新鲜度检测系统研究[J]. 物联网学报, 2023, 7(2): 143-152.
[2]	苏麟, 党小超, 郝占军, 汝春瑞, 尚旭. 基于WPT-MEC的动态自适应卸载方法[J]. 物联网学报, 2022, 6(4): 128-138.
[3]	赵广智, 周志鹏, 龚伟, 陈绍青, 周浩泉. 基于主动学习和Wi-Fi感知的人体识别系统[J]. 物联网学报, 2022, 6(1): 44-52.
[4]	王福展, 朱晓荣, 陈美娟, 朱洪波. 基于生成对抗网络的高精度室内无线定位方法[J]. 物联网学报, 2021, 5(2): 107-115.
[5]	张雷,张跃,李明雪,史新国,翟勃,王卫龙. 基于CSI的井下人员行为识别方法[J]. 物联网学报, 2020, 4(4): 26-31.
[6]	杨子元,许晓斌,李欣,赵一萌. 基于智能感知技术的用电事件识别方法研究[J]. 物联网学报, 2019, 3(4): 109-115.
[7]	李云鹤. 智能穿戴设备基于动态模板匹配算法的3D手势识别[J]. 物联网学报, 2019, 3(1): 97-105.
[8]	党小超,司雄,郝占军,黄亚宁,黑毅力. 复杂室内环境下基于CSI的定位方法[J]. 物联网学报, 2018, 2(4): 68-77.