混合骨骼特征的三帧间差分手势识别方法研究

doi:10.11959/j.issn.2096-6652.202211

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (4): 592-599.doi: 10.11959/j.issn.2096-6652.202211

混合骨骼特征的三帧间差分手势识别方法研究

张永强¹^,², 宋美霖¹, 刘天虎¹, 满梦华²

¹ 河北科技大学信息科学与工程学院，河北石家庄 050018
² 陆军工程大学石家庄校区电磁环境效应国家级重点实验室，河北石家庄 050003

修回日期:2021-10-17 出版日期:2022-12-15 发布日期:2022-12-01
作者简介:张永强（1981− ），男，博士，河北科技大学信息科学与工程学院副教授，主要研究方向为人工智能、物联网
宋美霖（1997− ），女，河北科技大学信息科学与工程学院硕士生，主要研究方向为人工智能和图像处理
刘天虎（1995− ），男，河北科技大学信息科学与工程学院硕士生，主要研究方向为人工智能和图像处理
满梦华（1984− ），男，博士，陆军工程大学石家庄校区电磁环境效应国家级重点讲师，主要研究方向为电磁防护理论与技术
基金资助:
国防基础科研计划(JCKYS2020DC202);河北省自然科学基金资助项目(F2018208116);河北省高等学校科学技术研究重点项目(ZD2020176);河北省高等学校科学技术研究重点项目(ZD2021048)

Research on three frame difference gesture recognition method based on mixed bone features

Yongqiang ZHANG¹^,², Meilin SONG¹, Tianhu LIU¹, Menghua MAN²

¹ School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang 050018, China
² National Key Laboratory on Electromagnetic Environment Effects, Shijiazhuang Campus, Army Engineering University, Shijiazhuang 050003, China

Revised:2021-10-17 Online:2022-12-15 Published:2022-12-01
Supported by:
The National Defense Basic Research Program(JCKYS2020DC202);The Natural Science Foundation of Hebei Province(F2018208116);The Key Project of Science and Technology Research in Colleges and Universities of Hebei Province(ZD2020176);The Key Project of Science and Technology Research in Colleges and Universities of Hebei Province(ZD2021048)

摘要/Abstract

摘要：

三帧间差分检测作为手势识别中的一种优秀算法，可在一定程度上解决二帧间差分法的“重影”问题。但三帧间差分检测在识别手势时会出现空洞，无法适应光照骤变等情况。针对此问题，提出了一种混合骨骼特征的三帧间差分手势识别方法。该方法首先对数据集进行二值化以统一大小，降低背景颜色干扰和计算量；然后使用混合骨骼特征的三帧间差分进行手势检测与追踪；最后通过神经网络进行手势识别分类。该方法可以有效地重点训练手部特征，显著降低了背景对手势识别的干扰，同时还提升了识别效率。经实验验证，该方法在复杂背景下的识别率最低为 93.38%，最高为 99.99%，满足鲁棒性的要求，为在复杂图像背景下的手势识别提供了一个新的思路。

关键词: 手势识别, 三帧间差分, 骨骼特征, 神经网络

Abstract:

As an excellent algorithm in gesture recognition, three frame difference detection can solve the “double shadow” problem of two frame difference method to a certain extent.But the three frame difference detection will have holes when recognizing gestures, and it cannot adapt to sudden changes in lighting and so on.To solve this problem, the three frame difference gesture recognition method based on mixed bone features was proposed.Firstly, the binary size of the data set was unified, and the background color inter-ference and computation were reduced.Secondly, the three frame difference of mixed bone features was used to detect and track gestures.Finally, the neural network was used for gesture recognition and classification.This method could effectively train hand characteristics, significantly reduce the interference of background on gesture recognition, and improve the recognition efficiency.The experimental results showed that the minimum recognition rate of this method was 93.38% and the maximum was 99.99% in complex background, which could meet the requirement of robustness.This method provides a new idea for gesture recognition in complex image background.

Key words: gesture recognition, three frame difference, bone feature, neural network

中图分类号:

TP391

张永强, 宋美霖, 刘天虎, 等. 混合骨骼特征的三帧间差分手势识别方法研究[J]. 智能科学与技术学报, 2022, 4(4): 592-599.

Yongqiang ZHANG, Meilin SONG, Tianhu LIU, et al. Research on three frame difference gesture recognition method based on mixed bone features[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 592-599.

图/表 18

表1

图1

图2

图3

图4

图5

图6

图7

表2

图8

图9

图10

图11

图12

图13

图14

图15

表3

参考文献 21

[1]	HAMDA M , MAHMOUDI A . Hand gesture recognition using Kinect’s geometric and HOG features[C]// Proceedings of the 2nd International Conference on Big Data,Cloud and Applications. New York:ACM Press, 2017: 1-5.
[2]	MALYSA G , WANG D , NETSCH L ,et al. Hidden Markov model-based gesture recognition with FMCW radar[C]// Proceedings of 2016 IEEE Global Conference on Signal and Information Processing. Piscataway:IEEE Press, 2016: 1017-1021.
[3]	SANGI P , MATILAINEN M , SILVéN O , . Rotation tolerant hand pose recognition using aggregation of gradient orientations[C]// Proceedings of 2016 International Conference on Image Analysis and Recognition. Cham:Springer, 2016.
[4]	MOHANTY A , RAMBHATLA S S , SAHAY R R . Deep gesture:static hand gesture recognition using CNN[M]/ Advances in intelligent systems and computing. Singapore: Springer Singapore, 2016: 449-461.
[5]	GIRSHICK R , DONAHUE J , DARRELL T ,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 580-587.
[6]	REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
[7]	REDMON J , DIVVALA S , GIRSHICK R ,et al. You only look once:unified,real-time object detection[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 779-788.
[8]	ZHANG Z G , ZHANG H M , ZHANG Z F . Using three frame difference algorithm to detect moving objects[C]// Proceedings of the International Conference on Cyber Security Intelligence and Analytics.[S.l.:s.n.], 2019.
[9]	JU J G , XING J S . Moving object detection based on smoothing three frame difference method fused with RPCA[J]. Multimedia Tools and Applications, 2019,78(21): 29937-29951.
[10]	皮览月 . 基于3D骨骼数据的动态手势识别研究[D]. 郑州:郑州大学, 2019.
	PI L Y . Research on dynamic gesture recognition based on 3D bone data[D]. Zhengzhou:Zhengzhou University, 2019.
[11]	王松林, 徐文胜 . 基于Kinect深度信息与骨骼信息的手指尖识别方法[J]. 计算机工程与应用, 2016,52(3): 169-173,235.
	WANG S L , XU W S . Fingertips recognition based on Kinect depth and skeleton data[J]. Computer Engineering and Applications, 2016,52(3): 169-173,235.
[12]	高远, 路杨, 李佩琛 . 一种基于三帧差分和混合高斯模型的运动目标检测方法[J]. 科技创新与应用, 2017(31): 21-23.
	GAO Y , LU Y , LI P C . A moving target detection method based on three-frame difference and mixed gaussian model[J]. Technology Innovation and Application, 2017(31): 21-23.
[13]	尹红娟, 栾帅 . 三帧差分运动目标检测算法分析与验证[J]. 计算机与数字工程, 2017,45(1): 69-71,87.
	YIN H J , LUAN S . Three-image difference algorithm for moving target detection[J]. Computer ＆ Digital Engineering, 2017,45(1): 69-71,87.
[14]	BADI H , HUSSEIN S H , KAREEM S A . Retraction note to:feature extraction and ML techniques for static gesture recognition[J]. Neural Computing and Applications, 2017,28(4): 851.
[15]	JADOOKI S , MOHAMAD D , SABA T ,et al. Fused features mining for depth-based hand gesture recognition to classify blind human communication[J]. Neural Computing and Applications, 2017,28(11): 3285-3294.
[16]	王秉睿, 兰慧盈, 陈云霁 . 深度学习编程框架[J]. 大数据, 2018,4(4): 56-63.
	WANG B R , LAN H Y , CHEN Y J . Programming frameworks for deep learning algorithms[J]. Big Data Research, 2018,4(4): 56-63.
[17]	于璠 . 新一代深度学习框架研究[J]. 大数据, 2020,6(4): 69-80.
	YU F . Research on the next-generation deep learning framework[J]. Big Data Research, 2020,6(4): 69-80.
[18]	SIMON T , JOO H , MATTHEWS I ,et al. Hand keypoint detection in single images using multiview bootstrapping[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 4645-4653.
[19]	CAO Z , HIDALGO G , SIMON T ,et al. OpenPose:realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(1): 172-186.
[20]	谢之宇, 蒋晓瑜, 汪熙 ,等. 改进的帧差法在目标匹配中的应用[J]. 计算机工程与应用, 2011,47(34): 208-211.
	XIE Z Y , JIANG X Y , WANG X ,et al. Improved frame difference algorithm in application of target-matching[J]. Computer Engineering and Applications, 2011,47(34): 208-211.
[21]	BOUTHILLIER X , KONDA K , VINCENT P ,et al. Dropout as data augmentation[J]. arXiv preprint, 2015,arXiv:1506.08700.

层	输出格式
conv2d_1	(None, 64, 64, 32)
activation_1	(None, 64, 64, 32)
batch_normalization_1	(None, 64, 64, 32)
max_pooling2d_1	(None, 32, 32, 32)
conv2d_2	(None, 32, 32, 64)
activation_2	(None, 32, 32, 64)
batch_normalization_2	(None, 32, 32, 64)
conv2d_3	(None, 32, 32, 64)
activation_3	(None, 32, 32, 64)
batch_normalization_3	(None, 32, 32, 64)
max_pooling2d_2	(None, 16, 16, 64)
conv2d_4	(None, 16, 16, 128)
activation_4	(None, 16, 16, 128)
batch_normalization_4	(None, 16, 16, 128)
conv2d_5	(None, 16, 16, 128)
activation_5	(None, 16, 16, 128)
batch_normalization_5	(None, 16, 16, 128)
conv2d_6	(None, 16, 16, 128)
activation_6	(None, 16, 16, 128)
batch_normalization_6	(None, 16, 16, 128)
max_pooling2d_3	(None, 8, 8, 128)
flatten_1	(None, 8 192)
dense_1	(None, 512)
activation_7	(None, 512)
batch_normalization_7	(None, 512)
dropout_1	(None, 512)
dense_2	(None, 4)
activation_8	(None, 4)

超参数	设置
初始化学习率	0.01
训练总轮数（epoch）	30
每轮训练次数	128
批次大小	32
激活函数	ReLU
dropout	0.6

网络	不使用骨骼特征识别时间/s	使用骨骼特征识别时间/s
AlexNet	2.31	1.25
ResNet	2.57	1.56

混合骨骼特征的三帧间差分手势识别方法研究

Research on three frame difference gesture recognition method based on mixed bone features

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 21

相关文章 15

Metrics

推荐阅读 0

[1]	罗欣欢, 王奕璇, 李炜, 陈曦. 基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用[J]. 智能科学与技术学报, 2023, 5(1): 58-68.
[2]	顾宙瑜, 於跃成, 者甜甜. Rapider-YOLOX：高效的轻量级目标检测网络[J]. 智能科学与技术学报, 2023, 5(1): 92-103.
[3]	卢经纬, 程相, 王飞跃. 求解微分方程的人工智能与深度学习方法：现状及展望[J]. 智能科学与技术学报, 2022, 4(4): 461-476.
[4]	丛晓峰, 桂杰, 章军. 基于视觉Transformer的多损失融合水下图像增强网络[J]. 智能科学与技术学报, 2022, 4(4): 522-532.
[5]	覃缘琪, 季青原, 葛俊, 戴星原, 陈圆圆, 王晓. 城市交通路网动态短时推理与精准预测研究[J]. 智能科学与技术学报, 2022, 4(3): 380-395.
[6]	邵虹, 张鸣坤, 崔文成. 基于分层卷积神经网络的皮肤镜图像分类方法[J]. 智能科学与技术学报, 2021, 3(4): 474-481.
[7]	李颖, 陈龙, 黄钊宏, 孙杨, 蔡国榕. 基于多尺度卷积神经网络特征融合的植株叶片检测技术[J]. 智能科学与技术学报, 2021, 3(3): 304-311.
[8]	张阳, 胡月, 辛东嵘. 一种考虑时空关联的深度学习短时交通流预测方法[J]. 智能科学与技术学报, 2021, 3(2): 172-178.
[9]	高寒冰, 张驰, 靳明艳, 肖杨, 丛丰裕. 基于多源域自适应的跨被试情感脑电识别[J]. 智能科学与技术学报, 2021, 3(1): 59-64.
[10]	权利敏,杨翠丽,乔俊飞. 基于CFNN的污水处理过程溶解氧浓度在线控制[J]. 智能科学与技术学报, 2020, 2(3): 261-267.
[11]	张国宾,王新迎. 基于混合神经网络的光伏组件输出特性数据驱动建模方法[J]. 智能科学与技术学报, 2020, 2(2): 169-178.
[12]	郝晓辰,冀亚坤,郑立召,史鑫,赵彦涛. 基于差分进化的水泥烧成系统动态优化算法[J]. 智能科学与技术学报, 2020, 2(2): 186-193.
[13]	林东源,陈晓丰,孙文涛,李伟凯,夏砚楠. 时滞可交换四元数神经网络稳定性分析[J]. 智能科学与技术学报, 2020, 2(1): 80-87.
[14]	刘文华,李浥东,王涛,邬俊,金一. 基于高维特征表示的交通场景识别[J]. 智能科学与技术学报, 2019, 1(4): 392-399.
[15]	牛培峰,马云鹏,张欣欣,胡晓宾. 基于人工智能技术的火电厂燃煤锅炉智能燃烧优化研究及应用[J]. 智能科学与技术学报, 2019, 1(2): 163-170.