混合骨骼特征的三帧间差分手势识别方法研究

doi:10.11959/j.issn.2096-6652.202211

Abstract

Abstract:

As an excellent algorithm in gesture recognition, three frame difference detection can solve the “double shadow” problem of two frame difference method to a certain extent.But the three frame difference detection will have holes when recognizing gestures, and it cannot adapt to sudden changes in lighting and so on.To solve this problem, the three frame difference gesture recognition method based on mixed bone features was proposed.Firstly, the binary size of the data set was unified, and the background color inter-ference and computation were reduced.Secondly, the three frame difference of mixed bone features was used to detect and track gestures.Finally, the neural network was used for gesture recognition and classification.This method could effectively train hand characteristics, significantly reduce the interference of background on gesture recognition, and improve the recognition efficiency.The experimental results showed that the minimum recognition rate of this method was 93.38% and the maximum was 99.99% in complex background, which could meet the requirement of robustness.This method provides a new idea for gesture recognition in complex image background.

Key words: gesture recognition, three frame difference, bone feature, neural network

CLC Number:

TP391

Yongqiang ZHANG, Meilin SONG, Tianhu LIU, et al. Research on three frame difference gesture recognition method based on mixed bone features[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 592-599.

Figures/Tables 18

References 21

[1]	HAMDA M , MAHMOUDI A . Hand gesture recognition using Kinect’s geometric and HOG features[C]// Proceedings of the 2nd International Conference on Big Data,Cloud and Applications. New York:ACM Press, 2017: 1-5.
[2]	MALYSA G , WANG D , NETSCH L ,et al. Hidden Markov model-based gesture recognition with FMCW radar[C]// Proceedings of 2016 IEEE Global Conference on Signal and Information Processing. Piscataway:IEEE Press, 2016: 1017-1021.
[3]	SANGI P , MATILAINEN M , SILVéN O , . Rotation tolerant hand pose recognition using aggregation of gradient orientations[C]// Proceedings of 2016 International Conference on Image Analysis and Recognition. Cham:Springer, 2016.
[4]	MOHANTY A , RAMBHATLA S S , SAHAY R R . Deep gesture:static hand gesture recognition using CNN[M]/ Advances in intelligent systems and computing. Singapore: Springer Singapore, 2016: 449-461.
[5]	GIRSHICK R , DONAHUE J , DARRELL T ,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 580-587.
[6]	REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
[7]	REDMON J , DIVVALA S , GIRSHICK R ,et al. You only look once:unified,real-time object detection[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 779-788.
[8]	ZHANG Z G , ZHANG H M , ZHANG Z F . Using three frame difference algorithm to detect moving objects[C]// Proceedings of the International Conference on Cyber Security Intelligence and Analytics.[S.l.:s.n.], 2019.
[9]	JU J G , XING J S . Moving object detection based on smoothing three frame difference method fused with RPCA[J]. Multimedia Tools and Applications, 2019,78(21): 29937-29951.
[10]	皮览月 . 基于3D骨骼数据的动态手势识别研究[D]. 郑州:郑州大学, 2019.
	PI L Y . Research on dynamic gesture recognition based on 3D bone data[D]. Zhengzhou:Zhengzhou University, 2019.
[11]	王松林, 徐文胜 . 基于Kinect深度信息与骨骼信息的手指尖识别方法[J]. 计算机工程与应用, 2016,52(3): 169-173,235.
	WANG S L , XU W S . Fingertips recognition based on Kinect depth and skeleton data[J]. Computer Engineering and Applications, 2016,52(3): 169-173,235.
[12]	高远, 路杨, 李佩琛 . 一种基于三帧差分和混合高斯模型的运动目标检测方法[J]. 科技创新与应用, 2017(31): 21-23.
	GAO Y , LU Y , LI P C . A moving target detection method based on three-frame difference and mixed gaussian model[J]. Technology Innovation and Application, 2017(31): 21-23.
[13]	尹红娟, 栾帅 . 三帧差分运动目标检测算法分析与验证[J]. 计算机与数字工程, 2017,45(1): 69-71,87.
	YIN H J , LUAN S . Three-image difference algorithm for moving target detection[J]. Computer ＆ Digital Engineering, 2017,45(1): 69-71,87.
[14]	BADI H , HUSSEIN S H , KAREEM S A . Retraction note to:feature extraction and ML techniques for static gesture recognition[J]. Neural Computing and Applications, 2017,28(4): 851.
[15]	JADOOKI S , MOHAMAD D , SABA T ,et al. Fused features mining for depth-based hand gesture recognition to classify blind human communication[J]. Neural Computing and Applications, 2017,28(11): 3285-3294.
[16]	王秉睿, 兰慧盈, 陈云霁 . 深度学习编程框架[J]. 大数据, 2018,4(4): 56-63.
	WANG B R , LAN H Y , CHEN Y J . Programming frameworks for deep learning algorithms[J]. Big Data Research, 2018,4(4): 56-63.
[17]	于璠 . 新一代深度学习框架研究[J]. 大数据, 2020,6(4): 69-80.
	YU F . Research on the next-generation deep learning framework[J]. Big Data Research, 2020,6(4): 69-80.
[18]	SIMON T , JOO H , MATTHEWS I ,et al. Hand keypoint detection in single images using multiview bootstrapping[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 4645-4653.
[19]	CAO Z , HIDALGO G , SIMON T ,et al. OpenPose:realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(1): 172-186.
[20]	谢之宇, 蒋晓瑜, 汪熙 ,等. 改进的帧差法在目标匹配中的应用[J]. 计算机工程与应用, 2011,47(34): 208-211.
	XIE Z Y , JIANG X Y , WANG X ,et al. Improved frame difference algorithm in application of target-matching[J]. Computer Engineering and Applications, 2011,47(34): 208-211.
[21]	BOUTHILLIER X , KONDA K , VINCENT P ,et al. Dropout as data augmentation[J]. arXiv preprint, 2015,arXiv:1506.08700.

Metrics

Recommended 0

No Suggested Reading articles found!

层	输出格式
conv2d_1	(None, 64, 64, 32)
activation_1	(None, 64, 64, 32)
batch_normalization_1	(None, 64, 64, 32)
max_pooling2d_1	(None, 32, 32, 32)
conv2d_2	(None, 32, 32, 64)
activation_2	(None, 32, 32, 64)
batch_normalization_2	(None, 32, 32, 64)
conv2d_3	(None, 32, 32, 64)
activation_3	(None, 32, 32, 64)
batch_normalization_3	(None, 32, 32, 64)
max_pooling2d_2	(None, 16, 16, 64)
conv2d_4	(None, 16, 16, 128)
activation_4	(None, 16, 16, 128)
batch_normalization_4	(None, 16, 16, 128)
conv2d_5	(None, 16, 16, 128)
activation_5	(None, 16, 16, 128)
batch_normalization_5	(None, 16, 16, 128)
conv2d_6	(None, 16, 16, 128)
activation_6	(None, 16, 16, 128)
batch_normalization_6	(None, 16, 16, 128)
max_pooling2d_3	(None, 8, 8, 128)
flatten_1	(None, 8 192)
dense_1	(None, 512)
activation_7	(None, 512)
batch_normalization_7	(None, 512)
dropout_1	(None, 512)
dense_2	(None, 4)
activation_8	(None, 4)

超参数	设置
初始化学习率	0.01
训练总轮数（epoch）	30
每轮训练次数	128
批次大小	32
激活函数	ReLU
dropout	0.6

网络	不使用骨骼特征识别时间/s	使用骨骼特征识别时间/s
AlexNet	2.31	1.25
ResNet	2.57	1.56

Research on three frame difference gesture recognition method based on mixed bone features

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 21

Related Articles 13

Metrics

Recommended 0

[1]	Xinhuan LUO, Yixuan WANG, Wei LI, Xi CHEN. Application of Fit CutMix data augmentation algorithm based on saliency information in medical images [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 58-68.
[2]	Zhouyu GU, Yuecheng YU, Tiantian Zhe. Rapider-YOLOX: lightweight object detection network with high precision [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 92-103.
[3]	Jingwei LU, Xiang CHENG, Fei-Yue WANG. Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 461-476.
[4]	Xiaofeng CONG, Jie GUI, Jun ZHANG. Underwater image enhancement network based on visual Transformer with multiple loss functions fusion [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 522-532.
[5]	Yuanqi QIN, Qingyuan JI, Jun GE, Xingyuan DAI, Yuanyuan CHEN, Xiao WANG. Short-term traffic state reasoning and precise prediction in urban networks [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 380-395.
[6]	Hong SHAO, Mingkun ZHANG, Wencheng CUI. Classification method of dermoscopic image based on hierarchical convolution neural network [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 474-481.
[7]	Yang ZHANG, Yue HU, Dongrong XIN. A deep learning short-term traffic flow prediction method considering spatial-temporal association [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 172-178.
[8]	Hanbing GAO, Chi ZHANG, Mingyan JIN, Yang XIAO, Fengyu CONG. Cross-subject emotional EEG recognition based on multi-source domain adaptation [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 59-64.
[9]	Limin QUAN,Cuili YANG,Junfei QIAO. CFNN-based online control for dissolved oxygen concentration of wastewater treatment processes [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(3): 261-267.
[10]	Guobin ZHANG,Xinying WANG. Research on data-driven modeling for photovoltaic characteristics based on hybrid neural network [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(2): 169-178.
[11]	Xiaochen HAO,Yakun JI,Lizhao ZHENG,Xin SHI,Yantao ZHAO. Dynamic optimization algorithm of cement firing system based on differential evolution [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(2): 186-193.
[12]	Dongyuan LIN,Xiaofeng CHEN,Wentao SUN,Weikai LI,Yannan XIA. Stability analysis of commutative quaternion valued neural network with time varying delays [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(1): 80-87.
[13]	Peifeng NIU,Yunpeng MA,Xinxin ZHANG,Xiaobin HU. Research and application on combustion optimization of coal-fired boiler in thermal power plant based on artificial intelligence technology [J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(2): 163-170.