大数据下的基于深度神经网的相似汉字识别

doi:10.3969/j.issn.1000-436x.2014.09.019

摘要/Abstract

摘要：

针对传统相似手写汉字识别系统(SHCCR)受特征提取方法的限制，提出采用深度神经网(DNN)对相似汉字自动学习有效特征并进行识别，介绍相似字符集生成方法和针对相似汉字识别的深度神经网络的具体结构，研究对比不同的训练数据规模对识别性能的影响。实验表明，DNN 能有效地进行特征学习，避免了人工设计特征的不足，与传统基于梯度特征的支持向量机（SVM）和最近邻分类器（1-NN）方法相比，识别率有较大的提高；且随着训练样本增加的同时，DNN 在提高识别性能上表现得更为优秀，大数据训练对提升深度神经网络的识别率作用明显。

关键词: 大数据, 深度神经网, 深度学习, 相似手写汉字识别

Abstract:

The recognition rates of the traditional similar handwritten Chinese character recognition (SHCCR) systems are not very high due to the restriction of feature extraction methods.In order to improve the recognition accuracy,a new method based on deep neural networks (DNN) was proposed to learn effective features automatically and conduct recog-nition.The method of how to generate similar handwritten Chinese character sets was introduced.The architecture of the DNN for SHCCR was presented.The performances with respect to different training data scale was compared.The ex-perimental results show that,DNN can learn features automatically and efficiently.The proposed DNN can achieve better performance comparing with support vector machine (SVM) and nearest neighbor classifier (1-NN) based on gradient features.Especially,with the increase of training data the recognition rate of DNN is improved observably,indicating that large training data is crucial for the performance of DNN.

Key words: big data, deep neural networks, deep learning, similar handwritten Chinese characters recognition

杨钊,陶大鹏,张树业,金连文. 大数据下的基于深度神经网的相似汉字识别[J]. 通信学报, 2014, 35(9): 184-189.

Zhao YANG,Da-peng TAO,Shu-ye ZHANG,Lian-wen JIN. Similar handwritten Chinese character recognition based on deep neural networks with big data[J]. Journal on Communications, 2014, 35(9): 184-189.

图/表 6

图1

表1

图2

表2

表3

图3

参考文献 25

[1]	LIU C L , YIN F , WANG Q F , et al. ICDAR 2011 Chinese handwriting recognition competition[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Beijing,China, 2011.1464-1469.
[2]	LEUNG K C , LEUNG C H . Recognition of handwritten Chinese characters by critical region analysis[J]. Pattern Recognition, 2010,43(3):949-961.
[3]	GAO T F , LIU C L . High accuracy handwritten Chinese character recognition using LDA-based compound distances[J]. Pattern Recognition, 2008,41(11):3442-3451.
[4]	TAO D P , LIANG L Y , JIN L W , et al. Similar handwritten Chinese character recognition by kernel discriminative locality alignment[J]. Pattern Recognition Letters, 2014,35(1):186-194.
[5]	BENGIO Y . Learning deep architectures for AI[J]. Foundations and trends in Machine Learning, 2009,2(1):1-127.
[6]	BENGIO Y , COURVILLE A . Deep Learning of Representations[M]. Handbook on Neural Information Processing. Springer Berlin Heidelberg, 2013.
[7]	DENG L , HINTON G , KINGSBURY B . New types of deep neural network learning for speech recognition and related applications: an overview[A]. Proceedings of IEEE International Conference on Acoustics,Speech,and Signal Processing[C]. Vancouver,Canada, 2013.
[8]	HINTON G E , SALAKHUTDINOV R R . Reducing the dimensionality of data with neural networks[J]. Science, 2006,313(5786):504-507.
[9]	LECUN Y , BOSERB , DENKER J S , et al. Handwritten digit recogni-tion with a back-propagation network[A]. Advances in neural informa-tion processing systems[C]. Denver,United States, 1990.396-404.
[10]	LECUN Y , KAVUKCUOGLU K , FARABET C . Convolutional net-works and applications in vision[A]. Proceedings of IEEE Interna-tional Symposium onCircuits and Systems[C]. Paris,France, 2010.253-256.
[11]	LECUN Y , BOTTOU L , BENGIO Y , et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11):2278-2324.
[12]	DENG H , STATHOPOULOS G , SUEN C Y . Error-correcting output coding for the convolutional neural network for optical character recognition[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Barcelona,Spain, 2009.581-585.
[13]	YUAN A , BAI G , JIAO L , et al. Offline handwritten English character recognition based on convolutional neural network[A]. Proceedings of IAPR International Workshop on Document Analysis Systems[C]. Gold Cost,QLD, 2012.125-129.
[14]	NETZER Y , WANG T , COATES A , et al. Reading digits in natural images with unsupervised feature learning[A]. NIPS Workshop on Deep Learning and Unsupervised Feature Learning[C]. Granada,Spain, 2011.
[15]	SERMANET P , CHINTALA S , LECUN Y . Convolutional neural networks applied to house numbers digit classification[A]. Proceedings of IEEE International Conference on Pattern Recognition[C]. Tsukuba,Japan, 2012.3288-3291.
[16]	COATES A , CARPENTER B , CASE C , et al. Text detection and character recognition in scene images with unsupervised feature learning[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Beijing,China, 2011.440-445.
[17]	WANG T , WU D J , COATES A , et al. End-to-end text recognition with convolutional neural networks[A]. Proceedings of IEEE Interna-tional Conference on Pattern Recognition[C]. Tsukuba,Japan, 2012.3304-3308.
[18]	SIMARD P , STEINKRAUS D , PLATT J C . Best practices for convolutional neural networks applied to visual document analysis[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Edinburgh,UK, 2003.958-963.
[19]	CIRESAN D C , MEIER U , GAMBARDELLA L M , et al. Convolutional neural network committees for handwritten character classification[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Beijing,China, 2011.1135-1139.
[20]	VAQUERO L M , CACERES J , MORAN D . The challenge of service level scalability for the cloud[J]. International Journal of Cloud Applications and Computing (IJCAC), 2011,1(1):34-44.
[21]	GAO Y , JIN L W , HE C , et al. Handwriting character recognition as a service: a new handwriting recognition system based on cloud com-puting[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Beijing,China, 2011.885-889.
[22]	LIU Z B , JIN L W . A static candidates generation technique and its application in two-stage LDA Chinese character recognition[A]. Proceedings of Chinese Control Conference[C]. Hunan,China, 2007.571-575.
[23]	BOUVRIE J . Notes on Convolution Neural networks[R]. MIT CBCL, 2006.
[24]	BAI Z L , HUO Q . A study on the use of 8-directional features for online handwritten Chinese character recognition[A]. Proceedings of IEEE International Conference on Document Analysis and Recognition[C]. Seoul,Korea, 2005.262-266.
[25]	FAN R E , CHANG K W , HSIEH C J , et al. LIBLINEAR: a library for large linear classification[J]. The Journal of Machine Learning Research, 2008,9:1871-1874.

工整	G1：大夫买去失天头长支走	G2：古去生士太友右在支左
工整	10.3969/j.issn.1000-436x.2014.09.019.F003	10.3969/j.issn.1000-436x.2014.09.019.F004
随意无约束	G3：春番高看青首香言音着	G4：夫关交久卡买欠失天头
随意无约束	10.3969/j.issn.1000-436x.2014.09.019.F005	10.3969/j.issn.1000-436x.2014.09.019.F006

样本(Tr-Ts)	方法	G1	G2	G3	G4	平均	错误率减少
	1-NN	3.60	1.32	4.10	10.24	4.82	75%
1 000～200	SVM	0.94	0.40	1.14	2.64	1.28	7%
	CNN	0.70	0.29	0.87	2.69	1.19	—
	1-NN	2.65	1.07	3.45	7.66	3.71	84%
3 000～1 000	SVM	0.78	0.30	0.90	2.02	1.00	39%
	CNN	0.32	0.18	0.50	1.43	0.61	—
	1-NN	2.16	0.67	2.68	6.73	3.06	85%
6 000～1 000	SVM	0.64	0.28	0.75	1.95	0.91	49%
	CNN	0.19	0.14	0.31	1.18	0.46	—
	1-NN	2.00	0.55	2.54	6.10	2.80	86%
9 000～1 000	SVM	0.60	0.30	0.74	1.92	0.89	57%
	CNN	0.14	0.12	0.26	1.02	0.38	—

样本(Tr-Ts)	方法	时间/s	存储量(参数个数)
	1-NN	10.32	—
1 000～200	SVM	0.06	5120
	CNN	0.68	3168
	1-NN	29.71	—
3 000～200	SVM	0.06	5120
	CNN	0.71	3168