中文文本分类方法综述

doi:10.11959/j.issn.2096-109x.2019045

Abstract

CLC Number:

TP391

You YU, Yu FU, Xiaoping WU. Summary of text classification methods[J]. Chinese Journal of Network and Information Security, 2019, 5(5): 1-8.

Figures/Tables 2

References 32

[33]	李森, 马军, 赵嫣 ,等. 对数字化科技论文的自动分类研究[J]. 山东大学学报(理学版), 2006(3): 81-84.
	LI S , MA J , ZHAO Y ,et al. Automatic classification of digital science and technology papers[J]. Journal of Shandong University(Science Edition), 2006(3): 81-84.
[34]	CUI W , . A chinese text classification system based on naive bayes algorithm[C]// MATEC Web of Conferences. 2016:1015.
[35]	ZHANG M Y , AI X B , HU Y Z . Chinese text classification system on regulatory information based on SVM[C]// IOP Conference Series:Earth and Environmental Science. 2019:252.
[36]	SAHA D , . Web text classification using a neural network[C]// 2011 Second International Conference on, 2011.
[37]	雷飞 . 基于神经网络和决策树的文本分类及其应用研究[D]. 成都:电子科技大学, 2018.
	LEI F . Text research on text classification based on neural network and decision tree and its application[D]. Chengdu:University of Electronic Science and Technology of China, 2018.
[38]	周朴雄 . 基于神经网络集成的Web文档分类研究[J]. 图书情报工作, 2008,(7): 110-112.
	ZHOU P X . Study on Web document classification based on neural network integration[J]. Library and Information Service, 2008,(7): 110-112.
[39]	ZENG D J , LIU K , LAI S W,et.cl . relation classification via convolutional deep neural network[C]// The 25th international Computational Linguistics: 2014: 2335-2344.
[40]	CHEN Y W , WANG J L , CAI Y Q ,et al. A method for chinese text classification based on apparent semantics and latent aspects[J]. Journal of Ambient Intelligence and Humanized Computing, 2015,6(4): 473-480.
[41]	LI H M , HUANG H N , CAO X ,et al. Falcon:a novel chinese short text classification method[J]. Journal of Computer and Communications, 2018,6: 216-226.
[1]	郑捷 . NLP 汉语自然语言处理原理与实践[M]. 北京: 电子工业出版社, 2017.
	ZHENG J . NLP chinese natural language processing principle and practice[M]. Beijing: Publishing House of Electronics IndustryPress, 2017.
[42]	王根生, 黄学坚 . 基于Word2vec和改进型TF-IDF的卷积神经网络文本分类模型[J]. 小型微型计算机系统, 2019,40(5): 1120-1126.
	WANG G S , HUANG X J . Convolutional neural network text classification model based on Word2vec and improved TF-IDF[J]. Journal of Chinese Computer Systems, 2019,40(5): 1120-1126.
[2]	常建秋, 沈炜 . 基于字符串匹配的中文分词算法的研究[J]. 工业控制计算机, 2016,29(2): 115-116+119.
	CHANG J Q , SHEN W . Research on chinese word segmentation algorithm based on string matching[J]. Industrial control computer, 2016,29(2): 115-116+119.
[3]	邹佳伦, 文汉云, 王同喜 . 基于统计的中文分词算法研究[J]. 电脑知识与技术, 2019,15(4): 149-150+1５３.
	ZOU J L , WEN H Y , WANG T X . Research on chinese word segmentation algorithm based on statistics[J]. Computer Knowledge and Technology, 2019,15(4): 149-150＋153.
[4]	刘群, 张华平, 俞鸿魁 ,等. 基于层叠隐马模型的汉语词法分析[J]. 计算机研究与发展, 2004(8): 1421-1429.
	LIU Q , ZHANG H P , YU H K ,et al. Chinese lexical analysis model using cascading hidden horse model[J]. Journal of Computer Research and Development, 2004(8): 1421-1429.
[5]	孙茂松, 陈新雄, 张开旭 ,等. THULAC:一个高效的中文词法分析工具包[R]. 2016.
	SUN M S , CHEN X X , ZHANG K X et al . THULAC:an efficient Chinese lexical analysis toolkit[R]. 2016.
[43]	王磊 . 基于混合神经网络的中文短文本分类方法研究[D]. 杭州:浙江理工大学, 2019.
	WANG L . Research on chinese short text classification based on hybrid neural network[D]. Hangzhou:Zhejiang University of Science and Technology, 2019.
[6]	熊回香, 叶佳鑫 . 基于同义词词林的社会化标签等级结构构建研究[J]. 情报杂志, 2018,37(1): 126-131.
	XIONG H X , YE J X . Research on t Structure construction of social tags TongYiCi CiLin[J]. Journal of Information, 2018,37(1): 126-131.
[44]	JIN W Z , ZHU H , YANG G C . An efficient character-level and word-level feature fusion method for chinese text classification[C]// Journal of Physics:Conference Series. 2019:12057.
[45]	WANG P , XU B , XU J M ,et al. Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification[J]. Neurocomputing, 2016,174: 806-814.
[7]	XIE R , YUAN X , LIU Z ,et al. Lexical sememe prediction via word embeddings and Matrix factorization[C]// The 26th International Joint Conference on Artificial Intelligence. 2017: 4200-4206.
[8]	唐籍涛, 李飞, 郭昌松 . 网络舆情监控中新词识别问题的研究[J]. 计算机技术与发展, 2012,22(1): 119-121+125.
	TANG J T , LI F , GUO C S . Research on new word pattern recognition in network monitoring public opinion[J]. Computer Technology and Development, 2012,22(01): 119-121+125.
[9]	刘泽文, 丁冬, 李春文 . 基于条件随机场的中文短文本分词方法[J]. 清华大学学报(自然科学版), 2015,55(08): 906-910+915.
	LIU Z W , DING D , LI C W . A chinese short text word segmentation method based on conditional random fields[J]. Journal of Tsinghua University(Science and Technology), 2015,55(8): 906-910+915.
[10]	杜丽萍, 李晓戈, 于根 ,等. 基于互信息改进算法的新词发现对中文分词系统改进[J]. 北京大学学报(自然科学版), 2016,52(01): 35-40.
[46]	龚千健 . 基于循环神经网络模型的文本分类[D]. 武汉:华中科技大学, 2016.
	GONG Q J . Text Classification based on recurrent neural network model[D]. Wuhan:Huazhong University of Science and Technology, 2016.
[10]	DU L P , LI X G , YU G ,et al. New word detection based on an improved PMI algorithm for enhancing segmentation system[J]. Journal of Peking University(Natural Science), 2016,52(01): 35-40.
[11]	胡婕, 张俊驰 . 双向循环网络中文分词模型[J]. 小型微型计算机系统, 2017,38(3): 522-526.
	HU J , ZHANG J C . Bidirectional recurrent network for chinese word segmentation[J]. Journal of Chinese Computer Systems, 2017,38(3): 522-526.
[12]	ZHANG Y N , XU J N , MIAO G Y ,et al. Improving neural chinese word segmentation using unlabeled data[J]. IOP Conference Series:Materials Science and Engineering, 2018,435(1).
[13]	杨光豹, 杨丰赫, 毛贵军 . 基于分组 hash 与变长匹配的中文分词技术[J]. 计算机时代, 2019(4): 52-55.
	YANG G B , YANG F H , MAO G J . Chinese word segmentation technology based on group hash and variable length matching[J]. Computer Age, 2019(4): 52-55.
[14]	CHEN X C , SHI Z , QIU X P ,et al. Adversarial multi-criteria learning for chinese word segmentation[C]// ACL2017:Computation and Language. 2017.
[15]	YU C H , WANG S P , GUO J J . Learning chinese word segmentation based on bidirectional GRU-CRF and CNN network model[J]. International Journal of Technology and Human Interaction (IJTHI), 2019,15(3).
[16]	DARSHAN S L , JAIDHAR C D . Performance evaluation of filter-based feature selection techniques in classifying portable executable files[J]. Procedia Computer Science, 2018,125: 346-356.
[17]	HANCER E . Differential evolution for feature selection:a fuzzy wrapper–filter approach[J]. Soft Computing, 2019,23(13): 5233-5248.
[18]	MALDONADO S ， LóPEZ J . Dealing with high-dimensional class-imbalanced datasets:embedded feature selection for SVM classification[J]. Applied Soft Computing, 2018(67): 228-246.
[19]	刘赫, 刘大有, 裴志利 ,等. 一种基于特征重要度的文本分类特征加权方法[J]. 计算机研究与发展, 2009,46(10): 1693-1703.
	LIU H , LIU D Y , YAN Z L ,et al. A Feature weighting scheme for text categorization based on feature importance[J]. Journal of Computer Research and Development, 2009,46(10): 1693-1703.
[20]	靖红芳, 王斌, 杨雅辉 ,等. 基于类别分布的特征选择框架[J]. 计算机研究与发展, 2009,46(9): 1586-1593.
	JING H F , WANG B , YANG Y H ,et al. Category distribution-based feature selection framework[J]. Journal of Computer Research and Development, 2009,46(9): 1586-1593.
[21]	单丽莉, 刘秉权, 孙承杰 . 文本分类中特征选择方法的比较与改进[J]. 哈尔滨工业大学学报, 2011,43(S1): 319-324.
	SHAN L L , LIU B Q , SUN C J . Comparison and improvement of feature selection methods in text categorization[J]. Journal of Harbin Institute of Technology, 2011,43(S1): 319-324.
[22]	王正琦, 冯晓兵, 张驰 . 基于两层分类器的恶意网页快速检测系统研究[J]. 网络与信息安全学报, 2017,3(8): 48-64.
	WANG Z Q , FENG X B , ZHANG C . Study of high-speed malicious web pages detection system based on two-type classifier[J]. Journal of Network and Information Security, 2017,3(8): 48-64.
[23]	蒋胜利 . 高维数据的特征选择与特征提取研究[D]. 西安:西安电子科技大学, 2011.
	JIANG S L . Research on feature selection and feature extraction of high dimensional data[D]. Xi’an:Xidian University, 2011.
[24]	李炜, 巢秀琴 . 改进的粒子群算法优化的特征选择方法[J]. 计算机科学与探索, 2019,(6): 990-1004.
	LI W , CAI X Q . Improved particle swarm optimization method for feature[J]. Journal of Frontiers of Computer Science and Technology, 2019,(6): 990-1004.
[25]	路永和, 曹利朝 . 基于粒子群优化的文本特征选择方法[J]. 现代图书情报技术, 2011,(Z1): 76-81.
	LU Y H , CAO L C . Text feature selection method based on particle swarm optimization[J]. New Technology of Library and Information Service, 2011,(Z1): 76-81.
[26]	张杰慧, 何中市, 王健 ,等. 基于自适应蚁群算法的组合式特征选择算法[J]. 系统仿真学报, 2009,21(6): 1605-1608+1614.
	ZHANG J H , HE Z S , WANG J ,et al. Hybrid feature selection algorithm based on adaptive ant colony algorithm[J]. Journal of System Simulation, 2009,21(6): 1605-1608+1614.
[27]	张海涛 . 基于文本降维和蚁群算法的文本聚类研究[D]. 合肥:安徽大学, 2016.
	ZHANG H T . Research on text clustering based on text dimension reduction and ant colony algorithm[D]. Hefei:Anhui University, 2016.
[28]	卢泓宇, 张敏, 刘奕群 ,等. 卷积神经网络特征重要性分析及增强特征选择模型[J]. 软件学报, 2017,28(11): 2879-2890.
	LU Y Y , ZHANG M , LIU Y Q ,et al. Convolutional neural networks importance analysis and feature selection enhanced model[J]. Journal of Software, 2017,28(11): 2879-2890.
[29]	WU Y , WU W , XING C ,et al. Sequential matching network:a new architecture for multi-turn response selection in retrieval-based chatbots[C]// ACL 2017:Computation and Language. 2017.
[30]	ZHAO Z , LIU T , LI S ,et al. Ngram2vec:learning improved word representations from ngram co-occurrence statistics[C]// Conference on Empirical Methods in Natural Language Processing. 2017: 244-253.
[31]	赵芃 . 基于主题模型与深度学习的短文本特征扩展与分类研究[D]. 天津:天津工业大学, 2018.
	ZHAO P . Research on short text feature expansion and classification based on topic model and deep learning[D]. Tianjin:Tianjin Polytechnic University, 2018.
[32]	孙锐, 郭晟, 姬东鸿 . 融入事件知识的主题表示方法[J]. 计算机学报, 2017,40(4): 791-804.
	SUN R , GUO W , JI D H . Topic representation integrated method of integrating with event knowledge[J]. Chinese Journal of Computers, 2017,40(4): 791-804.

Metrics

Recommended 0

No Suggested Reading articles found!

算法	优点	缺点
朴素贝叶斯算法	算法简单，分类效果稳定；适用于小规模数据的训练；所需估算的参数少，对缺失数据不敏感	算法前提假设属性之间相互独立，而实际中往往难以成立；属性多或者属性之间相关性较大时，分类效果不好；分类效果依赖于先验概率；对输入数据的表达形式很敏感
支持向量机算法	可用于小样本数据学习；具有较高的泛化能力；可用于高维数据的计算；可以解决非线性问题；可以避免神经网络结构选择和局部极小点问题	对缺失数据敏感；对非线性问题没有通用解决方案
神经网络算法	并行处理能力强；学习能力强、分类准确度高；对数据噪声有较强的顽健性和容错能力；能解决复杂的非线性关系，具有记忆的功能	神经网络训练过程中有大量的参数需要确定；不能观察网络之间的学习过程，输出结果难以解释；学习时间长，且效果不可保证
决策树算法	易于理解，逻辑表达式生成较简单；数据预处理要求低；能够处理不相关的特征；可通过静态测试对模型进行评测；能够短的时间内对大规模数据进行处理；能同时处理数据型和常规型属性，可构造多属性决策树	易倾向于具有更多数值的特征；处理缺失数据存在困难；易出现过拟合；易忽略数据集属性的相关性
模糊决策算法	顽健性强；可解决非线性、强耦合等问题；容错能力强；通过模糊选择，结果更加精确	不适用于简单信息的处理；无法精确定义目标

Summary of text classification methods

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 2

References 32

Related Articles 8

Metrics

Recommended 0

[1]	Long DAI, Jing ZHANG, Xuefeng FAN, Xiaoyi ZHOU. NLP neural network copyright protection based on black box watermark [J]. Chinese Journal of Network and Information Security, 2023, 9(1): 140-149.
[2]	Cong YI, Jun HU. Novel continuous identity authentication method based on mouse behavior [J]. Chinese Journal of Network and Information Security, 2022, 8(5): 179-188.
[3]	Yu ZHANG, Binglong LI, Xuejuan LI, Heyu ZHANG. Evidence classification method of chat text based on DSR and BGRU model [J]. Chinese Journal of Network and Information Security, 2022, 8(2): 150-159.
[4]	Xi FU,Hui LI,Xingwen ZHAO. Survey on phishing detection research [J]. Chinese Journal of Network and Information Security, 2020, 6(5): 1-10.
[5]	Fan CHAO,Zhi YANG,Xuehui DU,Yan SUN. Android malware detection method based on deep neural network [J]. Chinese Journal of Network and Information Security, 2020, 6(5): 67-79.
[6]	Sijia DU,Haining YU,Hongli ZHANG. Survey of text classification methods based on deep learning [J]. Chinese Journal of Network and Information Security, 2020, 6(4): 1-13.
[7]	Hui XIAO,Bin WENG,Tianqiang HUANG,Han PU,Zehui HUANG. Video inter-frame tampering detection algorithm fusing multiple features [J]. Chinese Journal of Network and Information Security, 2020, 6(1): 84-93.
[8]	Tie-ming CHEN,Xiao-hao WANG,Wei-wei PANG,Jie JIANG. Time series and semantics-based chinese microblog topic detection and tracking method [J]. Chinese Journal of Network and Information Security, 2016, 2(5): 21-29.