基于Highway-BiLSTM网络的汉语谓语中心词识别研究

doi:10.11959/j.issn.1000-436x.2021027

通信学报 ›› 2021, Vol. 42 ›› Issue (1): 100-107.doi: 10.11959/j.issn.1000-436x.2021027

基于Highway-BiLSTM网络的汉语谓语中心词识别研究

黄瑞章¹^,², 靳文繁¹, 陈艳平¹^,², 秦永彬¹^,², 郑庆华³

¹ 贵州大学计算机科学与技术学院，贵州贵阳 550025
² 贵州省公共大数据重点实验室，贵州贵阳 550025
³ 西安交通大学计算机科学与技术学院，陕西西安 710049

修回日期:2020-08-10 出版日期:2021-01-25 发布日期:2021-01-01
作者简介:黄瑞章（1979-），女，天津人，博士，贵州大学副教授，主要研究方向为数据融合分析、文本挖掘、网络挖掘和知识发现。
靳文繁（1996- ），男，贵州毕节人，贵州大学硕士生，主要研究方向为自然语言处理、信息抽取。
陈艳平（1980- ），男，贵州长顺人，博士，贵州大学副教授，主要研究方向为人工智能、自然语言处理。
秦永彬（1980- ），男，山东招远人，博士，贵州大学教授，主要研究方向为大数据治理与应用、多源数据融合。
郑庆华（1969- ），男，浙江嵊州人，博士，西安交通大学教授，主要研究方向为大数据知识工程、网络舆情监测。
基金资助:
国家自然科学基金资助项目(U1836205);国家自然科学基金资助项目(91746116);贵州省科学技术基金重点资助项目([2020]1Z055)

Research on Chinese predicate head recognition based on Highway-BiLSTM network

Ruizhang HUANG¹^,², Wenfan JIN¹, Yanping CHEN¹^,², Yongbin QIN¹^,², Qinghua ZHENG³

¹ College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
² Guizhou Provincial Key Laboratory of Public Big Data, Guiyang 550025, China
³ College of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China

Revised:2020-08-10 Online:2021-01-25 Published:2021-01-01
Supported by:
The National Natural Science Foundation of China(U1836205);The National Natural Science Foundation of China(91746116);The Key Project of Science and Technology of Guizhou([2020]1Z055)

摘要/Abstract

摘要：

针对汉语谓语中心词识别困难及唯一性的问题，提出了一种基于Highway-BiLSTM网络的深度学习模型。首先，通过多层 BiLSTM 网络叠加获取句子内部不同粒度抽象语义信息的直接依赖关系；然后，利用 Highway网络缓解深层模型出现的梯度消失问题；最后，通过约束层对输出路径进行规划，解决谓语中心词的唯一性问题。实验结果表明，该方法有效提升了谓语中心词识别的性能。

关键词: 谓语中心词, 高速公路连接, 双向长短期记忆网络, 唯一性

Abstract:

Aiming at the problem of difficult recognition and uniqueness of Chinese predicate head, a Highway-BiLSTM model was proposed.Firstly, multi-layer BiLSTM networks were used to capture multi-granular semantic dependence in a sentence.Then, a Highway network was adopted to alleviate the problem of gradient disappearance.Finally, the output path was optimized by a constraint layer which was designed to guarantee the uniqueness of predicate head.The experimental results show that the proposed method effectively improves the performance of predicate head recognition.

Key words: predicate head, highway connection, BiLSTM, uniqueness

中图分类号:

TN391

黄瑞章, 靳文繁, 陈艳平, 秦永彬, 郑庆华. 基于Highway-BiLSTM网络的汉语谓语中心词识别研究[J]. 通信学报, 2021, 42(1): 100-107.

Ruizhang HUANG, Wenfan JIN, Yanping CHEN, Yongbin QIN, Qinghua ZHENG. Research on Chinese predicate head recognition based on Highway-BiLSTM network[J]. Journal on Communications, 2021, 42(1): 100-107.

图/表 11

图1

图2

表1

表2

实验结果"

模型	准确率	召回率	F值
CRF	75.26%	58.12%	65.59%
BiLSTM+CRF	76.05%	71.45%	73.68%
BiLSTM+Attention+CRF	76.60%	74.48%	76.58%
Residual + BiLSTM + Softmax	80.135%	78.517%	79.318%
Highway + BiLSTM + Softmax	$80 . 757 %$	$80 . 095 %$	$80 . 424 %$

表2

表3

表4

图3

表5

图4

表6

图5

参考文献 29

[1]	LUO Z S , ZHENG B X . An approach to the automatic analysis and frequence statistics of Chinese sentence patterns[J]. Journal of Chinese Information Processing, 1994,8(2): 1-9.
[2]	LI G C , MENG J . A method of identifying the predicate head based on the correspondence between the subject and the predicate[J]. Journal of Chinese Information Processing, 2005,19(1): 1-7.
[3]	SUI Z F , YU S W . The research on recognizing the predicate head of a Chinese simple sentence in EBMT[J]. Journal of Chinese Information Processing, 1998,12(4): 39-46.
[4]	SUI Z F , YU S W . The acquisition and application of the knowledge for recognizing the predicate head of a Chinese simple sentence[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 1998,34(2-3): 221-229.
[5]	陈小荷, 石定栩 . 汉语句子的主题-主语标注[C]// 全国计算机语言学联合学术会议. 北京:中国计算机学会, 1997: 102-108.
	CHEN X H , SHI D X . Topic-subject labeling of Chinese sentences[C]// National Computer Linguistics Joint Academic Conference. Beijing:China Computer Federation, 1997: 102-108.
[6]	WANG H L , ZHOU G D . Feature engineering for predicate identification and classification in semantic analysis[J]. Computer Engineering and Application, 2010,46(9): 134-137.
[7]	谌志群 . 汉语句子谓语识别的自动识别方法研究[J]. 计算机工程与应用, 2007,43(17): 176-178.
	CHEN Z Q . Study on recognizing predicate of Chinese sentences[J]. Engineering and Applications, 2007,43(17): 176-178.
[8]	GONG X J , LUO Z S , LUO W H . Recognizing the predicate head of Chinese sentences[J]. Journal of Chinese Information Processing, 2003,17(2): 7-13.
[9]	HAN L , LUO S L , PAN L M . High accuracy Chinese predicate recognition method combining lexical and syntactic feature[J]. Journal of Zhejiang University (Engineering Edition), 2014,48(12): 2107-2114.
[10]	李琳, 赵维纳, 泽旺宽卓 . 基于词向量特征的藏语谓语动词短语识别模型[J]. 电子技术与软件工程, 2019(4): 242-243.
	LI L , ZHAO W N , ZEWANG K Z . Tibetan predicate verb phrase recognition model based on word vector features[J]. Electronic Technology ＆ Software Engineering, 2019(4): 242-243.
[11]	RAVINER L , JUANG B . An introduction to hidden Markov models[J]. IEEE ASSP Magazine, 1986,3(1): 4-16.
[12]	LAFFERTY J , MCCALLUM A , PEREIRA F C N . Conditional random fields:probabilistic models for segmenting and labeling sequence data[J]. Proceedings of the 18th International Conference on Machine Learning.New York:ACM Press, 2001: 282-289.
[13]	HAMMERTON J , . Named entity recognition with long-short term memory[C]// Proceedings of the 17th Conference on Natural Language Learning at HLT-NAACL. New York:ACM Press, 2003: 172-175.
[14]	LI F , ZHANG M , TIAN B ,et al. Recognizing irregular entities in biomedical text via deep neural networks[J]. Pattern Recognition Letters, 2018,105: 105-113.
[15]	CARRERAS X , MARQUEZ L . Introduction to the CoNLL-2005 shared task:semantic role labeling[C]// Proceedings of the 19th Conference on Computational Natural Language Learning. Boston:Association for Computational Linguistics, 2005: 152-164.
[16]	KOOMEN P , PUNYAKANOK V , ROTH D ,et al. Generalized inference with multiple semantic role labeling systems[C]// Proceedings of the 9th Conference on Computational Natural Language Learning. Boston:Association for Computational Linguistics, 2005: 181-184.
[17]	TACKSTROM O , GANCHEV K , DAS D . Efficient inference and structured learning for semantic role labeling[J]. Transactions of the Association for Computational Linguistics, 2015,3: 29-41.
[18]	ZHOU J , XU W . End-to-end learning of semantic role labeling using recurrent neural networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). Boston:Association for Computational Linguistics, 2015: 1127-1137.
[19]	GUO J , CHE W , WANG H ,et al. A unified architecture for semantic role labeling and relation classification[C]// Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.[S.n.:s.l.], 2016: 1264-1274.
[20]	王瑞波, 李济洪, 李国臣 ,等. 基于 Dropout 正则化的汉语框架语义角色识别[J]. 汉语信息学报, 2017,31(1): 147-154.
	WANG R B , LI J H , LI G C ,et al. Chinese frame semantic role recognition based on Dropout regularization[J]. Journal of Chinese Information Processing, 2017,31(1): 147-154.
[21]	STRUBELL E , VERGA P , ANDOR D ,et al. Linguistically-informed self-attention for semantic role labeling[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Boston:Association for Computational Linguistics, 2018: 5027-5038.
[22]	HE L , LEE K , LEWIS M ,et al. Deep semantic role labeling:what works and what’s next[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). Boston:Association for Computational Linguistics, 2017: 473-483.
[23]	PUNDAK G , SAINATH T N . Highway-LSTM and recurrent highway networks for speech recognition[C]// Interspeech,,[S.n.:s.l.], 2017.
[24]	ZHANG Y , CHEN G , YU D ,et al. Highway long short-term memory RNNs for distant speech recognition[C]// 2016 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2016: 5755-5759.
[25]	GAL Y , GHAHRAMAN Z . A theoretically grounded application of dropout in recurrent neural networks[J]. Advances in Neural Information Processing Systems, 2016,29: 1019-1027.
[26]	李婷, 秦永彬, 黄瑞章 ,等. 基于神经网络的汉语谓语动词识别研究[J]. 数据采集与处理, 2020,35(3): 582-590.
	LI T , QIN Y B , HUANG R Z ,et al. Research on Chinese predicate verb recognition based on neural network[J]. Journal of Data Acquisition ＆ Processing, 2020(3): 582-590.
[27]	SAXE A M , MCCLELLAND J L , GANGULI S . Exact solutions to the nonlinear dynamics of learning in deep linear neural networks[J]. arXiv Preprint,arXiv:1312.6120, 2013.
[28]	ZEILER M D . ADADELTA:an adaptive learning rate method[J]. arXiv Preprint,arXiv:1212.5701, 2012.
[29]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision ＆ Pattern Recognition. Los Alamitos:IEEE Computer Society, 2016: 770-778.

参数	设定值
词向量维度/维	100
上下文标记维度/维	128
LSTM隐含层维度/维	100
Batchsize	20
LSTM层数/层	8
学习率	0.001
Dropoutrate	0.2

句长/字	句子数/条	准确率	召回率	F值
1～6	1119	78.632%	84.404%	81.416%
7～14	3395	78.482%	75.573%	77.000%
15～25	2748	74.396%	71.628%	72.986%
≥26	2923	69.654%	73.704%	71.622%

实验方法	准确率	召回率	F值
预训练词向量	80.757%	80.095%	80.424%
随机初始化	79.790%	79.085%	79.436%

实验方法	准确率	召回率	F值
无连接	77.819%	78.580%	78.198%
残差连接	80.135%	78.517%	79.318%
Highway连接	80.757%	80.095%	80.424%

层数/层	准确率	召回率	F值
4	80.757%	80.095%	80.424%
2	80.743%	77.508%	79.092%

基于Highway-BiLSTM网络的汉语谓语中心词识别研究

Research on Chinese predicate head recognition based on Highway-BiLSTM network

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 29

相关文章 1

Metrics

推荐阅读 0