基于Highway-BiLSTM网络的汉语谓语中心词识别研究

doi:10.11959/j.issn.1000-436x.2021027

Abstract

Abstract:

Aiming at the problem of difficult recognition and uniqueness of Chinese predicate head, a Highway-BiLSTM model was proposed.Firstly, multi-layer BiLSTM networks were used to capture multi-granular semantic dependence in a sentence.Then, a Highway network was adopted to alleviate the problem of gradient disappearance.Finally, the output path was optimized by a constraint layer which was designed to guarantee the uniqueness of predicate head.The experimental results show that the proposed method effectively improves the performance of predicate head recognition.

Key words: predicate head, highway connection, BiLSTM, uniqueness

CLC Number:

TN391

Ruizhang HUANG, Wenfan JIN, Yanping CHEN, Yongbin QIN, Qinghua ZHENG. Research on Chinese predicate head recognition based on Highway-BiLSTM network[J]. Journal on Communications, 2021, 42(1): 100-107.

Figures/Tables 11

模型	准确率	召回率	F值
CRF	75.26%	58.12%	65.59%
BiLSTM+CRF	76.05%	71.45%	73.68%
BiLSTM+Attention+CRF	76.60%	74.48%	76.58%
Residual + BiLSTM + Softmax	80.135%	78.517%	79.318%
Highway + BiLSTM + Softmax	$80 . 757 %$	$80 . 095 %$	$80 . 424 %$

References 29

[1]	LUO Z S , ZHENG B X . An approach to the automatic analysis and frequence statistics of Chinese sentence patterns[J]. Journal of Chinese Information Processing, 1994,8(2): 1-9.
[2]	LI G C , MENG J . A method of identifying the predicate head based on the correspondence between the subject and the predicate[J]. Journal of Chinese Information Processing, 2005,19(1): 1-7.
[3]	SUI Z F , YU S W . The research on recognizing the predicate head of a Chinese simple sentence in EBMT[J]. Journal of Chinese Information Processing, 1998,12(4): 39-46.
[4]	SUI Z F , YU S W . The acquisition and application of the knowledge for recognizing the predicate head of a Chinese simple sentence[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 1998,34(2-3): 221-229.
[5]	陈小荷, 石定栩 . 汉语句子的主题-主语标注[C]// 全国计算机语言学联合学术会议. 北京:中国计算机学会, 1997: 102-108.
	CHEN X H , SHI D X . Topic-subject labeling of Chinese sentences[C]// National Computer Linguistics Joint Academic Conference. Beijing:China Computer Federation, 1997: 102-108.
[6]	WANG H L , ZHOU G D . Feature engineering for predicate identification and classification in semantic analysis[J]. Computer Engineering and Application, 2010,46(9): 134-137.
[7]	谌志群 . 汉语句子谓语识别的自动识别方法研究[J]. 计算机工程与应用, 2007,43(17): 176-178.
	CHEN Z Q . Study on recognizing predicate of Chinese sentences[J]. Engineering and Applications, 2007,43(17): 176-178.
[8]	GONG X J , LUO Z S , LUO W H . Recognizing the predicate head of Chinese sentences[J]. Journal of Chinese Information Processing, 2003,17(2): 7-13.
[9]	HAN L , LUO S L , PAN L M . High accuracy Chinese predicate recognition method combining lexical and syntactic feature[J]. Journal of Zhejiang University (Engineering Edition), 2014,48(12): 2107-2114.
[10]	李琳, 赵维纳, 泽旺宽卓 . 基于词向量特征的藏语谓语动词短语识别模型[J]. 电子技术与软件工程, 2019(4): 242-243.
	LI L , ZHAO W N , ZEWANG K Z . Tibetan predicate verb phrase recognition model based on word vector features[J]. Electronic Technology ＆ Software Engineering, 2019(4): 242-243.
[11]	RAVINER L , JUANG B . An introduction to hidden Markov models[J]. IEEE ASSP Magazine, 1986,3(1): 4-16.
[12]	LAFFERTY J , MCCALLUM A , PEREIRA F C N . Conditional random fields:probabilistic models for segmenting and labeling sequence data[J]. Proceedings of the 18th International Conference on Machine Learning.New York:ACM Press, 2001: 282-289.
[13]	HAMMERTON J , . Named entity recognition with long-short term memory[C]// Proceedings of the 17th Conference on Natural Language Learning at HLT-NAACL. New York:ACM Press, 2003: 172-175.
[14]	LI F , ZHANG M , TIAN B ,et al. Recognizing irregular entities in biomedical text via deep neural networks[J]. Pattern Recognition Letters, 2018,105: 105-113.
[15]	CARRERAS X , MARQUEZ L . Introduction to the CoNLL-2005 shared task:semantic role labeling[C]// Proceedings of the 19th Conference on Computational Natural Language Learning. Boston:Association for Computational Linguistics, 2005: 152-164.
[16]	KOOMEN P , PUNYAKANOK V , ROTH D ,et al. Generalized inference with multiple semantic role labeling systems[C]// Proceedings of the 9th Conference on Computational Natural Language Learning. Boston:Association for Computational Linguistics, 2005: 181-184.
[17]	TACKSTROM O , GANCHEV K , DAS D . Efficient inference and structured learning for semantic role labeling[J]. Transactions of the Association for Computational Linguistics, 2015,3: 29-41.
[18]	ZHOU J , XU W . End-to-end learning of semantic role labeling using recurrent neural networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). Boston:Association for Computational Linguistics, 2015: 1127-1137.
[19]	GUO J , CHE W , WANG H ,et al. A unified architecture for semantic role labeling and relation classification[C]// Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.[S.n.:s.l.], 2016: 1264-1274.
[20]	王瑞波, 李济洪, 李国臣 ,等. 基于 Dropout 正则化的汉语框架语义角色识别[J]. 汉语信息学报, 2017,31(1): 147-154.
	WANG R B , LI J H , LI G C ,et al. Chinese frame semantic role recognition based on Dropout regularization[J]. Journal of Chinese Information Processing, 2017,31(1): 147-154.
[21]	STRUBELL E , VERGA P , ANDOR D ,et al. Linguistically-informed self-attention for semantic role labeling[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Boston:Association for Computational Linguistics, 2018: 5027-5038.
[22]	HE L , LEE K , LEWIS M ,et al. Deep semantic role labeling:what works and what’s next[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). Boston:Association for Computational Linguistics, 2017: 473-483.
[23]	PUNDAK G , SAINATH T N . Highway-LSTM and recurrent highway networks for speech recognition[C]// Interspeech,,[S.n.:s.l.], 2017.
[24]	ZHANG Y , CHEN G , YU D ,et al. Highway long short-term memory RNNs for distant speech recognition[C]// 2016 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2016: 5755-5759.
[25]	GAL Y , GHAHRAMAN Z . A theoretically grounded application of dropout in recurrent neural networks[J]. Advances in Neural Information Processing Systems, 2016,29: 1019-1027.
[26]	李婷, 秦永彬, 黄瑞章 ,等. 基于神经网络的汉语谓语动词识别研究[J]. 数据采集与处理, 2020,35(3): 582-590.
	LI T , QIN Y B , HUANG R Z ,et al. Research on Chinese predicate verb recognition based on neural network[J]. Journal of Data Acquisition ＆ Processing, 2020(3): 582-590.
[27]	SAXE A M , MCCLELLAND J L , GANGULI S . Exact solutions to the nonlinear dynamics of learning in deep linear neural networks[J]. arXiv Preprint,arXiv:1312.6120, 2013.
[28]	ZEILER M D . ADADELTA:an adaptive learning rate method[J]. arXiv Preprint,arXiv:1212.5701, 2012.
[29]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision ＆ Pattern Recognition. Los Alamitos:IEEE Computer Society, 2016: 770-778.

Metrics

Recommended 0

No Suggested Reading articles found!

参数	设定值
词向量维度/维	100
上下文标记维度/维	128
LSTM隐含层维度/维	100
Batchsize	20
LSTM层数/层	8
学习率	0.001
Dropoutrate	0.2

句长/字	句子数/条	准确率	召回率	F值
1～6	1119	78.632%	84.404%	81.416%
7～14	3395	78.482%	75.573%	77.000%
15～25	2748	74.396%	71.628%	72.986%
≥26	2923	69.654%	73.704%	71.622%

实验方法	准确率	召回率	F值
预训练词向量	80.757%	80.095%	80.424%
随机初始化	79.790%	79.085%	79.436%

实验方法	准确率	召回率	F值
无连接	77.819%	78.580%	78.198%
残差连接	80.135%	78.517%	79.318%
Highway连接	80.757%	80.095%	80.424%

层数/层	准确率	召回率	F值
4	80.757%	80.095%	80.424%
2	80.743%	77.508%	79.092%

Research on Chinese predicate head recognition based on Highway-BiLSTM network

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 29

Related Articles 1

Metrics

Recommended 0