基于预训练机制的自修正复杂语义分析方法

doi:10.11959/j.issn.1000-436x.2019195

Abstract

Abstract:

In the process of knowledge service,in order to meet the fragmentation management needs of intellectualization,knowledge ability,refinement and reorganization content resources.Through deep analysis and mining of semantic hidden knowledge,technology,experience,and information,it broke through the existing bottleneck of traditional semantic parsing technology from Text-to-SQL.The PT-Sem2SQL based on the pre-training mechanism was proposed.The MT-DNN pre-training model mechanism combining Kullback-Leibler technology was designed to enhance the depth of context semantic understanding.A proprietary enhancement module was designed that captured the location of contextual semantic information within the sentence.Optimize the execution process of the generated model by the self-correcting method to solve the error output during decoding.The experimental results show that PT-Sem2SQL can effectively improve the parsing performance of complex semantics,and its accuracy is better than related work.

Key words: Text-to-SQL, semantic parsing, natural language processing, complex event processing

CLC Number:

TP302.1

Qing LI,Jiang ZHONG,Lili LI,Qi LI. Self-correcting complex semantic analysis method based on pre-training mechanism[J]. Journal on Communications, 2019, 40(12): 41-50.

Figures/Tables 10

References 38

[1]	CHOI E , HE H , IYYER M ,et al. QuAC:question answering in context[C]// The 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 2174-2184.
[2]	SUHR A , IYER S , ARTZI Y . Learning to map context-dependent sentences to executable formal queries[C]// The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long Papers). 2018: 2238-2249.
[3]	李青, 钟将, 李立力 ,等. 领域专业知识富关联关系提取方法研究[J]. 控制与决策, 2019: 1-9.
	LI Q , ZHONG J , LI L L ,et al. Research on the extraction method of multiple semantic relations in domain knowledge[J]. Control and De-cision, 2019: 1-9.
[4]	孙加东, 赵铁军, 梁华参 . 基于结构对齐的统计机器翻译模型[J]. 通信学报, 2009,30(7): 124-129.
	SUN J D , ZHAO T J , LIANG H S . Statistical machine translation model based on structure alignment[J]. Journal on Communications, 2009,30(7): 124-129.
[5]	LI Q , ZHONG J , TAO Y ,et al. Research of the processing technology for time complex event based on LSTM[J]. Cluster Computing, 2019,22: 9571-9579.
[6]	GUO Y , SHANG X , LI Z . Identification of cancer subtypes by integrating multiple types of transcriptomics data with deep learning in breast cancer[J]. Neurocomputing, 2019,324: 20-30.
[7]	YU T , ZHANG R , YANG K ,et al. Spider:a large-scale human-labeled dataset for complex and cross-domain semantic parsing and Text-to-SQL task[C]// The 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 3911-3921.
[8]	YU T , ZHANG R , YASUNAGA M ,et al. SParC:cross-domain semantic parsing in context[J]. arXiv Preprint,arXiv:1906.02285, 2019.
[9]	ZHONG V , XIONG C , SOCHER R . Seq2sql:generating structured queries from natural language using reinforcement learning[J]. arXiv Preprint,arXiv:1709.00103, 2017.
[10]	United States.Defense advanced research projects agency . speech and natural language:proceedings of a workshop held at Hidden Valley,Pennsylvania,June 24-27,1990[M]. Morgan Kaufmann Pub, 1990.
[11]	DAHL D A , BATES M , BROWN M ,et al. Expanding the scope of the ATIS task:The ATIS-3 corpus[C]// The Workshop on Human Language Technology. Association for Computational Linguistics, 1994: 43-48.
[12]	FINEGAN-DOLLAK C , KUMMERFELD J K , ZHANG L ,et al. Improving Text-to-SQL Evaluation Methodology[C]// The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). 2018: 351-360.
[13]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]// The 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers). 2019: 4171-4186.
[14]	LIU X , HE P , CHEN W ,et al. Multi-task deep neural networks for natural language understanding[J]. arXiv Preprint,arXiv:1901.11504, 2019.
[15]	SOLOMON J , CHARETTE F . Hierarchical multi-task deep neural network architecture for end-to-end driving[J]. arXiv Preprint,arXiv:1902.03466, 2019.
[16]	KULLBACK S . Letter to the editor:the Kullback-Leibler distance[J]. American Statistician, 1987,41(4): 340-341.
[17]	WARREN D H D , PEREIRA F C N . An efficient easily adaptable system for interpreting natural language queries[J]. Computational Linguistics, 1982,8(3-4): 110-122.
[18]	WONG Y W , MOONEY R . Learning synchronous grammars for semantic parsing with lambda calculus[C]// The 45th Annual Meeting of the Association of Computational Linguistics. 2007: 960-967.
[19]	ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming[C]// The National Conference on Artificial Intelligence. 1996: 1050-1055.
[20]	ZETTLEMOYER L S , COLLINS M . Learning to map sentences to logical form:structured classification with probabilistic categorial grammars[C]// The Twenty-First Conference on Uncertainty in Artificial Intelligence. AUAI Press, 2005: 658-666.
[21]	ZHAO K , HUANG L . Type-driven incremental semantic parsing with polymorphism[C]// The 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2015: 1416-1421.
[22]	HERZIG J , BERANT J . Decoupling structure and lexicon for zero-shot semantic parsing[C]// The 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 1619-1629.
[23]	SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks[C]// Advances in Neural Information Processing Systems. 2014: 3104-3112.
[24]	BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate[C]// International Conference on Learning Representations,abs/1409. 0473, 2014.
[25]	YU T , LI Z , ZHANG Z ,et al. TypeSQL:knowledge-based type-aware neural Text-to-SQL generation[C]// The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2 (Short Papers). 2018: 588-594.
[26]	DONG L , LAPATA M . Coarse-to-fine decoding for neural semantic parsing[C]// The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). 2018: 731-742.
[27]	WANG C , BROCKSCHMIDT M , SINGH R . Pointing out SQL queries from text[J]. 2018:
[28]	PENNINGTON J , SOCHER R , MANNING C . Glove:global vectors for word representation[C]// The 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014: 1532-1543.
[29]	HWANG W , YIM J , PARK S ,et al. A comprehensive exploration on WikiSQL with table-aware word contextualization[J]. arXiv Preprint,arXiv:1902.01069, 2019.
[30]	RADFORD A , NARASIMHAN K , SALIMANS T ,et al. Improving language understanding by generative pre-training[J]. American Statistician, 2018.
[31]	YU T , YASUNAGA M , YANG K ,et al. SyntaxSQLNet:Syntax tree networks for complex and cross-domain Text-to-SQL task[C]// The 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 1653-1663.
[32]	TANG L R , MOONEY R J . Using multiple clause constructors in inductive logic programming for semantic parsing[C]// European Conference on Machine Learning. Springer, 2001: 466-477.
[33]	POPESCU A M , ETZIONI O , KAUTZ H . Towards a theory of natural language interfaces to databases[C]// The 8th International Conference on Intelligent User Interfaces. ACM, 2003: 149-157.
[34]	ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming[C]// The National Conference on Artificial Intelligence. 1996: 1050-1055.
[35]	IYER S , KONSTAS I , CHEUNG A ,et al. Learning a neural semantic parser from user feedback[J]. arXiv Preprint,arXiv:1704.08760, 2017.
[36]	LI F , JAGADISH H V . Constructing an interactive natural language interface for relational databases[J]. Proceedings of the VLDB Endowment, 2014,8(1): 73-84.
[37]	YAGHMAZADEH N , WANG Y , DILLIG I ,et al. SQLizer:query synthesis from natural language[J]. Proceedings of the ACM on Programming Languages, 2017,1(OOPSLA):63.
[38]	SMITH S L , KINDERMANS P J , YING C ,et al. Don’t decay the learning rate,increase the batch size[J]. arXiv Preprint,arXiv:1711.00489, 2017,41(4): 340-341.

Metrics

Recommended 0

No Suggested Reading articles found!

数据集	CONTEXT	CROSS DOMAIN	QUESTION /个	TURN	QUESTION LENGTH	QUESTION VOCABULARY	SELECT	WHEREGROUP BYORDER BY	KEYWORDS
Spider	NO	YES	11 840	1.0	13.4	4 818	98.9%	55.2% 24.0% 21.5%	68.6%
SParC	YES	YES	12 726	3.0	8.1	3 794	97.4%	42.8% 20.1% 17.0%	41.2%

模型方法	测试集					验证集
模型方法	Easy	Medium	Hard	Extra Hard	All	All
seq2seq	11.9%	1.9%	1.3%	0.5%	3.7%	1.9%
SQLNet	26.2%	12.6%	6.6%	1.3%	12.4%	10.9%
TypeSQL	19.6%	7.6%	3.8%	0.8%	8.2%	8.0%
IncSQL	29.5%	13.1%	12.3%	2.5%	14.6%	13.2%
SyntaxSQLNet	48.0%	27.0%	24.3%	4.6%	27.2%	24.8%
SQLove	56.7%	32.9%	22.3%	4.3%	29.1%	27.9%
PT-Sem2SQL-SE	68.1%	40.2%	31.6%	7.3%	36.8%	33.7%
PT-Sem2SQL	73.2%	45.1%	36.2%	9.2%	40.9%	38.1%

模型方法	SELECT	WHERE	GROUP BY	ORDER BY	KEYWORDS
seq2seq	13.0%	1.5%	3.3%	5.3%	8.7%
SQLNet	44.5%	19.8%	29.5%	48.8%	64.0%
TypeSQL	36.4%	16.0%	17.2%	47.7%	66.2%
IncSQL	59.2%	41.9%	23.4%	52.7%	67.3%
SyntaxSQLNet	62.5%	34.8%	55.6%	60.9%	69.6%
SQLove	78.4%	47.8%	61.6%	66.2%	74.6%
PT-Sem2SQL-SE	81.3%	50.4%	68.0%	69.1%	77.4%
PT-Sem2SQL	89.2%	65.9%	77.1%	75.6%	86.1%

模型方法	测试集					验证集
模型方法	Easy	Medium	Hard	Extra Hard	All	All
SyntaxSQLNet	38.4%	8.7%	2.4%	1.3%	12.7%	11.9%
CD-Seq2Seq	36.1%	8.4%	3.7%	2.1%	12.6%	11.8%
SQLove	32.2%	6.9%	1.3%	0.5%	10.2%	8.9%
PT-Sem2SQL-SE	46.3%	23.8%	12.4%	7.3%	22.5%	19.7%
PT-Sem2SQL	53.1%	28.4%	15.5%	9.2%	26.6%	24.1%

模型方法	问题匹配度		上下文相关交互匹配度
模型方法	验证集	测试集	验证集	测试集
SyntaxSQLNet	18.5%	20.2%	4.3%	5.2%
CD-Seq2Seq	21.9%	23.2%	8.1%	7.5%
SQLove	20.3%	22.0%	6.9%	6.4%
PT-Sem2SQL-SE	29.7%	32.7%	13.1%	11.9%
PT-Sem2SQL	32.5%	34.1%	15.8%	13.4%

Self-correcting complex semantic analysis method based on pre-training mechanism

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 38

Related Articles 5

Metrics

Recommended 0

[1]	Wenjian ZHANG,Ke SONG,Libo TAN,Shuai WEI,Chunlei DONG. Programmable semantic parsing approach for mimic arbitration [J]. Journal on Communications, 2020, 41(4): 62-69.
[2]	Aodi LIU, Xuehui DU, Na WANG, Rui QIAO. ABAC access control policy generation technique based on deep learning [J]. Journal on Communications, 2020, 41(12): 8-20.
[3]	Yangchen HUANG,Yan JIA,Liang GAN,Jing XU,Jiuming HUANG,Zhonghe HE. Multi-factor person entity relation extraction model based on distant supervision [J]. Journal on Communications, 2018, 39(7): 103-112.
[4]	Shao-feng GENG,Yong-heng WANG,Ren-fa LI,Jia ZHANG. Research of proactive complex event processing method [J]. Journal on Communications, 2016, 37(9): 111-120.
[5]	Hao CHEN,Yu LI,Song-lin HU,Ying LIANG. Parallel complex event processing system based on S4 framework [J]. Journal on Communications, 2012, 33(Z1): 165-169.