数字人文视域下面向历史古籍的信息抽取方法研究

doi:10.11959/j.issn.2096-0271.2022058

摘要/Abstract

摘要：

数字人文旨在采用现代计算机网络技术助力传统人文研究，文言历史古籍是进行历史研究和学习的重要基础，但由于其写作语言为文言文，与现代所用的白话文在语法和词义上均有较大差别，因此不易于阅读和理解。针对上述问题，本文提出基于预训练模型对历史古籍中的实体和关系等进行知识抽取，从而有效获取历史古籍文本中蕴含的丰富信息。该模型首先采用多级预训练任务代替BERT原有的预训练任务以充分捕获语义信息，此外在BERT模型的基础上添加了卷积层以及句子级聚合等结构，以进一步优化生成的词表示。之后，针对文言文标注数据稀缺的问题，构建了一个面向历史古籍文本标注任务的众包系统，获取高质量、大规模的实体和关系数据并完成文言文知识抽取数据集的构建，从而评估模型性能并对模型进行微调。在本文构建的数据集以及GulianNER数据集上的实验证明了本文提出模型的有效性。

关键词: 历史古籍, 预训练模型, 信息抽取, 众包机制

Abstract:

Digital humanities aims to use modern computer network technology to help traditional humanities research. Classical Chinese historical books are an important basis for historical research and learning, but since their writing language is classical Chinese, which is quite different from the vernacular Chinese in grammar and meaning, it is not easy to read and understand. In view of the above problems, this paper proposes to extract entities and relations in historical books based on pre-trained models, so as to effectively obtain the rich information contained in historical texts. The model first uses multi-level pre-training tasks instead of BERT's original pre-training tasks to fully capture semantic information, and adds structures such as convolutional layers and sentence-level aggregations on the basis of the BERT model to further optimize the generated word representation. After that, in view of the scarcity of classical Chinese annotation data, a crowdsourcing system for the task of labeling historical classics is constructed, and high-quality, large-scale entity and relation data is obtained to construct the classical Chinese knowledge extraction dataset, so as to evaluate the performance of the model and fine-tune the model. Experiments on the dataset constructed in this paper and on the GulianNER dataset demonstrate the effectiveness of the model proposed in this paper.

Key words:

"> historical classics, pre-trained models, information extraction, crowdsourcing mechanism

季紫荆, , 陈子睿, , 韩立帆, , 王鑫, , . 数字人文视域下面向历史古籍的信息抽取方法研究[J]. 大数据, doi: 10.11959/j.issn.2096-0271.2022058.

JI Zijing, , CHEN Zirui , , HAN Lifan, , WANG Xin, , .

Research on information extraction methods for historical classics under the perspective of digital humanities [J]. Big Data Research, doi: 10.11959/j.issn.2096-0271.2022058.

参考文献

[1] SINGHAL A. Introducing the knowledge graph: Things, not strings [EB/OL]. [2021-11-19].

https://blog.google/products/search/introducing-knowledge-graph-things-not/.

[2] J DEVLIN, M CHANG, K LEE, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv: 1810.04805, 2018.

[3] HOWE J. The Rise of Crowdsourcing[J]. Wired, 2006, 14(6): 176-183.

[4] HOLLEY R. Crowdsourcing: How and Why Should Libraries Do It?[J]. D-Lib Magazine, 2010, 16(3/4): 1-21.

[5] OOMEN J, AROYO L. Crowdsourcing in the cultural heritage domain: opportunities and challenges[C]//Proceedings of the 5th International Conference on Communities and Technologies, 2011, Brisbane, Australia. New York: Association for Computing Machinery. 2011: 138-149.

[6] TERRAS M. Digital curiosities: resource creation via amateur digitization[J]. Literary and linguistic computing, 2010, 25(4): 425-438.

[7] RIDGE M. Citizen History and Its Discontents[C]//Proceedings of the 2014 IHR Digital History Seminar. Humanities Commons, 18 November 2014, London. 2014: 1-13.

[8] ZHANG X, SONG S, ZHAO Y, et al. Motivations of volunteers in the Transcribe Sheng project: a grounded theory approach[J]. Proceedings of the Association for Information Science and Technology, 2018, 55(1): 951-953.

[9] RIDGE M. From tagging to theorizing: deepening engagement with cultural heritage through crowdsourcing[J]. Curator: The Museum Journal, 2013, 56(4): 435-450.

[10] DANIELS C, HOLTZE T L, HOWARD R I, et al. Community as resource: Crowdsourcing transcription of an historic newspaper[J]. Journal of electronic resources librarianship, 2014, 26(1): 36-48.

[11] CONCILIO G, VITELLIO I. Co-creating intangible cultural heritage by crowd-mapping: The case of mappi [na][C]//2016 IEEE 2nd International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI), 14 November 2016, Bologna, Italy. Bologna: IEEE, 2016: 1-5.

[12] D E RUMELHART, G E HINTON, R J WILLIAMS. Learning internal representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536.

[13] G E HINTON, J L MCCLELLAND, D E RUMELHART. Distributed representations[M]. Cambridge: MIT Press, 1986: 77-109.

[14] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781, 2013.

[15] MCCANN B, BRADBURY J, XIONG C, et al. Learned in translation: Contextualized word vectors[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, Long Beach, California, USA. Red Hook: Curran Associates Inc, 2017: 6297-6308.

[16] M E PETERS, M NEUMANN, M IYYER, et al. Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2018, New Orleans, Louisiana. New York: Association for Computational Linguistics, 2018: 2227-2237.

[17] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.

[18] STAUDEMEYER R C, MORRIS E R. Understanding LSTM--a tutorial into long short-term memory recurrent neural networks[J]. arXiv preprint arXiv: 1909.09586, 2019.

[19] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[J]. 2018.

[20] MA X, HOVY E. End-to-end sequence labeling via bi-directional lstm-cnns-crf[J]. arXiv preprint arXiv: 1603.01354, 2016.

[21] LIU Y, OTT M, GOYAL N, et al. Roberta: A robustly optimized bert pretraining approach[J]. arXiv preprint arXiv: 1907.11692, 2019.

[22] LAN Z, CHEN M, GOODMAN S, et al. Albert: A lite bert for self-supervised learning of language representations[J]. arXiv preprint arXiv: 1909.11942, 2019.

[23] YANG Z, DAI Z, YANG Y, et al. Xlnet: Generalized autoregressive pretraining for language understanding[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019, Red Hook, NY, USA. Red Hook: Curran Associates Inc, 2019: 5753-5763.

[24] CUI Y, CHE W, LIU T, et al. Pre-training with whole word masking for chinese bert[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.

[25] 王东波，刘畅，朱子赫，等. SikuBERT与SikuRoBERTa：面向数字人文的《四库全书》预训练模型构建及应用研究[J/OL]. 图书馆论坛: 1-14[2022-03-10].

WANG D B, LIU C, ZHU Z H, et al. Construction and Application of Pre-training Model of “Siku Quanshu” Oriented to Digital Humanities. [J/OL]. Library Tribune:1-14[2022-03-10].