基于概率分布差异的医学命名实体识别方法

doi:10.11959/j.issn.2096-0271.2023008

摘要/Abstract

摘要：

医学命名实体识别是从医学文本中抽取出指代特定概念的医学实体，是医学信息抽取的基础性任务。当前主流的医学命名实体识别算法普遍基于深度学习技术，需要大量高质量的标注样本进行模型训练。然而医学领域的样本标注成本很高，严重限制了模型性能的提升。为了降低模型对标注样本的需求，一种重要方法是基于主动学习思想，设计合理的样本采样策略，自动选取高价值样本优先标注，从而使模型提前收敛。现有算法普遍基于样本长度、样本识别的概率等特征设计采样策略，忽视了样本类别分布这一深层次特征，导致命名实体识别召回率较低。提出了一种基于概率分布差异的主动学习算法，通过计算样本间的概率分布差异评估样本的标注价值，并在标注样本更新时动态优化模型。在真实的医学检查文本上的实验表明，相比已有算法，达到同等的模型性能，本文算法所需要的标注数据缩减10%以上。在同样标注样本量的情况下，F1值提高5%以上。

关键词:

医学命名实体识别, 深度学习, 主动学习, 概率分布

Abstract:

Medical named entity recognition is the task of extracting text parts referring to specific concepts from medical domain texts, which is a basic task of medical information extraction. The current mainstream approaches are generally based on deep learning, and it requires a lot of high-quality labeled samples for model training. However, the cost of labeling samples in the medical field is expensive, and it severely limits the performance of the model. In order to reduce the demands of the model on labeled samples, one important approach is based on the active learning, design a reasonable sampling strategy to automatically select high-value samples and label them first, thus allow the model converge earlier. Existing algorithms typically design their sampling strategies based on the length or recognition probability of samples, ignoring the feature of sample category distribution, which results in low recall of named entity recognition. An active learning algorithm based on the difference of probability distribution is proposed. It evaluate the labeling value of samples by calculating the difference of probability distribution between samples and dynamically optimize the model when the labeling samples are updated. Experiments on real medical examination text data show that the algorithm in this paper requires 10% less labeled data to achieve the same model performance compared to existing algorithms. With the same amount of annotated data, the F1 value is improved by more than 5%.

Key words:

medical named entity recognition, deep learning, active learning, probability distribution

刘聪, 吕雪峰, 王宏林, 王晓伟, 陆瑾, 孙顺, 胡松奇.

基于概率分布差异的医学命名实体识别方法 [J]. 大数据, doi: 10.11959/j.issn.2096-0271.2023008.

LIU Cong, LV Xuefeng, WANG Honglin, WANG Xiaowei, LU Jin, SUN Shun, HU Songqi.

Medical Named Entity Recognition Algorithm Based on Probability Distribution Difference [J]. Big Data Research, doi: 10.11959/j.issn.2096-0271.2023008.

参考文献

[1] 杨威, 刘艳如, 孟颖, 等. 浅谈临床医学术语的标准化管理[J]. 中国卫生标准管理, 2021, 12(12):1-4.

YANG W, LIU Y R, MENG Y, et al. Discussion on Standardization Management of Clinical Medical Terminology[J], China Health Standard Management | Chin Heal standard management. 2021, (12):1-4.

[2] 赵嘉莹, 高鹏, 朱勇俊, 等. 人工智能的应用将改进中国基层医疗卫生服务效能[J]. 中国全科医学, 2017, 20(34):4219-4233.

ZHAI J Y, GAO P, ZHU Y J, et al. The application of artificial intelligence could improve primary health care provision in china[J]. Chinese General Practice,2017,20(24):4219-4223.

[3] 曾晓天, 徐春园, 张勇, 等. 人工智能在医学大数据标准化体系建设中的研究进展[J]. 北京生物医学工程, 2019, 38(6): 640-644.

ZENG X T, XU C Y, ZHANG Y, et al. Research progress on artificial intelligence in the standardization system construction of medical big data[J]. Beijing Biomedical Engineering, 2019, 38(6): 640-644.

[4] 郑强, 刘齐军, 王正华, 等.生物医学命名实体识别的研究与进展[J]. 计算机应用研究,2010, 27(3):811-815,832.

ZHEN Q, LIU Q J, WANG Z H, et al. Research and development on biomedical named entity recognition[J]. APPLICATION RESEARCH OF COMPUTERS, 2010, 27(3):811-815,832.

[5] SETTLES B. Active learning literature suvery[J]. Machine Learning, 2010,15(2):201-221.

[6] Hanisch D, Fundel K, Mevissen H T, et al. ProMiner: rule-based protein and gene entity recognition. 2005.

[7] 刘一佳, 车万翔, 刘挺, 等. 基于序列标注的中文分词,词性标注模型比较分析[C]//第六届全国青年计算语言学会议论文集,上海, 2012年11月17日. 2012:26-34.

LIU Y J, CHE W X, LIU T, et al. A Comparison Study of Sequence Labeling Methods for Chinese Word Segmentation, POS Tagging Models[C]//The 6th Youth Conference of Computational Linguistics, ShangHai, Nov 17, 2012. 2012:26-34.

[8] 王浩畅, 赵铁军. 基于SVM的生物医学命名实体的识别[J]. 哈尔滨工程大学学报, 2006, 27(z1):570-574.

WANG H C, ZHAO T J, et al. SVM-based biomedical name entity recognition[J]. JOURNAL OF HARBIN ENGINEERING UNIVERSITY, 2006,27(z1):570-574.

[9] Morwal S, Chopra D. NERHMM: A Tool for Named Entity Recognition Based on Hidden Markov Model[J]. International Journal on Natural Language Computing, 2013, 2(2):43-49.

[10] Patil N, Patil A, Pawar B V. Named Entity Recognition using Conditional Random Fields[J]. Procedia Computer Science, 2020, 167:1181-1188.

[11] Lample G, Ballesteros M, Subramanian S, et al. Neural Architectures for Named Entity Recognition[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.

[12] Ouyang E, Li Y, Jin L, et al. Exploring N-gram Character Presentation in Bidirectional RNN-CRF for Chinese Clinical Named Entity Recognition[C]// CCKS: China Conference on Knowledge Graph and Semantic Computing 2017. 2017.

[13] Dong X, Chowdhury S, Qian L, et al. Transfer bi-directional LSTM RNN for named entity recognition in Chinese electronic medical records[C]//2017 IEEE 19Th international conference on e-health networking, applications and services.

[14] Zhang Z, Zhang Y, Zhou T. Medical knowledge attention enhanced neural model for named entity recognition in chinese emr[M]//Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. Springer, Cham, 2018: 376-385.

[15] Wang Q, Zhou Y, Ruan T, et al. Incorporating dictionaries into deep neural networks for the chinese clinical named entity recognition[J]. Journal of biomedical informatics, 2019, 92: 103133.

[16] Qiu J, Wang Q, Zhou Y, et al. Fast and accurate recognition of chinese clinical named entities with residual dilated convolutions[C]//2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2018: 935-942.

[17] Li X, Zhang H, Zhou X H. Chinese Clinical Named Entity Recognition with Variant Neural Structures Based on BERT Methods[J]. Journal of Biomedical Informatics, 2020, 107(5):103422.

[18] 张岑芳. 基于主动学习的命名实体识别算法[J]. 计算机与现代化, 2021(7): 18-22.

ZHANG C F. Named Entity Recognition Algorithm Based on Active Learning[J]. Computer and Modernization, 2021(7): 18-22.

[19] 卢宁杰. 结合主动学习的中文医疗命名实体识别研究[D]. 华东师范大学, 2020.

Lu N J. Research on Chinese Medical Named Entity Recognition Combined with Active Learning[D]. Shanghai: East China Normal University, 2020.

[20] C.E. Shannon. A mathematical theory of communication. Bell System Technical Journal[J], 1948, 27:379-423, 623-656.

[21] Lewis and J. Catlett. Heterogeneous uncertainty sampling for supervised learning[C]//In Proceedings of the International Conference on Machine Learning(ICML), 1994, 148-156.

[22] T. Scheffer, C. Decomain, and S. Wrobel. Active hidden Markov models for information extraction[C]//In Proceedings of the International Conference on Advances in Intelligent Data Analysis (CAIDA), 2001, 309-318.

[23] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// 2018.

[24] Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18(5-6): 602-610.

[25] Sutton C, Mccallum A. An Introduction to Conditional Random Fields[J]. Foundations and Trends® in Machine Learning, 2010, 4(4): 267-373.

[26] Kingma D, Ba J. Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.

[27] Zan H, Li W, Zhang H, et al. Building a Pediatric Medical Corpus: Word Segmentation and Named Entity Annotation[C]. CLSW2020: 652-664.

[28] Lan Z, Chen M, Goodman S, et al. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations[C]//International Conference on Learning Representations. hgpu.org, 2020.

[29] Diao S, Bai J, Song Y, et al. ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations[C]// Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020.