基于深度学习的警情记录关键信息自动抽取

doi:10.11959/j.issn.2096-0271.2022052

摘要/Abstract

摘要： 随着智慧警务的兴起，民众报警渠道拓宽，非结构化警情激增，警情实体识别难度增大。针对这一业务痛点，引入了BERT模型获取词向量，融合了自注意力机制来捕获文字之间的长距离依赖关系，并构建了BERT-BiGRU-SelfAtt-CRF警情实体识别模型。为验证模型的性能和泛化能力，在公开数据集上进行了实验。为验证模型在警情领域的效率和可行性，在构建的警情数据集上进行了实验。最终，结果表明BERT-BiGRU-SelfAtt-CRF模型在警情数据集上的精确度达到了82.45%，F1分数达到了80.72%，表现优于其他模型。可见，此模型可以满足实际公安工作需要，是有效可行的。

关键词: 深度学习, 预训练语言模型, 自注意力机制, 警情实体识别

Abstract:

As the emergence of intelligent policing, the channels of mass to call police are widened, unstructured police records increase immensely, and the difficulty of police entity recognition is Public Security Behavioral Science Laboratory, Peoples Public Security University of China (2020sys08).magnified. For this pain point, BERT model was introduced to generate the word vector, the Self-attention mechanism was integrated to capture the long-distance dependence between words, and the BERT-BiGRU-SelfAtt-CRF police entity recognition model was constructed. In order to verify the performance and generalization ability of the model, experiments were carried out on public datasets. And to prove the feasibility and efficiency of the model in the police field, experiments were run on the annotated police dataset. Ultimately, the results showed that BERT-BiGRU-SelfAtt-CRF model outperformed other models on the police dataset, with the precision of 82.45% and F1 score of 80.72%. It is concluded that this model could meet the requirements of actual police work, and it is effective and feasible in the field of police entity recognition.

Key words: deep learning, pretrained language model, self-attention mechanism, entity recognition in police records

崔雨萌, 王靖亚, 闫尚义, 陶知众.

基于深度学习的警情记录关键信息自动抽取 [J]. 大数据, doi: 10.11959/j.issn.2096-0271.2022052.

CUI Yumeng, WANG Jingya, YAN Shangyi, TAO Zhizhong.

Automatic Key Information Extraction of Police Records Based on Deep Learning [J]. Big Data Research, doi: 10.11959/j.issn.2096-0271.2022052.

参考文献

[1] 张晓艳，王挺，陈火旺. 命名实体识别研究[J]. 计算机科学，2005，32（4）： 44-48.
Zhang X Y, Wang T, Chen H W. Research on Named Entity Recognition[J]. Computer Science，2005，32(4): 44-48.
[2] 何玉洁，杜方，史英杰，等. 基于深度学习的命名实体识别研究综述[J]. 计算机工程与应用，2021，57（11）：21-36.
He Y J, Du F, Shi Y J, et al. Survey of Named Entity Recognition Based on Deep Learning[J]. Computer Engineering and Application, 2021, 57(11): 21-36.
[3] 王月，王孟轩，张胜，等. 基于BERT的警情文本命名实体识别[J]. 计算机应用，2020，40（2）：535-540.
Wang Y, Wang M X, Zhang S, et al. Alarm text named entity recognition based on BERT[J]. Journal of Computer Applications, 2020, 40(2): 535-540.
[4] Zhang Y, Yang J. Chinese NER Using Lattice LSTM[C]//In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
[5] Isozaki H, Kazawa H. Efficient support vector classifiers for named entity recognition[C]//COLING 2002. The 19th International Conference on Computational Linguistics. 2002.
[6] Liu K, Hu Q, Liu J, et al. Named entity recognition in Chinese electronic medical records based on CRF[C]//2017 14th Web Information Systems and Applications Conference (WISA). IEEE, 2017: 105-110.
[7] Han A.L.F, Wong D.F, Chao L.S. Chinese named entity recognition with conditional random fields in the light of Chinese characteristics[C]//In Proceedings of the Intelligent Information Systems Symposium, June 2013, Springer: Berlin/Heidelberg, Germany. pp. 57-68.
[8] Morwal S, Jahan N, Chopra D. Named entity recognition using hidden Markov model (HMM)[J]. International Journal on Natural Language Computing (IJNLC), 2012.
[9] Fu G H, Luke K K. Chinese named entity recognition using lexicalized HMMs[J]. ACM SIGKDD Explorations Newsletter, 2005, 7(1): 19-25.
[10] Bender O, Och F J, Ney H. Maximum entropy models for named entity recognition[C]//Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003. 2003: 148-151.
[11] Chieu H L, Ng H T. Named entity recognition: a maximum entropy approach using global information[C]//COLING 2002: The 19th International Conference on Computational Linguistics. 2002.
[12] 吴超，王汉军. 基于GRU的电力调度领域命名实体识别方法[J]. 计算机系统应用，2020, 29（8）：185-191.
Wu C, Wang H J. Named Entity Recognition in Electric Power Dispatching Field Based on GRU[J]. Computer Systems & Applications, 2020, 29(8): 185-191.
[13] Dong C H, Wu H J, Zhang J J, et al. Multichannel LSTM-CRF for Named Entity Recognition in Chinese Social Media[C]//In Proceedings of the China National Conference on Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, 2017; pp. 197-208.
[14] Wu F Z, Liu J X, Wu C H, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation[C]//The World Wide Web Conference. 2019: 3342-3348.
[15] Dong C H, Zhang J J, Zong C Q, et al. Character-Based LSTM-CRF with Radical-Level Features for Chinese Named Entity Recognition[C]//In Proceedings of Natural Language Understanding and Intelligent Applications, Springer, Cham, 2016; pp. 239–250.
[16] Tang B Z, Wang X L, Yan J, et al. Entity recognition in Chinese clinical text using attention-based CNN-LSTM-CRF[J]. BMC medical informatics and decision making, 2019, 19(3): 89-97.
[17] Huang Z H, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv preprint arXiv:1508.01991, 2015.
[18] Chen Y, Zhou C J, Li T X, et al. Named entity recognition from Chinese adverse drug event reports with lexical feature based BiLSTM-CRF and tri-training[J]. Journal of biomedical informatics, 2019, 96: 103252.
[19] 李一斌，张欢欢. 基于双向GRU-CRF的中文包装产品实体识别[J]. 华东理工大学学报（自然科学版），2019, 45, 486-490.
Li Y B, Zhang H H. Chinese Packaging Product Entity Recognition Based on Bidirectional GRU-CRF[J]. Journal of East China University of Science and Technology (Nature Science Edition), 2019, 45, 486-490.
[20] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in neural information processing systems. 2013: 3111-3119.
[21] Devlin J, Chang M. W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[22] Li X Y, Zhang H, Zhou X H. Chinese clinical named entity recognition with variant neural structures based on BERT methods[J]. Journal of biomedical informatics, 2020, 107: 103422.
[23] Yin X Z, Zhao H, Zhao J B, et al. Multi-neural network collaboration for Chinese military named entity recognition[J]. Journal of Tsinghua University (Science and Technology), 2020, 60(8): 648-655.
[24] Gu L, Zhang W J, Wang Y, et al. Named Entity Recognition in Judicial Field Based on BERT-BiLSTM-CRF Model[C]//In Proceedings of 2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI), Shanghai, China, June 2020; pp. 170–174.
[25] Nie Y Y, Tian Y H, Wan X, et al. Named Entity Recognition for Social Media Texts with Semantic Augmentation[C]//In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, 2020; pp. 1383–1391.
[26] Gong C, Tang J Y, Zhou S W, et al. Chinese named entity recognition with bert[J]. DEStech Transactions on Computer Science and Engineering, 2019 (cisnrc).
[27] Wu G H, Tang G G, Wang Z R, et al. An Attention-Based BiLSTM-CRF Model for Chinese Clinic Named Entity Recognition[J]. IEEE Access 2019, 7, 113942–113949.
[28] Zhong Q, Tang Y. An Attention-Based BILSTM-CRF for Chinese Named Entity Recognition[C]//In Proceedings of 2020 IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, China, April 2020; pp. 550–555.
[29] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// In Proceedings of the Advances in neural information processing systems, 2017; pp. 5998-6008.
[30] Cho K, Van M B, Gulcehre C, et al. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation[J]. Computer Science, 2014, 1724–1734.
[31] Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate [J]. arXiv preprint arXiv:1409.0473, 2018.
[32] Lafferty J, Mccallum A, Pereira F. Conditional random fields: Probabilistic models for segmenting and labeling sequence data [C]//In Proceedings of 18th International Conference on Machine Learning, 2001, 3(2):282-289.
[33] Gina A L. The third international Chinese language processing bakeoff : Word segmentation and named entity recognition[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Proceeding, 2006: 548-554.