一种半监督学习的金融新闻文本分类算法

doi:10.11959/j.issn.2096-0271.2022019

Abstract

Abstract:

Classifying financial texts is a common task for identifying financial risks.Traditional financial news classification requires a large number of labeled texts to train the classifier.However, labeling financial news requires not only professional financial background knowledge, but also time-consuming and labor-intensive.In order to reduce the dependence on labeled text, a semi-supervised learning financial text classification algorithm- SSF (semi-supervised learning financial news classification algorithm) was proposed, which uses a consistent training method of supervised learning and unsupervised learning to improve the use of unlabeled data.And unsupervised data augmentation for financial texts was introduced, that is, use specific target data augmentation methods for specific tasks to generate more effective data.Experiments on multiple financial news data sets were conducted to verify that the proposed SSF algorithm has a significant improvement in effectiveness compared with other text classification algorithms.

Key words: natural language processing, text classification, semi-supervised learning, finance

CLC Number:

TP312

Xiaolong ZHANG, Long ZHI, Jian GAO, Zhongchen MIAO, Yuefeng LIN, Yali XIANG, Yun XIONG. A semi-supervised learning financial news classification algorithm[J]. Big Data Research, 2022, 8(2): 134-144.

Figures/Tables 9

References 25

[1]	CHAPELLE O , SCHOLKOPF B , ZIEN E . Semi-supervised learning (Chapelle,O.et al.Eds.; 2006)[book reviews][J]. IEEE Transactions on Neural Networks, 2009,20(3): 542.
[2]	RASMUS A , VALPOLA H , HONKALA M ,et al. Semi-supervised learning with ladder network[J]. arXiv preprint,2015,arXiv:1507.02672.
[3]	LAINE S , AILA T M . Temporal ensembling for semi-supervised learning[J]. arXiv preprint,2016,arXiv:1610.02242.
[4]	TARVAINEN A , VALPOLA H . Weightaveraged,consistency targets improve semisupervised deep learning results[Z]. 2017.
[5]	BACHMAN P , ALSHARIF O , PRECUP D . Learning with pseudo-ensembles[J]. arXiv preprint,2014,arXiv:1412. 4864.
[6]	MIYATO T , MAEDA S I , KOYAMA M ,et al. Virtual adversarial training:a regularization method for supervised and semi-supervised learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019,41(8): 1979-1993.
[7]	SAJJADI M , JAVANMARDI M , TASDIZEN T . Regularization with stochastic transformations and perturbations for deep semi-supervised learning[J]. arXiv preprint,2016,arXiv:1606.04586.
[8]	CLARK K , LUONG M T , MANNING C D ,et al. Semi-supervised sequence modeling with cross-view training[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2018.
[9]	VERMA V , KAWAGUCHI K , LAMB A ,et al. Interpolation consistency training for semi-supervised learning[J]. arXiv preprint,2019,arXiv:1903.03825.
[10]	BERTHELOT D , CARLINI N , GOODFELLOW I J ,et al. MixMatch:a holistic approach to semi-supervised learning[J]. arXiv preprint,2019,arXiv:1905.02249.
[11]	XIE Q Z , DAI Z H , HOVY E ,et al. Unsupervised data augmentation for consistency training[J]. arXiv preprint,2019,arXiv:1904.12848.
[12]	RADFORD A , WU J , CHILD R ,et al. Language models are unsupervised multitask learners[Z]. 2019.
[13]	CHEN J A , CHEN J S , YU Z . Incorporating structured commonsense knowledge in story completion[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019,33(1): 6244-6251.
[14]	AKBIK A , BERGMANN T , VOLLGRAF R . Pooled contextualized embeddings for named entity recognition[C]// Proceedings of the 2019 Conference of the North. Stroudsburg:Association for Computational Linguistics, 2019.
[15]	HOWARD J , RUDER S . Universal language model fine-tuning for text classification[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2018.
[16]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint,2013,arXiv:1301. 3781.
[17]	LAN Z Z , CHEN M D , GOODMAN S ,et al. ALBERT:a lite BERT for self-supervised learning of language representations[J]. arXiv preprint,2019,arXiv:1909.11942.
[18]	RADFORD A , NARASIMHAN K , SALIMANS T ,et al. Improving language understanding by generative pretraining[Z]. 2018.
[19]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint,2018,arXiv:1810.04805.
[20]	BEYER L , ZHAI X H , OLIVER A ,et al. S4L:self-supervised semi-supervised learning[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 1476-1485.
[21]	OLIVER A , ODENA A , RAFFEL C ,et al. Realistic evaluation of deep semisupervised learning algorithms[J]. arXiv preprint,2018,arXiv:1804.09170.
[22]	PENNINGTON J , SOCHER R , MANNING C . Glove:global vectors for word representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2014.
[23]	PETERS M E , NEUMANN M , IYYER M ,et al. Deep contextualized word representations[J]. arXiv preprint,2018,arXiv:1802.05365.
[24]	JOULIN A , GRAVE E , BOJANOWSKI P ,et al. FastText.zip:compressing text classification models[J]. arXiv preprint,2016,arXiv:1612.03651.
[25]	GURURANGAN S , DANG T , CARD D ,et al. Variational pretraining for semisupervised text classification[J]. arXiv preprint,2019,arXiv:1906.02242.

Metrics

Recommended 0

No Suggested Reading articles found!

违规类别	合规	违规类别1	违规类别2	违规类别3	违规类别4	无标签数据
数量/条	307	68	52	35	60	57 000

主题类别	豆粕期权	贵州茅台	国泰君安	股指期权	汇率波动	区块链	中国石油	贷款基础利率（LPR）	无标签数据
数量/条	16	273	102	200	79	1 325	46	42	29 969

机构名称	淡水泉	敦和资管	高毅资产	汉和资本	景林资产	凯丰投资	林园投资	明汯投资	千合资本	石锋资产	无标签数据
数量/条	20	20	20	20	20	20	20	20	20	20	4 000

模型	违规类别数据集			期货期权数据集			机构相关数据集
模型	精度	召回率	F1值	精度	召回率	F1值	精度	召回率	F1值
GloVe	0.709	0.684	0.696	0.785	0.789	0.787	0.710	0.704	0.707
ELMo	0.705	0.712	0.708	0.769	0.781	0.775	0.724	0.718	0.721
FastText	0.743	0.729	0.736	0.756	0.749	0.752	0.789	0.801	0.795
VAMPIRE	0.749	0.735	0.742	0.778	0.762	0.770	0.838	0.853	0.845
BERT	0.789	0.784	0.786	0.878	0.873	0.875	0.919	0.928	0.923
UDA	0.811	0.802	0.806	0.920	0.915	0.917	0.938	0.951	0.944
SSF	0.819	0.807	0.813	0.939	0.932	0.935	0.973	0.976	0.974

模型	违规类别数据集		期货期权数据集		机构相关数据集
模型	F1值	F1值下降率	F1值	F1值下降率	F1值	F1值下降率
GloVe	0.385	44.68%	0.521	33.80%	0.525	25.74%
ELMo	0.407	42.51%	0.536	30.84%	0.548	23.99%
FastText	0.561	23.78%	0.589	21.68%	0.631	20.63%
VAMPIRE	0.406	45.28%	0.531	31.04%	0.550	34.91%
BERT	0.583	25.83%	0.633	27.66%	0.683	26.00%
UDA	0.684	15.14%	0.782	14.72%	0.821	13.03%
SSF	0.772	5.04%	0.883	5.56%	0.887	8.93%

A semi-supervised learning financial news classification algorithm

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 25

Related Articles 10

Metrics

Recommended 0

变量	违规类别数据集			期货期权数据集			机构相关数据集
变量	精度	召回率	F1值	精度	召回率	F1值	精度	召回率	F1值
SSF-w/o-aug	0.805	0.789	0.797	0.917	0.920	0.918	0.952	0.956	0.954
SSF	0.819	0.807	0.813	0.939	0.932	0.935	0.973	0.976	0.974

[1]	Xiaojun WAN. Intelligent text generation: recent advances and challenges [J]. Big Data Research, 2023, 9(2): 99-109.
[2]	Yimin DENG, Shijing SI, Jianzong WANG, Zeyuan LI, Jing XIAO. Exchange mechanism for decentralized finance: a survey [J]. Big Data Research, 2022, 8(4): 67-84.
[3]	Qian SUN, Yongbin QIN, Ruizhang HUANG, Lijuan LIU, Yanping CHEN. Charge prediction method combined with case elements sequence [J]. Big Data Research, 2021, 7(6): 30-40.
[4]	Yifeng WANG, Liru SUN, Liangle CUI, Yi ZHAO. Adaptive feature spectrum neural networks for special types of natural language classification [J]. Big Data Research, 2020, 6(4): 92-104.
[5]	Dong HAN, Qihua LI, Wei CAI, Yuwei XIA, Jia NING, Feng HUANG. Research and application of artificial intelligence in medical imaging [J]. Big Data Research, 2019, 5(1): 39-67.
[6]	Chonggang LI, Huiquan XU. Smoke index:big data technologies monitor Internet financial risks [J]. Big Data Research, 2018, 4(4): 76-84.
[7]	Xiaoxia WANG, Fusong JIANG, Yu WANG, Yun XIONG. Chronic disease complications clustering based on ICD-10 diagnoses code [J]. Big Data Research, 2018, 4(3): 37-45.
[8]	Xiaofeng MA, Mingxiao DU, Wenbing YU, Yi WANG. Blockchain based supply-chain finance service platform [J]. Big Data Research, 2018, 4(1): 13-21.
[9]	Yunsheng SONG. Application of an emotion discriminant analysis system in the management of automobile brand [J]. Big Data Research, 2017, 3(6): 55-64.
[10]	KamFai WONG. Beware of traps of big data analytics in business [J]. Big Data Research, 2017, 3(2): 26-30.