基于双向邻居修正的局部异常因子算法

doi:10.11959/j.issn.1000-436x.2020119

Abstract

Abstract:

A local outlier factor algorithm based on bidirectional neighbor correction was proposed to solve the problems of existing outlier detection algorithms such as difficulty in parameter selection,poor efficiency and low accuracy.The bidirectional neighbor searching algorithm was used to reduce the neighbor search time.Then the bidirectional neighbor pruning algorithm was used to reduce the number of parameters and unnecessary calculations.And the correction factor based on bidirectional neighbors was used to improve the calculation accuracy.Experimental results show that the proposed algorithm has better performance in parameter selection and time efficiency than other outlier detection methods.The correction factor improves the accuracy of the algorithm,in the synthetic data set and UCI data set.

Key words: outlier detection, local outlier factor, bidirectional neighbor, correction factor

CLC Number:

TP181

Xiaohui YANG,Xiaoming LIU. Local outlier factor algorithm based on correction of bidirectional neighbor[J]. Journal on Communications, 2020, 41(8): 130-140.

Figures/Tables 13

References 29

[1]	MALINI N , PUSHPA M . Analysis on credit card fraud identification techniques based on KNN and outlier detection[C]// Third International Conference on Advances in Electrical. Piscataway:IEEE Press, 2017, 255-258.
[2]	杨加, 李笑难, 张扬 ,等. 基于大数据分析的校园电子邮件异常行为检测技术研究[J]. 通信学报, 2018,39(z1): 116-123.
	YANG J , LI X N , ZHANG Y ,et al. Abnormal behavior detection for campus email systems based on big data analysis[J]. Journal on Communications, 2018,39(z1): 116-123.
[3]	GUO J , HUANG W , WILLIAMS B M . Real time traffic flow outlier detection using short-term traffic conditional variance prediction[J]. Transportation Research Part C:Emerging Technologies, 2015,(50): 160-172.
[4]	SCHUBERT E , ZIMEK A , KRIEGEL H P . Local outlier detection reconsidered:a generalized view on locality with applications to spatial video,and network outlier detection[J]. Data Mining and Knowledge Discovery, 2014,28(1): 190-237.
[5]	琚安康, 郭渊博, 李涛 ,等. 基于网络通信异常识别的多步攻击检测方法[J]. 通信学报, 2019,40(7): 57-66.
	JU A K , GUO Y B , LI T ,et al. Multi-step attack detection method based on network communication anomaly recognition[J]. Journal on Communications, 2019,40(7): 57-66.
[6]	KIM G , LEE S , KIM S . A novel hybrid intrusion detection method integrating anomaly detection with misuse detection[J]. Expert Systems with Applications, 2014,41(4): 1690-1700.
[7]	丁兆云, 周斌, 贾焰 ,等. 微博中基于统计特征与双向投票的垃圾用户发现[J]. 计算机研究与发展, 2013,50(11): 2336-2348.
	DING Z Y , ZHOU B , JIA Y ,et al. Detecting spammers with a bidirectional vote algorithm based on statistical features in microblogs[J]. Journal of Computer Research and Development, 2013,50(11): 2336-2348.
[8]	KONTAKI M , GOUNARIS A , PAPADOPOULOS A N ,et al. Efficient and flexible algorithms for monitoring distance-based outliers over data streams[J]. Information Systems, 2016,55: 37-53.
[9]	CHEN H , FU Y , ZHENG Y . Survey on big data analysis algorithms for network security measurement[J]. Network and System Security,2017, 1039,4: 128-142.
[10]	DU H , ZHAO S , ZHANG D ,et al. Novel clustering-based approach for local outlier detection[C]// IEEE International Conference on Computer Communications Workshops. Piscataway:IEEE Press, 2016: 802-811.
[11]	PEROZZI B , AKOGLU L , IGLESIAS SáNCHEZ P ,et al. Focused clustering and outlier detection in large attributed graphs[C]// Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York:ACM Press, 2014: 1346-1355.
[12]	LIU F T , TING K M , ZHOU Z H . Isolation-based anomaly detection[J]. ACM Transactions on Knowledge Discovery from Data, 2012,6(1): 1-39.
[13]	WANG P , LI W , GAO Z ,et al. Action recognition from depth maps using deep convolutional neural networks[J]. IEEE Transactions on Human-Machine Systems, 2016,46(4): 498-509.
[14]	HOULE M E , SCHUBERT E , ZIMEK A . On the correlation between local intrinsic dimensionality and outlierness[J]. International Conference on Similarity Search and Applications, 2018,11223: 177-191.
[15]	TANG B , HE H . A local density-based approach for local outlier detection[J]. Pattern Recognition Letters, 2017,241: 171-180.
[16]	BREUNIG M M , KRIEGEL H P , NG R T ,et al. LOF:identifying density-based local outliers[C]// Proceedings of the 2000 ACM SIGMOD international conference on Management of data. New York:ACM Press, 2000: 93-104.
[17]	HA J , SEOK S , LEE J S . Robust outlier detection using the instability factor[J]. Knowledge-Based Systems, 2014,63: 15-23.
[18]	JIN W , TUNG A K H , HAN J ,et al. Ranking outliers using symmetric neighborhood relationship[J]. Lecture Notes in Computer Science, 2007,3918: 577-593.
[19]	ZHU Q , FENG J , HUANG J . Natural neighbor:a self-adaptive neighborhood method without parameter K[J]. Pattern Recognition Letters, 2016,80: 30-36.
[20]	CHENG D , ZHU Q , HUANG J ,et al. Natural neighbor-based clustering algorithm with local representatives[J]. Knowledge-Based Systems, 2017,123: 238-253.
[21]	HUANG J , ZHU Q , YANG L ,et al. A non-parameter outlier detection algorithm based on natural neighbor[J]. Knowledge-Based Systems, 2016,92(C): 71-77.
[22]	YANG L , ZHU Q , HUANG J ,et al. Adaptive edited natural neighbor algorithm[J]. Neurocomputing, 2017,230: 427-433.
[23]	NING J , CHEN L , ZHOU C ,et al. Parameter k search strategy in outlier detection[J]. Pattern Recognition Letters, 2018,112: 56-62.
[24]	AUSKALNIS J , PAULAUSKAS N , BASKYS A . Application of local outlier factor algorithm to detect anomalies in computer network[J]. Elektronika Ir Elektrotechnika, 2018,24(3): 96-99.
[25]	NA G S , KIM D , YU H . DILOF:Effective and memory efficient local outlier detection in data streams[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining. New York:ACM Press, 2018: 1993-2002.
[26]	YAO H , FU X , YANG Y ,et al. An incremental local outlier detection method in the data stream[J]. Applied Sciences, 2018,8(8): 1248-1256
[27]	YANG P , WANG D , WEI Z ,et al. An outlier detection approach based on improved self-organizing feature map clustering algorithm[J]. IEEE Access, 2019,7: 115914-115925.
[28]	GAO J , JI W , ZHANG L ,et al. Cube-based incremental outlier detection for streaming computing[J]. Information Sciences, 2020,517: 361-376
[29]	ZHAO Y , HRYNIEWICKI M K . XGBOD:improving supervised outlier detection with unsupervised representation learning[C]// 2018 International Joint Conference on Neural Networks (IJCNN). Piscataway:IEEE Press, 2018: 1-8.

Metrics

Recommended 0

No Suggested Reading articles found!

序号	名称	数据集大小/个	维度	备注
1	数据集1	1 500	2	3个局类簇，60个异常值
2	数据集2	3 600	2	4个局类簇，85个异常值
3	数据集3	1 641	2	3个局类簇，45个异常值
4	数据集4	880	2	5个局类簇，72个异常值
5	数据集5	1 400	2	4个局类簇，100个异常值
6	高斯分布数据集1	40 000	2	添加了5%噪声
7	高斯分布数据集2	40 000	2	添加了5%噪声

数据集名称	训练集/测试集	维度	异常值比例
UCI Letter	6 000/10 000	16	5%异常值
UCI Pendigit	4 435/6 000	16	5%异常值
Smtp	9 598/9 655	36	1.2%异常值
Http	61 886/62 309	36	6%异常值
Credit	142 157/28 480	28	2%异常值
Mnist	12 000/10 000	784	10%异常值

数据集	CFLOF	SVM	iForest
Http	0.859	0.834	0.644
Smtp	0.800	0.814	0.745
Credit	0.762	0.699	0.620
Mnist	0.728	0.628	0.616

Local outlier factor algorithm based on correction of bidirectional neighbor

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 29

Related Articles 2

Metrics

Recommended 0

[1]	Zhongping ZHANG, Weixiong LIU, Yuting ZHANG, Yu DENG, Mianxin WEI. ERDOF: outlier detection algorithm based on entropy weight distance and relative density outlier factor [J]. Journal on Communications, 2021, 42(9): 133-143.
[2]	Yu-fei ZHAO,Gang XIONG,Long-tao HE,Zhou-jun LI. Approach to detecting SQL injection behaviors in network environment [J]. Journal on Communications, 2016, 37(2): 89-98.