基于斜率密度聚类的相似文本标定

doi:10.3969/j.issn.1000-436x.2013.Z2.030

通信学报 ›› 2013, Vol. 34 ›› Issue (Z2): 157-162.doi: 10.3969/j.issn.1000-436x.2013.Z2.030

• 数字化校园与典型应用 • 上一篇下一篇

基于斜率密度聚类的相似文本标定

邹杜¹,唐文军¹,龙卫江²,张凌³

¹ 华南理工大学信息网络工程研究中心，广东广州 510640
² 华南理工大学理学院，广东广州 510640
³ 华南理工大学计算机学院，广东广州 510640

出版日期:2013-12-25 发布日期:2017-06-16
基金资助:
国家自然科学基金资助项目

Similar text positioning method based on slope-density cluster

Du ZOU¹,Wen-jun TANG¹,Wei-jiang LONG²,Ling ZHANG³

¹ Information Network Engineering and Research Center,South China University of Technology,Guangzhou 510640,China
² School of Science,South China University of Technology,Guangzhou 510640,China
³ School of Computer Science ＆Engineering,South China University of Technology,Guangzhou 510640,China

Online:2013-12-25 Published:2017-06-16
Supported by:
The National Natural Science Foundation of China

摘要/Abstract

摘要：

相似文本标定是抄袭检测的一个重要环节，现有标定方法大多采用直接对文本或指纹进行合井的方式，标定精度受干扰信息影响较大。针对这种局限性，分析了匹配指纹对的语义特征，提出基于斜率密度的相似文本聚类方法，将文本匹配合井问题转化成稠密样本点聚类问题，井在 PAN 公用语料库上对该方法进行了测试，得到的主要指标优于 PAN10 前 3 名。目前已将该方法用于华南理工大学特色专业教学平台的作业查抄，取得了较好的效果。

关键词: 抄袭检测, 相似文本标定, 聚类, 指纹

Abstract:

Similar text positioning is an important part of plagiarism detection.The existing positioning method directly merges text or fingerprint to obtain similar text.Due to the disturb information in the similar text,the positioning accuracy is poor.The semantic features of the match fingerprints were analyzed,and a cluster method based on slope density for similar text positioning was proposed,which converts the text merge problem into dense sample points clustering problem,and improves the efficiency and accuracy of the positioning.Through the experiment on the PAN public corpus,the result shows it performs better than the PAN10 top three.This method has been used in the South China University of Technology 's feature professional teaching platform to detect the plagiarism of homework.

Key words: plagiarism detection, similar text positioning, cluster, fingerprint

邹杜,唐文军,龙卫江,张凌. 基于斜率密度聚类的相似文本标定[J]. 通信学报, 2013, 34(Z2): 157-162.

Du ZOU,Wen-jun TANG,Wei-jiang LONG,Ling ZHANG. Similar text positioning method based on slope-density cluster[J]. Journal on Communications, 2013, 34(Z2): 157-162.

图/表 4

参考文献 10

[11]	KASPRZAK J , BRANDEJS M , KRIPAC M . Finding plagiarism by evaluating document similarities[A]. 3rd PAN Workshop Uncovering Plagiarism,Authorship and Social Software Misuse[C]. 2009. 32-36.
[12]	ZOU D , LONG WJ , ZHANG L . A two-phase plagiarism detection method[A]. 2011 International Conference on Internet Technology and Applications[C]. Wuhan,China, 2011. 1-4.
[13]	POTTHAST M . Overview of the 1st International Competition on Plagiarism Detection[A]. 3rd PAN Workshop Uncovering Plagiarism,Authorship and Social Software Misuse[C]. 2009. 1-9.
[1]	MANBER U . Finding similar files in a large file system[A]. Proceedings of the Winter USENIX Conference[C]. San Francisco,CA, 1994. 1-10.
[2]	BRIN S , DAVIS J,GARCIA-MOLINA H . Copy detection mechanisms for digital documents[A]. Proceedings of the ACM SIGMOD Conference on Management of Data[C]. ACM,NY,USA, 1995. 126-141.
[3]	BRODER A Z . On the resemblance and containment of documents[A]. Proceedings of Compression and Complexity of Sequences 1997[C]. Salemo,Italy, 1997. 21-29.
[4]	BRODER A Z . Identifying and filtering near-duplicate documents[A]. Proceedings of Combinatorial Pattern Matching[C]. Springer Berlin Heidelberg,Germany, 2000. 1-10.
[5]	STEIN B , POTTHAST M . Applying Hash-based indexing in text-based information retrieval[A]. Proceedings of The 7th DutchBelgian Information Retrieval Workshop (DIR 07)[C]. Leuven,Belgium, 2007. 29-35.
[6]	CHARIKAR M S . Similarity estimation techniques from rounding algorithms[A]. Proceedings of The Thiry-Fourth Annual ACM Symposium on Theory of Computing[C]. ACM,NY,USA, 2002. 380-388.
[7]	SHIVAKUMAR N GARCIA-MOLINA H . Building a scalable and accurate copy detection mechanism[A]. Proceedings of The First ACM International Conference on Digital Libraries[C]. ACM,NY,USA, 1996. 160-168.
[8]	SCHLEIMER S , WILKERSON D S , AIKEN A . Winnowing:local algorithms for document fingerprinting[A]. Proceedings of The 2003 ACM SIGMOD International Conference on Management of Data[C]. ACM,New York, 2003. 76-85.
[9]	SEDIYONO A , MAHAMUD K R K . Algorithm of the Longest Commonly Consecutive Word for Plagiarism Detection in Text Based Document[A]. Proceedings of The Third International Conference on Digital Information Management[C]. London,UK, 2008. 253-259.
[10]	ZASLAVSKY A , BIA A , MONOSTORI K . Using copy-detection and text comparison algorithms for cross-referencing multiple editions of literary works[A]. Research and Advanced Technology for Digital Libraries[C]. Springer Berlin Heidelberg,Germany, 2001. 103-114.

[1]	马玲, 樊漆亮, 许婷, 郭冠琛, 张圣林, 孙永谦, 张玉志. 基于强化学习的在线离线混部云环境下的调度框架[J]. 通信学报, 2023, 44(6): 90-102.
[2]	余晟兴, 陈泽凯, 陈钟, 刘西蒙. DAGUARD：联邦学习下的分布式后门攻击防御方案[J]. 通信学报, 2023, 44(5): 110-122.
[3]	董煜, 张友鹏. 基于聚类赋权的冲突证据组合方法[J]. 通信学报, 2023, 44(3): 157-163.
[4]	袁程胜, 郭强, 付章杰. 基于差分隐私的深度伪造指纹检测模型版权保护算法[J]. 通信学报, 2022, 43(9): 181-193.
[5]	赵静, 李俊, 龙春, 万巍, 魏金侠, 陈凯. 基于多层次特征的RoQ隐蔽攻击无监督检测方法[J]. 通信学报, 2022, 43(9): 224-239.
[6]	李骜, 冯聪, 牛宇童, 徐士彪, 张英涛, 孙广路. 面向视角非对齐数据的多视角聚类方法[J]. 通信学报, 2022, 43(7): 143-152.
[7]	赵增华, 童跃凡, 崔佳洋. 基于域自适应的Wi-Fi指纹设备无关室内定位模型[J]. 通信学报, 2022, 43(4): 143-153.
[8]	毛伊敏, 甘德瑾, 廖列法, 陈志刚. 基于Spark框架和ASPSO的并行划分聚类算法[J]. 通信学报, 2022, 43(3): 148-163.
[9]	张忠平, 李森, 刘伟雄, 刘书霞. 基于快速密度峰值聚类离群因子的离群点检测算法[J]. 通信学报, 2022, 43(10): 186-195.
[10]	王惠琴, 侯文斌, 彭清斌, 曹明华, 黄瑞, 刘玲. 基于K均值聚类的SPPM分步分类检测算法[J]. 通信学报, 2022, 43(1): 161-171.
[11]	胡强, 沈嘉吉, 荆广辉, 杜军威. 基于描述语境特征词与改进GSDMM模型的服务聚类方法[J]. 通信学报, 2021, 42(8): 176-187.
[12]	贾春福, 李瑞琪, 王雅飞. 基于同态加密的DBSCAN聚类隐私保护方案[J]. 通信学报, 2021, 42(2): 1-11.
[13]	刘留, 张建华, 樊圆圆, 于力, 张嘉驰. 机器学习在信道建模中的应用综述[J]. 通信学报, 2021, 42(2): 134-153.
[14]	刘桂岐, 钱志鸿, 李华亮, 孙佳妮, 冯一诺, 王雪. 基于有效AP选择和多分类LDA的室内定位算法[J]. 通信学报, 2021, 42(11): 109-120.
[15]	周牧, 龙玥辛, 蒲巧林, 王勇, 何维. 基于CSI张量分解的室内Wi-Fi指纹定位方法[J]. 通信学报, 2021, 42(11): 159-171.

基于斜率密度聚类的相似文本标定

Similar text positioning method based on slope-density cluster

在线阅读

PDF下载

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 10

相关文章 15

Metrics

推荐阅读 0