Apply Language Nature Rhythm to Large Scale Duplicated Text Detection-Reference-Cited by-同舟云学术

Apply Language Nature Rhythm to Large Scale Duplicated Text Detection

Published:2012-01 Issue: Volume:457-458 Page:635-640
ISSN:1662-8985
Container-title:Advanced Materials Research
language:
Short-container-title:AMR

Author:

Chen Fan¹,Feng Zhi Yong¹,Zhao Geng²

Affiliation:

1. Tianjin University

2. Hebei University of Technology

Abstract

It is urgent that detect the duplication in large scale text in the Web. An arithmetic based on language rhythm for text duplication detection is proposed here. Get the nature rhythm marked by punctuations in text and build the rhythm compare matrix to complete the publication detection for each paragraph. This arithmetic is different with the other one which is based on words analysis. And it has a high accuracy and a low complicacy.

Publisher

Trans Tech Publications, Ltd.

Subject

General Engineering

Link

https://www.scientific.net/AMR.457-458.635.pdf

Reference10 articles.

1. The 25th of China Internet Development Statistics Report http: /www. cnnic. net. cn/html/Dir/2010/01/15/5767. htm . Jan. (2010).

2. N. Shivakumar H.G. Molina. Finding Near-replicas of Documents and Servers on the Web. Proceedings of the International Workshop on World Wide Web and Databases, 1998, Valencia, Spain: 204–212.

3. Manber U Finding similar files in a large file system Proeeedings of the Winter USENIX Conferenee1994 1~10.

4. Brin S, Davis J, Garcia-Molina H. Copy detection mechanisms for digital documents. In：Proceedings of the ACM SIGMOD Annual Conference. 1995. http: /www-db. stanford. edu/pub/brin/1995/copy. ps.

5. Shivakumar N. Garcia-Molina H. SCAM: A copy detection mechanism for digital documents. In: Proceedings of the 2nd International Conference in Theory and Practice of Digital Libraries (DL'95). 1995. http: /www-db. stanford. edu/~shiva/publns. html.