Linearithmic Corpus to Corpus Comparison by Sentence Hashing Algorithm SHAPD2

摘要

This work presents an innovative method of comparing sets of textual documents with an aim to identify common phrase sequences. The SHAPD2 (Sentence Hashing Algorithm for Plagiarism Detection 2) algorithm was designed to achieve the goal of a single-pass corpus to corpus comparison. The algorithm was developed taking into account results and observations from previous research activities. It is a highly efficient solution that finds application with considerable amounts of data and excels over other approaches. One of its possible applications is detection of potential plagiarisms comparing not a document against a corpus, but corpus to corpus. Algorithm's performance allows for applications in situations where results have to be served an instant after issuing a query. This makes the SHAPD2 algorithm a valuable alternative to the available solutions.