The increasing reliance on large language models (LLMs) in academic writing has led to a rise in plagiarism. Existing AI-generated text classifiers have limited accuracy and often produce false positives. We propose a novel approach using natural language processing (NLP) techniques, offering quantifiable metrics at both sentence and document levels for easier interpretation by human evaluators. Our method employs a multi-faceted approach, generating multiple paraphrased versions of a given question and inputting them into the LLM to generate answers. By using a contrastive loss function based on cosine similarity, we match generated sentences with those from the student's response. Our approach achieves up to 94% accuracy in classifying human and AI text, providing a robust and adaptable solution for plagiarism detection in academic settings. This method improves with LLM advancements, reducing the need for new model training or reconfiguration, and offers a more transparent way of evaluating and detecting AI-generated text.
@article{arxiv.2306.08122,
title = {Beyond Black Box AI-Generated Plagiarism Detection: From Sentence to Document Level},
author = {Mujahid Ali Quidwai and Chunhui Li and Parijat Dube},
journal= {arXiv preprint arXiv:2306.08122},
year = {2023}
}
Comments
10 Pages, 4 Figures, 9 Tables, to be published in 18th Workshop on Innovative Use of NLP for Building Educational Applications