A Comprehensive Dataset for Human vs. AI Generated Text Detection
Abstract
The rapid advancement of large language models (LLMs) has led to increasingly human-like AI-generated text, raising concerns about content authenticity, misinformation, and trustworthiness. Addressing the challenge of reliably detecting AI-generated text and attributing it to specific models requires large-scale, diverse, and well-annotated datasets. In this work, we present a comprehensive dataset comprising over 73,193 text samples that combine authentic New York Times articles with synthetic versions generated by multiple state-of-the-art LLMs including Gemma-2-9b, Mistral-7B, Qwen-2-72B, LLaMA-8B, Yi-Large, and GPT-4-o. The dataset provides original article abstracts as prompts, full human-authored narratives. We establish baseline results for two key tasks: distinguishing human-written from AI-generated text, achieving an accuracy of 58.35\%, and attributing AI texts to their generating models with an accuracy of 8.92\%. By bridging real-world journalistic content with modern generative models, the dataset aims to catalyze the development of robust detection and attribution methods, fostering trust and transparency in the era of generative AI. Our dataset is available at: https://huggingface.co/datasets/Rajarshi-Roy-research/Defactify_Text_Dataset
Cite
@article{arxiv.2510.22874,
title = {A Comprehensive Dataset for Human vs. AI Generated Text Detection},
author = {Rajarshi Roy and Gurpreet Singh and Ashhar Aziz and Shashwat Bajpai and Nasrin Imanpour and Shwetangshu Biswas and Kapil Wanaskar and Parth Patwa and Subhankar Ghosh and Shreyas Dixit and Nilesh Ranjan Pal and Vipula Rawte and Ritvik Garimella and Gaytri Jena and Amitava Das and Amit Sheth and Vasu Sharma and Aishwarya Naresh Reganti and Vinija Jain and Aman Chadha},
journal= {arXiv preprint arXiv:2510.22874},
year = {2026}
}
Comments
Defactify4 @AAAI 2025