The proliferation of Large Language Models (LLMs) poses challenges in detecting and mitigating digital deception, as these models can emulate human conversational patterns and facilitate chat-based social engineering (CSE) attacks. This study investigates the dual capabilities of LLMs as both facilitators and defenders against CSE threats. We develop a novel dataset, SEConvo, simulating CSE scenarios in academic and recruitment contexts, and designed to examine how LLMs can be exploited in these situations. Our findings reveal that, while off-the-shelf LLMs generate high-quality CSE content, their detection capabilities are suboptimal, leading to increased operational costs for defense. In response, we propose ConvoSentinel, a modular defense pipeline that improves detection at both the message and the conversation levels, offering enhanced adaptability and cost-effectiveness. The retrieval-augmented module in ConvoSentinel identifies malicious intent by comparing messages to a database of similar conversations, enhancing CSE detection at all stages. Our study highlights the need for advanced strategies to leverage LLMs in cybersecurity.
@article{arxiv.2406.12263,
title = {Defending Against Social Engineering Attacks in the Age of LLMs},
author = {Lin Ai and Tharindu Kumarage and Amrita Bhattacharjee and Zizhou Liu and Zheng Hui and Michael Davinroy and James Cook and Laura Cassani and Kirill Trapeznikov and Matthias Kirchner and Arslan Basharat and Anthony Hoogs and Joshua Garland and Huan Liu and Julia Hirschberg},
journal= {arXiv preprint arXiv:2406.12263},
year = {2024}
}