On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs

Gefté Almeida; Marcio Pohlmann; Alex Severo; Diego Kreutz; Tiago Heinrich; Lourenço Pereira

On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs

Cryptography and Security 2025-11-20 v1 Artificial Intelligence Machine Learning

Authors: Gefté Almeida , Marcio Pohlmann , Alex Severo , Diego Kreutz , Tiago Heinrich , Lourenço Pereira

Abstract

In this study, we evaluate open-source models for security incident classification, comparing them with proprietary models. We utilize a dataset of anonymized real incidents, categorized according to the NIST SP 800-61r3 taxonomy and processed using five prompt-engineering techniques (PHP, SHP, HTP, PRP, and ZSL). The results indicate that, although proprietary models still exhibit higher accuracy, locally deployed open-source models provide advantages in privacy, cost-effectiveness, and data sovereignty.

Keywords

prompt engineering incident classification SOC CSIRT open-source models security NIST SP 800-61r3

Cite

@article{arxiv.2511.14908,
  title  = {On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs},
  author = {Gefté Almeida and Marcio Pohlmann and Alex Severo and Diego Kreutz and Tiago Heinrich and Lourenço Pereira},
  journal= {arXiv preprint arXiv:2511.14908},
  year   = {2025}
}

Comments

5 pages, 3 figures, 3 tables, submitted to ERRC/WRSeg 2025

On-Premise SLMs vs. Commercial LLMs: Prompt Engineering and Incident Classification in SOCs and CSIRTs

Abstract

Keywords

Cite

Comments

Related papers