English

Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering

Computation and Language 2025-09-30 v1 Machine Learning

Abstract

Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages.

Keywords

Cite

@article{arxiv.2509.23715,
  title  = {Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering},
  author = {Eduard Barbu and Adrian Marius Dumitran},
  journal= {arXiv preprint arXiv:2509.23715},
  year   = {2025}
}

Comments

Accepted@ CONSILR 2025 Bucharest Romania 9-10 October