Uncertainty quantification in fine-tuned LLMs using LoRA ensembles

Oleksandr Balabanov; Hampus Linander

Uncertainty quantification in fine-tuned LLMs using LoRA ensembles

Machine Learning 2025-05-22 v2 Artificial Intelligence Computation and Language Machine Learning

Authors: Oleksandr Balabanov , Hampus Linander

Abstract

Fine-tuning large language models can improve task specific performance, although a general understanding of what the fine-tuned model has learned, forgotten and how to trust its predictions is still missing. We derive principled uncertainty quantification for fine-tuned LLMs with posterior approximations using computationally efficient low-rank adaptation ensembles. We analyze three common multiple-choice datasets using low-rank adaptation ensembles based on Mistral-7b, and draw quantitative and qualitative conclusions on their perceived complexity and balance between retained prior knowledge and domain specific adaptation during and after fine-tuning. We identify unexpected retention of acquired knowledge during fine-tuning in the overfitting regime.

Keywords

parameter-efficient fine-tuning instruction tuning uncertainty quantification

Cite

@article{arxiv.2402.12264,
  title  = {Uncertainty quantification in fine-tuned LLMs using LoRA ensembles},
  author = {Oleksandr Balabanov and Hampus Linander},
  journal= {arXiv preprint arXiv:2402.12264},
  year   = {2025}
}

Comments

Accepted for ICLR2025 Workshop "Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI"

Uncertainty quantification in fine-tuned LLMs using LoRA ensembles

Abstract

Keywords

Cite

Comments

Related papers