English

Trust by Design: Skill Profiles for Transparent, Cost-Aware LLM Routing

Artificial Intelligence 2026-02-03 v1 Information Retrieval Machine Learning

Abstract

How should Large Language Model (LLM) practitioners select the right model for a task without wasting money? We introduce BELLA (Budget-Efficient LLM Selection via Automated skill-profiling), a framework that recommends optimal LLM selection for tasks through interpretable skill-based model selection. Standard benchmarks report aggregate metrics that obscure which specific capabilities a task requires and whether a cheaper model could suffice. BELLA addresses this gap through three stages: (1) decomposing LLM outputs and extract the granular skills required by using critic-based profiling, (2) clustering skills into structured capability matrices, and (3) multi-objective optimization to select the right models to maximize performance while respecting budget constraints. BELLA provides natural-language rationale for recommendations, providing transparency that current black-box routing systems lack. We describe the framework architecture, situate it within the landscape of LLM routing and evaluation, and discuss its application to financial reasoning as a representative domain exhibiting diverse skill requirements and cost-variation across models. Our framework enables practitioners to make principled and cost-performance trade-offs for deploying LLMs.

Keywords

Cite

@article{arxiv.2602.02386,
  title  = {Trust by Design: Skill Profiles for Transparent, Cost-Aware LLM Routing},
  author = {Mika Okamoto and Ansel Kaplan Erol and Glenn Matlin},
  journal= {arXiv preprint arXiv:2602.02386},
  year   = {2026}
}

Comments

Appeared at MLSys YPS 2025

R2 v1 2026-07-01T09:32:23.696Z