Related papers: Risk Aware Benchmarking of Large Language Models

This paper addresses the challenge of model uncertainty in quantitative finance, where decisions in portfolio allocation, derivative pricing, and risk management rely on estimating stochastic models from limited data. In practice, the…

This study addresses the reliability of automatic summarization in high-risk scenarios and proposes a large language model framework that integrates uncertainty quantification and risk-aware mechanisms. Starting from the demands of…

Uncertainty is prevalent in engineering design, data-driven problems, and decision making broadly. Due to inherent risk-averseness and ambiguity about assumptions, it is common to address uncertainty by formulating and solving conservative…

Large language model (LLM) benchmarks inform LLM use decisions (e.g., "is this LLM safe to deploy for my use case and context?"). However, benchmarks may be rendered unreliable by various failure modes that impact benchmark bias, variance,…

Managing a portfolio to a risk model can tilt the portfolio toward weaknesses of the model. As a result, the optimized portfolio acquires downside exposure to uncertainty in the model itself, what we call "second order risk." We propose a…

Portfolio optimization methods have evolved significantly since Markowitz introduced the mean-variance framework in 1952. While the theoretical appeal of this approach is undeniable, its practical implementation poses important challenges,…

We develop a new classification framework based on the theory of coherent risk measures and systemic risk. The proposed approach is suitable for multi-class problems when the data is noisy, scarce (relative to the dimension of the problem),…

A risk analyst assesses potential financial losses based on multiple sources of information. Often, the assessment does not only depend on the specification of the loss random variable but also various economic scenarios. Motivated by this…

We develop a generalized stability framework for stochastic discrete-time systems, where the generality pertains to the ways in which the distribution of the state energy can be characterized. We use tools from finance and operations…

We propose an computational framework for real-time risk assessment and prioritizing for random outcomes without prior information on probability distributions. The basic model is built based on satisficing measure (SM) which yields a…

We explore a stochastic model that enables capturing external influences in two specific ways. The model allows for the expression of uncertainty in the parametrisation of the stochastic dynamics and incorporates patterns to account for…

Uncertainty quantification is a critical aspect of machine learning models, providing important insights into the reliability of predictions and aiding the decision-making process in real-world applications. This paper proposes a novel way…

Robust estimation for modern portfolio selection on a large set of assets becomes more important due to large deviation of empirical inference on big data. We propose a distributionally robust methodology for high-dimensional mean-variance…

We develop an averaging approach to robust risk measurement under payoff uncertainty. Instead of taking a worst-case value over an uncertainty neighborhood, we weight nearby payoffs more heavily under a chosen metric and average the…

The recent explosion in the capabilities of large language models has led to a wave of interest in how best to prompt a model to perform a given task. While it may be tempting to simply choose a prompt based on average performance on a…

We study the consistency of sample mean-variance portfolios of arbitrarily high dimension that are based on Bayesian or shrinkage estimation of the input parameters as well as weighted sampling. In an asymptotic setting where the number of…

We introduce a new approach for prudent risk evaluation based on stochastic dominance, which will be called the model aggregation (MA) approach. In contrast to the classic worst-case risk (WR) approach, the MA approach produces not only a…

Alignment of large language models (LLMs) typically involves training a reward model on preference data, followed by policy optimization with respect to the reward model. However, optimizing policies with respect to a single reward model…

Risk management often plays an important role in decision making under uncertainty. In quantitative risk management, assessing and optimizing risk metrics requires efficient computing techniques and reliable theoretical guarantees. In this…

Beta-sorted portfolios -- portfolios comprised of assets with similar covariation to selected risk factors -- are a popular tool in empirical finance to analyze models of (conditional) expected returns. Despite their widespread use, little…