Related papers: Risk Aware Benchmarking of Large Language Models

Reasoning has emerged as the next major frontier for language models (LMs), with rapid advances from both academic and industrial labs. However, this progress often outpaces methodological rigor, with many evaluations relying on…

In this paper, we present a unified framework for decision making under uncertainty. Our framework is based on the composite of two risk measures, where the inner risk measure accounts for the risk of decision given the exact distribution…

Although being a crucial question for the development of machine learning algorithms, there is still no consensus on how to compare classifiers over multiple data sets with respect to several criteria. Every comparison framework is…

Analytical, free of time consuming Monte Carlo simulations, framework for credit portfolio systematic risk metrics calculations is presented. Techniques are described that allow calculation of portfolio-level systematic risk measures…

Higher order risk measures are stochastic optimization problems by design, and for this reason they enjoy valuable properties in optimization under uncertainties. They nicely integrate with stochastic optimization problems, as has been…

Ensuring the reliability and verifiability of large language model (LLM)-enabled systems remains a significant challenge in software engineering. We propose a probabilistic framework for systematically analyzing and improving these systems…

Inspired by widely-used techniques of causal modelling in risk, failure, and accident analysis, this work discusses a compositional framework for risk modelling. Risk models capture fragments of the space of risky events likely to occur…

High precision analytical approximation is proposed for variance-covariance based risk allocation in a portfolio of risky assets. A general case of a single-period multi-factor Merton-type model with stochastic recovery is considered. The…

Large language models are reshaping quantitative investing by turning unstructured financial information into evidence-grounded signals and executable decisions. This survey synthesizes research with a focus on equity return prediction and…

In statistical exercises where there are several candidate models, the traditional approach is to select one model using some data driven criterion and use that model for estimation, testing and other purposes, ignoring the variability of…

We propose a risk measurement approach for a risk-averse stochastic problem. We provide results that guarantee that our problem has a solution. We characterize and explore the properties of the argmin as a risk measure and the minimum as a…

Models continue to increase their already broad use across industry as well as their sophistication. Worldwide regulation oblige financial institutions to manage and address model risk with the same severity as any other type of risk, which…

Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to…

Large language model safety is usually assessed with static benchmarks, but key failures are dynamic: value drift under distribution shift, jailbreak attacks, and slow degradation of alignment in deployment. Building on a recent Second Law…

This study introduces a benchmark framework for evaluating the financial decision-making capabilities of large language models (LLMs) through portfolio optimization problems with mathematically explicit solutions. Unlike existing financial…

We study mean-risk optimal portfolio problems where risk is measured by Recovery Average Value at Risk, a prominent example in the class of recovery risk measures. We establish existence results in the situation where the joint distribution…

Currently, legal requirements demand that insurance companies increase their emphasis on monitoring the risks linked to the underwriting and asset management activities. Regarding underwriting risks, the main uncertainties that insurers…

Measuring and managing risk has become crucial in modern decision making under stochastic uncertainty. In two-stage stochastic programming, mean risk models are essentially defined by a parametric recourse problem and a quantification of…

This paper develops an axiomatic framework for ranking metrics, a general class of functionals for evaluating and ordering financial or insurance positions. Unlike traditional risk-adjusted performance measures-such as the Sharpe ratio,…

System modeling is a classical approach to ensure their reliability since it is suitable both for a formal verification and for software testing techniques. In the context of model-based testing an approach combining random testing and…