Related papers: Trading Performance for Stability in Markov Decisi…
We investigate stability of a solution of a hybrid system in the sense that the graphs of solutions from nearby initial conditions remain close and tend towards the graph of the given solution. In this manner, a small continuous-time…
This tutorial describes recently developed general optimality conditions for Markov Decision Processes that have significant applications to inventory control. In particular, these conditions imply the validity of optimality equations and…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
Typically, it is desirable to design a control system that is not only robustly stable in the presence of parametric uncertainties but also guarantees an adequate level of system performance. However, most of the existing methods need to…
We present the conditional value-at-risk (CVaR) in the context of Markov chains and Markov decision processes with reachability and mean-payoff objectives. CVaR quantifies risk by means of the expectation of the worst p-quantile. As such it…
An algorithm is proposed for computing equilibrium averages of Markov chains which suffer from metastability -- the tendency to remain in one or more subsets of state space for long time intervals. The algorithm, called the parallel replica…
We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…
Algorithmic decision-making in high-stakes settings can have profound impacts on individuals and populations. While much prior work studies fairness in static settings, recent results show that enforcing static fairness constraints may…
In the application of machine learning to real-life decision-making systems, e.g., credit scoring and criminal justice, the prediction outcomes might discriminate against people with sensitive attributes, leading to unfairness. The commonly…
We consider a nonlinear pricing environment with private information. We provide profit guarantees (and associated mechanisms) that the seller can achieve across all possible distributions of willingness to pay of the buyers. With a…
When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far…
For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step $t_0$, the decision maker formulates an optimization…
The problem of controlling hybrid dynamical systems using model predictive control (MPC) is formulated and sufficient conditions for asymptotic stability of a set are provided. Hybrid dynamical systems are modeled in terms of hybrid…
For Markov processes over discrete configurations, an asymptotic bound on the uncertainty of stochastic fluxes is derived in terms of the harmonic mean of decay rates with respect to the stationary distribution. This bound is necessarily…
Markov chain Monte Carlo (MCMC) methods to sample from a probability distribution $\pi$ defined on a space $(\Theta,\mathcal{T})$ consist of the simulation of realisations of Markov chains $\{\theta_{n},n\geq1\}$ of invariant distribution…
We present a scheme for sequential decision making with a risk-sensitive objective and constraints in a dynamic environment. A neural network is trained as an approximator of the mapping from parameter space to space of risk and policy with…
We propose and analyze a framework for discrete-time robust mean-field control problems under common noise uncertainty. In this framework, the mean-field interaction describes the collective behavior of infinitely many cooperative agents'…
This paper considers a simulation-based estimator for a general class of Markovian processes and explores some strong consistency properties of the estimator. The estimation problem is defined over a continuum of invariant distributions…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
In this paper we derive a novel characterization result for time-consistent stochastic control problems with higher-order moments, originally formulated by Wang et al. [SIAM J. Control. Optim., 63 (2025), 1560--1589], and newly explore many…