Related papers: Ergodic-risk Criterion for Stochastically Stabiliz…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

Policy optimization algorithms are crucial in many fields but challenging to grasp and implement, often due to complex calculations related to Markov decision processes and varying use of discount and average reward setups. This paper…

Stochastic approximation (SA) is an iterative algorithm for finding the fixed point of an operator using noisy samples and widely used in optimization and Reinforcement Learning (RL). The noise in RL exhibits a Markovian structure, and in…

In this article, we study the ergodic risk-sensitive control problem for controlled regime-switching diffusions. Under a blanket stability hypothesis, we solve the associated nonlinear eigenvalue problem for weakly coupled systems and…

The main challenge for adaptive regulation of linear-quadratic systems is the trade-off between identification and control. An adaptive policy needs to address both the estimation of unknown dynamics parameters (exploration), as well as the…

In this paper we study infinite horizon nonzero-sum stochastic games for controlled discrete-time Markov chains on a Polish state space with risk-sensitive ergodic cost criterion. Under suitable assumptions we show that the associated…

In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision u_n is made as a function of past observations X_0,...,X_{n-1}, and a loss l(u_n,X_n) is incurred. In this setting, it is known that…

To profit from price oscillations, investors frequently use threshold-type strategies where changes in the portfolio position are triggered by some indicators reaching prescribed levels. In this paper, we investigate threshold-type…

In this paper we consider the problem of obtaining sharp bounds for the performance of temporal difference (TD) methods with linear function approximation for policy evaluation in discounted Markov decision processes. We show that a simple…

Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…

We consider stationary stochastic dynamical systems evolving on a compact metric space, by perturbing a deterministic dynamics with a random noise, added according to an arbitrary probabilistic distribution. We prove the maximal and…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…

The entropic risk measure is widely used in high-stakes decision-making across economics, management science, finance, and safety-critical control systems because it captures tail risks associated with uncertain losses. However, when data…

Two-timescale stochastic approximation (TTSA) is among the most general frameworks for iterative stochastic algorithms. This includes well-known stochastic optimization methods such as SGD variants and those designed for bilevel or minimax…

In this paper, we concern with the ergodic linear-quadratic closed-loop optimal control problems, in which the state equation is the mean-field stochastic differential equation with periodic coefficients. We first study the asymptotic…

This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…

Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their…

We consider a stationary regularly varying time series which can be expressedas a function of a geometrically ergodic Markov chain. We obtain practical conditionsfor the weak convergence of the tail array sums and feasible estimators…

We study a regulation problem for stochastic systems subject to both continuous fluctuations and rare but significant shocks, modeled as a jump-diffusion with uncertainty in both the drift and the jump intensity. Such settings arise in…