Related papers: Dense periodic optimization for countable Markov s…
This note describes sufficient conditions under which total-cost and average-cost Markov decision processes (MDPs) with general state and action spaces, and with weakly continuous transition probabilities, can be reduced to discounted MDPs.…
We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative…
Let $(M,d,\mu)$ be a uniformly discrete metric measure space satisfying space homogeneous volume doubling condition. We consider discrete time Markov chains on $M$ symmetric with respect to $\mu$ and whose one-step transition density is…
Ergodic optimization aims to single out dynamically invariant Borel probability measures which maximize the integral of a given "performance" function. For a continuous self-map of a compact metric space and a dense set of continuous…
We provide a quantification of the uniqueness of Gibbs measure for topologically mixing countable Markov shifts with locally H\"older continuous potentials. Corollaries for speed of convergence for approximation by finite subsystems are…
We construct countable Markov partitions for non-uniformly hyperbolic diffeomorphisms on compact manifolds of any dimension, extending earlier work of O. Sarig for surfaces. These partitions allow us to obtain symbolic coding on invariant…
We consider estimating the transition probability matrix of a finite-state finite-observation alphabet hidden Markov model with known observation probabilities. The main contribution is a two-step algorithm; a method of moments estimator…
We consider the problem of when a symbolic dynamical system supports a Borel probability measure that is invariant under every element of its automorphism group. It follows readily from a classical result of Parry that the full shift on…
For strongly positively recurrent countable state Markov shifts, we bound the distance between an invariant measure and the measure of maximal entropy in terms of the difference of their entropies. This extends an earlier result for…
We introduce a generalization of the Adaptive Multilevel Splitting algorithm in the discrete time dynamic setting, namely when it is applied to sample rare events associated with paths of Markov chains. By interpreting the algorithm as a…
We study the optimization of ergodic averages for multi-valued dynamical systems, i.e. where points may have multiple different forward orbits. Under upper semi-continuity assumptions, we show that the maximum space average with respect to…
Ergodic Optimization is the process of finding invariant probability measures that maximize the integral of a given function. It has been conjectured that "most" functions are optimized by measures supported on a periodic orbit, and it has…
We study relations between transitivity, mixing and periodic points on dendrites. We prove that when there is a point with dense orbit which is not an endpoint, then periodic points are dense and there is a terminal periodic decomposition…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…
Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and non-defective processes and all possible scenarios we identify the corresponding…
We study the optimization of the expected long-term reward in finite partially observable Markov decision processes over the set of stationary stochastic policies. In the case of deterministic observations, also known as state aggregation,…
Calculating averages with respect to multimodal probability distributions is often necessary in applications. Markov chain Monte Carlo (MCMC) methods to this end, which are based on time averages along a realization of a Markov process…
The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We…