Related papers: Dense periodic optimization for countable Markov s…
We prove a generalised Yuan--Hunt--Ma\~n\'e Conjecture: if $\mathcal{F}$ is the Banach space of $\alpha$-H\"older functions, and $\mathcal{T}$ is either a space of Lipschitz expanding maps, or of Anosov diffeomorphisms, or the family of…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
We study various aspects of periodic points for random substitution subshifts. In order to do so, we introduce a new property for random substitutions called the disjoint images condition. We provide a procedure for determining the property…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
In this work, we treat subshifts, defined in terms of an alphabet $A$ and (usually infinite) forbidden list $F$, where the number of $n$-letter words in $F$ has "slow growth rate" in $n$. We show that such subshifts are well-behaved in…
We deal with countable alphabet locally compact random subshifts of finite type (the latter merely meaning that the symbol space is generated by an incidence matrix) under the absence of Big Images Property and under the absence of uniform…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
The parametrization of adiabatic paths is optimal when tunneling is minimized. Hamiltonian evolutions do not have unique optimizers. However, dephasing Lindblad evolutions do. The optimizers are simply characterized by an Euler-Lagrange…
We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its…
This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…
We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards. Consequently, observing a particular state transition might…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
We develop in this paper a new framework for discrete calculus of variations when the actions have densities involving an arbitrary discretization operator. We deduce the discrete Euler-Lagrange equations for piecewise continuous critical…
This paper aims to better understand the link better understand the links between aperiodicity in subshifts and pattern complexity. Our main contribution deals with substitutive subshifts, an equivalent to substitutive tilings in the…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
This article describes a method for using optimization to derive efficient independent transition functions for Markov chain Monte Carlo simulations. Our interest is in sampling from a posterior density $\pi(x)$ for problems in which the…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
For a dynamical system, we study the set of points $\cal W$ whose orbit approximates any chosen point at certain specified rates. Our basic setting is that of left shift acting on topological Markov chains endowed with a local weak Gibbs…
This article is concerned with sampling from Gibbs distributions $\pi(x)\propto e^{-U(x)}$ using Markov chain Monte Carlo methods. In particular, we investigate Langevin dynamics in the continuous- and the discrete-time setting for such…