Related papers: Computing the minimal rebinding effect for non-rev…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
Stochastic resetting, where a dynamical process is intermittently returned to a fixed reference state, has emerged as a powerful mechanism for optimizing first-passage properties. Existing theory largely treats static, non-learning…
We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter…
Stochastic processes on graphs can describe a great variety of phenomena ranging from neural activity to epidemic spreading. While many existing methods can accurately describe typical realizations of such processes, computing properties of…
We investigate the shrinking target and recurrence set associated to non-autonomous measure-preserving systems on compact metric spaces, establishing zero-one criteria in the spirit of classical Borel-Cantelli results. Our first main…
We investigate the problem of monitoring partially observable systems with nondeterministic and probabilistic dynamics. In such systems, every state may be associated with a risk, e.g., the probability of an imminent crash. During runtime,…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
The identification of states and parameters from noisy measurements of a dynamical system is of great practical significance and has received a lot of attention. Classically, this problem is expressed as optimization over a class of models.…
This paper focuses on recursive estimation of time varying autoregressive processes in a nonparametric setting. The stability of the model is revisited and uniform results are provided when the time-varying autoregressive parameters belong…
The underdamped, non-linear, generalized Langevin equation is widely used to model coarse-grained dynamics of soft and biological materials. By means of a projection operator formalism, we show under which approximations this equation can…
We study the problem of zero-delay coding for the transmission of a Markov source over a noisy channel with feedback and present a reinforcement learning solution which is guaranteed to achieve near-optimality. To this end, we formulate the…
Understanding temporal processes and their correlations in time is of paramount importance for the development of near-term technologies that operate under realistic conditions. Capturing the complete multi-time statistics defining a…
The developing of (non-Markovian) memory effects strongly depends on the underlying system-environment dynamics. Here we study this problem in multipartite arrangements where all subsystems are coupled to each other by non-diagonal…
We study square loss in a realizable time-series framework with martingale difference noise. Our main result is a fast rate excess risk bound which shows that whenever a trajectory hypercontractivity condition holds, the risk of the…
A tripartite state $\rho_{ABC}$ forms a Markov chain if there exists a recovery map $\mathcal{R}_{B \to BC}$ acting only on the $B$-part that perfectly reconstructs $\rho_{ABC}$ from $\rho_{AB}$. To achieve an approximate reconstruction, it…
This paper investigates a class of optimal control problems associated with Markov processes with local state information. The decision-maker has only local access to a subset of a state vector information as often encountered in…
This paper presents a state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspond to the minimum number of…
We consider a walker moving in a one-dimensional interval with absorbing boundaries under the effect of Markovian resettings to the initial position. The walker's motion follows a random walk characterized by a general waiting time…
This brief paper presents simple simulation-based algorithms for obtaining an approximately optimal policy in a given finite set in large finite constrained Markov decision processes. The algorithms are adapted from playing strategies for…
Consider the global optimisation of a function $U$ defined on a finite set $V$ endowed with an irreducible and reversible Markov generator.By integration, we extend $U$ to the set $\mathcal{P}(V)$ of probability distributions on $V$ and we…