Related papers: Optimal time estimation and the clock uncertainty …
For Markov processes over discrete configurations, an asymptotic bound on the uncertainty of stochastic fluxes is derived in terms of the harmonic mean of decay rates with respect to the stationary distribution. This bound is necessarily…
The exploration of far-from-equilibrium systems has been at the forefront of nonequilibrium thermodynamics, with a particular focus on understanding the fluctuations and response of thermodynamic systems to external perturbations. In this…
We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…
We study the estimation of parameters pertaining to non-Markovian quantum open systems, such as the dissipation rate and environmental memory time. A key challenge is identifying the optimal measurement time, which must allow sufficient…
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…
The intuition that the precision of observables is constrained by thermodynamic costs has recently been formalized through thermodynamic and kinetic uncertainty relations. While such trade-offs have been extensively studied in Markovian…
Tasks that require information about the world imply a trade-off between the time spent on observation and the variance of the response. In particular, fast decisions need to rely on uncertain information. However, standard estimates of…
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…
Observing stochastic trajectories with rare transitions between states, practically undetectable on time scales accessible to experiments, makes it impossible to directly quantify the entropy production and thus infer whether and how far…
This work addresses stochastic optimal control problems where the unknown state evolves in continuous time while partial, noisy, and possibly controllable measurements are only available in discrete time. We develop a framework for…
Recent large deviation results have provided general lower bounds for the fluctuations of time-integrated currents in the steady state of stochastic systems. A corollary are so-called thermodynamic uncertainty relations connecting precision…
We derive an inequality relating the finite-frequency linear response and fluctuations of an observable in a physical system. The relation holds for arbitrary observables and perturbations in general Markovian dynamics, including over- and…
Linear two-timescale stochastic approximation (SA) scheme is an important class of algorithms which has become popular in reinforcement learning (RL), particularly for the policy evaluation problem. Recently, a number of works have been…
We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
Peak estimation bounds extreme values of a function of state along trajectories of a dynamical system. This paper focuses on extending peak estimation to continuous and discrete settings with time-independent and time-dependent uncertainty.…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
Relative fluctuations of observables in discrete stochastic systems are bounded at all times by the mean dynamical activity in the system, quantified by the mean number of jumps. This constitutes a kinetic uncertainty relation that is…
We have shown how the intrinsic properties of a noise process can set an upper bound for the time derivative of entropy in a nonequilibrium system. The interplay of dissipation and the properties of noise processes driving the dynamical…