Related papers: Optimal time estimation and the clock uncertainty …
We present a general method to identify an arbitrary number of fluctuating quantities which satisfy a detailed fluctuation theorem for all times within the framework of time-inhomogeneous Markovian jump processes. In doing so we provide a…
We study a continuous time stochastic optimal control problem under partial observations that are available only at discrete time instants. This hybrid setting, with continuous dynamics and intermittent noisy measurements, arises in…
A dual formulation for the problem of determining absolute performance limitations on overshoot, undershoot, maximum amplitude and fluctuation minimization for continuous-time feedback systems is constructed. Determining, for example, the…
Estimating entropy production from real observation data can be difficult due to finite resolution in both space and time and finite measurement statistics. We characterize the statistical error introduced by finite sample size and compare…
This paper establishes the first almost sure convergence rate and the first maximal concentration bound with exponential tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain…
We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a…
The use of stochastic models, in effect piecewise deterministic Markov processes (PDMP), has become increasingly popular especially for the modeling of chemical reactions and cell biophysics. Yet, exact simulation methods, for the…
First passage time plays a fundamental role in dynamical characterization of stochastic processes. Crucially, our current understanding on the problem is almost entirely relies on the theoretical formulations, which assume the processes…
The precision and response of trajectory observables offer valuable insights into the behavior of nonequilibrium systems. For classical systems, trade-offs between these characteristics and thermodynamic costs, such as entropy production…
The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article we provide a sufficient stochastic maximum principle for the optimal control…
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
Understanding temporal processes and their correlations in time is of paramount importance for the development of near-term technologies that operate under realistic conditions. Capturing the complete multi-time statistics defining a…
This paper defines theoretical lower bounds of uncertainty of observations of macroeconomic variables that depend on statistical moments and correlations of random values and volumes of market trades. Any econometric assessments of…
We establish general conditions under which there exists uniform in time convergence between a stochastic process and its approximated system. These standardised conditions consist of a local in time estimate between the original and the…
In this paper we study a class of time-inconsistent terminal Markovian control problems in discrete time subject to model uncertainty. We combine the concept of the sub-game perfect strategies with the adaptive robust stochastic to tackle…
Motivated by applications in large-scale and multi-agent reinforcement learning, we study the non-asymptotic performance of stochastic approximation (SA) schemes with delayed updates under Markovian sampling. While the effect of delays has…
Stochastic processes with temporal delay play an important role in science and engineering whenever finite speeds of signal transmission and processing occur. However, an exact mathematical analysis of their dynamics and thermodynamics is…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
In this article we consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises observed in discrete time moments. An adaptive model selection procedure is proposed. A sharp…