Related papers: A derivation of the master equation from path entr…
The stochastic properties of a Langevin-type Markov process can be extracted from a given time series by a Markov analysis. Also processes that obey a stochastically forced second order differential equation can be analyzed this way by…
Time-irreversible stochastic processes are frequently used in natural sciences to explain non-equilibrium phenomena and to design efficient stochastic algorithms. Our main goal in this thesis is to analyse their dynamics by means of large…
Master equations describing open quantum dynamics are typically first order differential equations. When such dynamics brings the trajectories in state space of more than one initial state to the same point at finite instants in time, the…
We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
By modeling the interaction of an open quantum system with its environment through a natural generalization of the classical concept of continuous time random walk, we derive and characterize a class of non-Markovian master equations whose…
In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…
We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension…
Master equations are typically adopted to describe the dynamics of open quantum systems. Such equations are either in integro-differential or in time-local form, with the latter class more frequently adopted due to the simpler numerical…
For a stochastic process $(X_t)_{t\geq 0}$ we establish conditions under which the inverse first-passage time problem has a solution for any random variable $\xi >0$. For Markov processes we give additional conditions under which the…
The principle of maximum irreversible is proved to be a consequence of a stochastic order of the paths inside the phase space; indeed, the system evolves on the greatest path in the stochastic order. The result obtained is that, at the…
The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which detailed balance and time-reversal symmetry are broken. Despite…
Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Non stationary reciprocal processes have been extensively studied in the…
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time noisy algorithms, a prominent analysis technique relies on…
Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their…
Many biological systems can be described by finite Markov models. A general method for simplifying master equations is presented that is based on merging adjacent states. The approach preserves the steady-state probability distribution and…
In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
I formulate an entropy-rate maximization problem at the observable level for stochastic processes observed through an information-reducing observation map. For a visible stationary law, the map determines an observational fiber of hidden…