Related papers: A pseudo-Markov property for controlled diffusion …
Dynamic heterogeneity has often been modeled by assuming that a single-particle observable, fluctuating at a molecular scale, is influenced by its coupling to environmental variables fluctuating on a second, perhaps slower, time scale.…
A simple model of an irreversible process is introduced. The equation of iterations in the model includes a noise generation term. We study the properties of the system when the noise generation term is a stochastic process (e.g. a random…
We consider stochastic thermodynamics as a theory of statistical inference for experimentally observed fluctuating time-series. To that end, we introduce a general framework for quantifying the knowledge about the dynamical state of the…
We propose a parallel algorithm for the numerical solution of a class of second order semi-linear equations coming from stochastic optimal control problems, by means of a dynamic domain decomposition technique. The new method is an…
Suppose X is a multivariate diffusion process that is observed discretely in time. At each observation time, a transformation of the state of the process is observed with noise. The smoothing problem consists of recovering the path of the…
In this work, we consider the controllability of a discrete-time linear dynamical system with sparse control inputs. Sparsity constraints on the input arises naturally in networked systems, where activating each input variable adds to the…
Stochastic systems have a control-theoretic interpretation in which noise plays the role of control. In the weak-noise limit, relevant at low temperatures or in large populations, this leads to a precise mathematical mapping: the most…
As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow…
Simulated annealing solves global optimization problems by means of a random walk in a cooling energy landscape based on the objective function and a temperature parameter. However, if the temperature is decreased too quickly, this…
We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed…
Semi-Markov processes play an important role in the effective description of partially accessible systems in stochastic thermodynamics. They occur, for instance, in coarse-graining procedures such as state lumping and when analyzing waiting…
We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…
A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…
The stochastic properties of a Langevin-type Markov process can be extracted from a given time series by a Markov analysis. Also processes that obey a stochastically forced second order differential equation can be analyzed this way by…
In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk…
We consider covariance control problems for nonlinear stochastic systems. Our objective is to find an optimal control strategy to steer the state from an initial distribution to a terminal one with specified mean and covariance. This…
We provide a condition for f-ergodicity of strong Markov processes at a subgeometric rate. This condition is couched in terms of a supermartingale property for a functional of the Markov process. Equivalent formulations in terms of a drift…
The connection between forward backward doubly stochastic differential equations and the optimal filtering problem is established without using the Zakai's equation. The solutions of forward backward doubly stochastic differential equations…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
A discrete time branching process where the offspring distribution is generation-dependent, and the number of reproductive individuals is controlled by a random mechanism is considered. This model is a Markov chain but, in general, the…