Related papers: Supports of Invariant Measures for Piecewise Deter…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…
In this paper, we present a geometric approach for computing controlled invariant sets for hybrid control systems. While the problem is well studied in the ellipsoidal case, this family is quite conservative for constrained or switched…
We present a new method for statistical verification of quantitative properties over a partially unknown system with actions, utilising a parameterised model (in this work, a parametric Markov decision process) and data collected from…
We introduce Multi-Environment Markov Decision Processes (MEMDPs) which are MDPs with a set of probabilistic transition functions. The goal in a MEMDP is to synthesize a single controller with guaranteed performances against all…
There is a growing interest in methods for detecting and interpreting changes in experimental time evolution data. Based on measured time series, the quantitative characterization of dynamical phase transitions at bifurcation points of the…
Piecewise Deterministic Monte Carlo algorithms enable simulation from a posterior distribution, whilst only needing to access a sub-sample of data at each iteration. We show how they can be implemented in settings where the parameters live…
A general class of Markov processes with a local interaction is introduced, which includes exclusion and Kawasaki processes as a very particular case. Bernoulli invariant measures are found for this class of processes.
An approach for the description of stochastic systems is derived. Some of the variables in the system are studied forward in time, others backward in time. The approach is based on a perturbation expansion in the strength of the coupling…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
In this paper, we study the supports of measures in multiplicative free semigroups on the positive real line and on the unit circle. We provide formulas for the density of the absolutely continuous parts of measures in these semigroups. The…
Quantum trajectories are Markov processes describing the evolution of a quantum system subject to indirect measurements. They can be viewed as place dependent iterated function systems or the result of products of dependent and non…
We consider a stochastic electroconvection model describing the nonlinear evolution of a surface charge density in a two-dimensional fluid with additive stochastic forcing. We prove the existence and uniqueness of solutions and we show that…
We present some rigorous results on the absence of a wide class of invariant measures for dynamical systems possessing attractors. We then consider a generalization of the classical nonholonomic Suslov problem which shows how previous…
We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain…
We present conservativeness criteria for sub-Markovian semigroups generated by divergence type operators with specified infinitesimally invariant measures. The conservativeness criteria in this article are derived by $L^1$-uniqueness and…
We explore a class of hybrid (piecewise deterministic) systems characterized by a large number of individuals inhabiting an environment whose state is described by a set of continuous variables. We use analytical and numerical methods from…
We consider a discrete-time temporally-homogeneous conservative Markov process. We show that extremality of reversible measure implies extremality of invariant measure. Using analogue of Dirichlet form, we modify a proof that in stochastic…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
Consider a non-autonomous continuous-time linear system in which the time-dependent matrix determining the dynamics is piecewise constant and takes finitely many values $A_1, \dotsc, A_N$. This paper studies the equality cases between the…
We study natural invariance properties of functionals defined on L\'evy processes and show that they can be described by a simplified structure of the deterministic chaos kernels in It\^o's chaos expansion. These structural properties of…