Related papers: P-adic Markov process and the problem of the first…
We consider the recurrence and transience problem for a time-homogeneous Markov chain on the real line with transition kernel $p(x,\mathrm{d}y)=f_x(y-x)\,\mathrm{d}y$, where the density functions $f_x(y)$, for large $|y|$, have a power-law…
The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…
Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
A new stochastic process is introduced and considered - squared Bessel process with special stochastic time. The analogues of fundamental properties for Brownian motion are deduced for squared Bessel process. In particular an analogue of…
In this article, we introduce \textit{Mallows processes}, defined to be continuous-time c\`adl\`ag processes with Mallows distributed marginals. We show that such processes exist and that they can be restricted to have certain natural…
We consider an additive functional driven by a time-inhomogeneous Markov chain with a finite state space. Our study focuses on the joint distribution of the two-sided exit time and the state of the driving Markov chain at the time of exit,…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
In this paper we analyze an eigenvalue problem related to the nonlocal $p-$laplace operator plus a potential. After reviewing some elementary properties of the first eigenvalue of these operators (existence, positivity of associated…
Often sharp changes in cellular behavior are triggered by thresholded events, i.e., by the attainment of a threshold value of a relevant cellular or molecular dynamical variable. Since the governing variable itself typically undergoes noisy…
We analyze and partially solve system of recurrences that can be derived from the properties of martingale orthogonal polynomials that characterize quadratic harnesses (QH). We also specify conditions for the existence of moments of one…
We revise the classical problem of characterizing first exit times of a harmonically trapped particle whose motion is described by one- or multi-dimensional Ornstein-Uhlenbeck process. We start by recalling the main derivation steps of a…
Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…
We prove a limit theorem for an integral functional of a Markov process. The Markovian dynamics is characterized by a linear Boltzmann equation modeling a one-dimensional test particle of mass $\lambda^{-1}\gg 1$ in an external periodic…
State-space models can be used to incorporate subject knowledge on the underlying dynamics of a time series by the introduction of a latent Markov state-process. A user can specify the dynamics of this process together with how the state…
We develop and analyse a first-order algorithm for the A-optimal experimental design problem. The problem is first presented as a special case of a parametric family of optimal design problems for which duality results and optimality…
Consider the problem to explicitly calculate the law of the first passage time T(a) of a general Levy process Z above a positive level a. In this paper it is shown that the law of T(a) can be approximated arbitrarily closely by the laws of…
The influence of a time-periodic forcing on stochastic processes can essentially be emphasized in the large time behaviour of their paths. The statistics of transition in a simple Markov chain model permits to quantify this influence. In…
We analyze the asymptotic behavior of sequences of random variables defined by an initial condition, a stationary and ergodic sequence of random matrices, and an induction formula involving multiplication is the so-called max-plus algebra.…
We aim at computing the derivative of the solution to a parametric optimization problem with respect to the involved parameters. For a class broader than that of strongly convex functions, this can be achieved by automatic differentiation…