Related papers: Weak approximation rates for integral functionals …
We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate…
A stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information and the sequence of partial subgradients is determined by a general…
We consider a sequence of additive functionals {\phi_n}, set on a sequence of Markov chains {X_n} that weakly converges to a Markov process X. We give sufficient condition for such a sequence to converge in distribution, formulated in terms…
We investigate a particular form of weak convergence of the local empirical process.
In the article the distributions of overjump functionals for almost semi-continuous processes on a finite irreducible Markov chain are considered.
This paper proves that, under a monotonicity condition, the invariant probability measure of a McKean--Vlasov process can be approximated by weighted empirical measures of some processes including itself. These processes are described by…
This paper presents a nonparametric method for estimating the conditional density associated to the jump rate of a piecewise-deterministic Markov process. In our framework, the estimation needs only one observation of the process within a…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
For a wide class of continuous-time Markov processes, including all irreducible hypoelliptic diffusions evolving on an open, connected subset of $\RL^d$, the following are shown to be equivalent: (i) The process satisfies (a slightly weaker…
We construct intrinsic on-and off-diagonal upper and lower estimates for the transition probability density of a L\'evy process in small time. By intrinsic we mean that such estimates reflect the structure of the characteristic exponent of…
We study fast approximation of integrals with respect to stationary probability measures associated to iterated functions systems on the unit interval. We provide an algorithm for approximating the integrals under certain conditions on the…
Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…
In this paper, the weak convergence of impulsive recurrent process with Markov switching in the scheme of Levy approximation is proved. For the relative compactness, a method proposed by R. Liptser for semimartingales is used with a…
In an observed generalized semi-Markov regime, estimation of transition rate of regime switching leads towards calculation of locally risk minimizing option price. Despite the uniform convergence of estimated step function of transition…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
In this paper, we obtain sufficient conditions in terms of projective criteria under which the partial sums of a stationary process with values in ${\mathcal{H}}$ (a real and separable Hilbert space) admits an approximation, in…
We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…
Clustered multistate process data are commonly encountered in multicenter observational studies and clinical trials. A clinically important estimand with such data is the marginal probability of being in a particular transient state as a…