Related papers: Stability in Distribution of Randomly Perturbed Qu…
Planning problems where effects of actions are non-deterministic can be modeled as Markov decision processes. Planning problems are usually goal-directed. This paper proposes several techniques for exploiting the goal-directedness to…
By constructing jointly a random graph and an associated exploration process, we define the dynamics of a "parking process" on a class of uniform random graphs as a measure-valued Markov process, representing the empirical degree…
We study Markov processes generated by iterated function systems (IFS). The constituent maps of the IFS are monotonic transformations of the interval. We first obtain an upper bound on the number of SRB (Sinai-Ruelle-Bowen) measures for the…
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…
We treat three cubic recurrences, two of which generalize the famous iterated map $x \mapsto x (1-x)$ from discrete chaos theory. A feature of each asymptotic series developed here is a constant, dependent on the initial condition but…
We consider the invariant measure of a homogeneous continuous- time Markov process in the quarter-plane. The basic solutions of the global balance equation are the geometric distributions. We first show that the invariant measure can not be…
A sequence of real numbers (x_n) is Benford if the significands, i.e. the fraction parts in the floating-point representation of (x_n) are distributed logarithmically. Similarly, a discrete-time irreducible and aperiodic finite-state Markov…
In this work we present iterated function systems with general measures(IFSm) formed by a set of maps $\tau_{\lambda}$ acting over a compact space $X$, for a compact space of indices, $\Lambda$. The Markov process $Z_k$ associated to the…
In sequential design strategies, common in geostatistics and Bayesian optimization, the selection of a new observation point $X_{n+1}$ of a random function $\mathbf f$ is informed by past data, captured by the filtration $\mathcal…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We describe estimators $\chi_n(X_0,X_1,...,X_n)$, which when applied to an unknown stationary process taking values from a countable alphabet ${\cal X}$, converge almost surely to $k$ in case the process is a $k$-th order Markov chain and…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
Let \xi_t, t\in[0,T], be a strong Markov process with values in a complete separable metric space (X,\rho) and with transition probability function P_{s,t}(x,dy), 0\le s\le t\le T, x\in X. For any h\in[0,T] and a>0, consider the function…
This work considers the problem of learning the Markov parameters of a linear system from observed data. Recent non-asymptotic system identification results have characterized the sample complexity of this problem in the single and…
Let $X_1,X_2,...$ be a sequence of random variables satisfying the distributional recursion $X_1=0$ and $X_n= X_{n-I_n}+1$ for $n=2,3,...$, where $I_n$ is a random variable with values in $\{1,...,n-1\}$ which is independent of…
This paper is devoted to parameter estimation for partially observed polynomial state space models. This class includes discretely observed affine or more generally polynomial Markov processes. The polynomial structure allows for the…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
In many contexts such as queuing theory, spatial statistics, geostatistics and meteorology, data are observed at irregular spatial positions. One model of this situation involves considering the observation points as generated by a Poisson…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…