Related papers: On exponential recurrence of "Markov-up'' processe…
We consider a general piecewise deterministic Markov process (PDMP) $X=\{X_t\}_{t\geqslant 0}$ with measure-valued generator $\mathcal{A}$, for which the conditional distribution function of the inter-occurrence time is not necessarily…
This paper introduces a periodic multivariate Poisson autoregression with potentially infinite memory, with a special focus on the network setting. Using contraction techniques, we study the stability of such a process and provide upper…
We study a class of Markov chains that model the evolution of a quantum system subject to repeated measurements. Each Markov chain in this class is defined by a measure on the space of matrices. It is then given by a random product of…
Entropy regularized Markov decision processes have been widely used in reinforcement learning. This paper is concerned with the primal-dual formulation of the entropy regularized problems. Standard first-order methods suffer from slow…
The paper considers a stochastic differential equation of Duffing type with Markov coefficients. The existence of unpredictable solutions is considered. The unpredictability is a property of bounded functions characterized by unbounded…
There is a relation between the irreversibility of thermodynamic processes as expressed by the breaking of time-reversal symmetry, and the entropy production in such processes. We explain on an elementary mathematical level the relations…
Extreme-value theory for random vectors and stochastic processes with continuous trajectories is usually formulated for random objects all of whose univariate marginal distributions are identical. In the spirit of Sklar's theorem from…
This brief article gives an overview of quantum mechanics as a {\em quantum probability theory}. It begins with a review of the basic operator-algebraic elements that connect probability theory with quantum probability theory. Then quantum…
We propose Markov two-components processes (M2CP) as a probabilistic model of asynchronous systems based on the trace semantics for concurrency. Considering an asynchronous system distributed over two sites, we introduce concepts and tools…
Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…
Comparison results for Markov processes w.r.t. function class induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
In this paper, we introduce a time-continuous production model that enables random machine failures, where the failure probability depends historically on the production itself. This bidirectional relationship between historical failure…
New relations between ergodic rate, L_p convergence rates, and asymptotic behavior of tail probabilities for hitting times of a time homogeneous Markov process are established. For L_p convergence rates and related spectral and functional…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…
We consider a pair of correlated processes {Z_n} and {S_n} (two sided), where the former is observable and the later is hidden. The uncertainty in the estimation of Z_n upon its finite past history is H(Z_n|Z_0^{n-1}), and for estimation of…