相关论文: Projective surjectivity of quadratic stochastic op…

We study a model of repeated interaction between quantum systems which can be thought of as a non-commutative Markov chain. It is shown that there exists an outgoing Cuntz scattering system associated to this model which induces an…

SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.\ data, where a single sample trajectory is…

In this paper, we deal with a class of time-homogeneous continuous-time Markov processes with transition probabilities bearing a nonparametric uncertainty. The uncertainty is modeled by considering perturbations of the transition…

We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…

We consider the problem of approximating optimal in the Minimum Mean Squared Error (MMSE) sense nonlinear filters in a discrete time setting, exploiting properties of stochastically convergent state process approximations. More…

Modern methods of simulating molecular systems are based on the mathematical theory of Markov operators with a focus on autonomous equilibrated systems. However, non-autonomous physical systems or non-autonomous simulation processes are…

We consider iterated function systems (finite or countable), together with linear and continuous operators on Hilbert spaces, which enable us to construct Markov-type operators. Under suitable conditions, these Markov-type operators have…

Markov chain Monte Carlo is an inherently serial algorithm. Although likelihood calculations for individual steps can sometimes be parallelized, the serial evolution of the process is widely viewed as incompatible with parallelization,…

Reversible Markov chains play a central role in stochastic modelling and in algorithms such as Markov chain Monte Carlo (MCMC). Motivated by the fundamental importance of reversibility in classical settings, this paper develops a…

Iteration of randomly chosen quadratic maps defines a Markov process: X_{n+1}=\epsilon_{n+1}X_n(1-X_n), where \epsilon_n are i.i.d. with values in the parameter space [0,4] of quadratic maps F_{\theta}(x)=\theta x(1-x). Its study is of…

We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…

We consider stochastic variational inequalities with monotone operators defined as the expected value of a random operator. We assume the feasible set is the intersection of a large family of convex sets. We propose a method that combines…

A noncommutative Fornasini-Marchesini system (a multi-variable version of a linear system) can be realized within a weak Markov process (a model for quantum evolution). For a discrete time parameter the resulting structure is worked out…

A probabilistic Markov Chain (MC) surrogate model for a two-dimensional system of interacting particles within a square domain having inherent symmetries is developed. Particles are assumed to be circular and identical, colliding with each…

We consider general Markov chains with discrete time in an arbitrary measurable (phase) space and homogeneous in time. Markov chains are defined by the classical transition function which within the framework of the operator treatment…

We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC…

Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…

Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ…

We consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state…

In this paper, we propose a novel stochastic process that serves as a natural discrete-time counterpart to the continuous-time model known as the ``Poisson hyperbolic staircase'' proposed by Levikson et al. (1999), and clarify its…