相关论文: Distributed Algorithms in an Ergodic Markovian Env…
We study the nature and mechanisms of broken ergodicity (BE) in specific random walk models corresponding to diffusion on random potential surfaces, in both one and high dimension. Using both rigorous results and nonrigorous methods, we…
We consider the statistics of occupation times, the number of visits at the origin and the survival probability for a wide class of stochastic processes, which can be classified as renewal processes. We show that the distribution of these…
We consider diffusion processes with a spatially varying diffusivity giving rise to anomalous diffusion. Such heterogeneous diffusion processes are analysed for the cases of exponential, power-law, and logarithmic dependencies of the…
Consider a simple random walk on the integers with the following transition mechanism. At each site $x$, the probability of jumping to the right is $\omega(x)\in[\frac12,1)$, until the first time the process jumps to the left from site $x$,…
Within the framework of probably approximately correct Markov decision processes (PAC-MDP), much theoretical work has focused on methods to attain near optimality after a relatively long period of learning and exploration. However,…
In this paper, we present a kernel-based, multi-task Gaussian Process (GP) model for approximating the underlying function of an individual's mobility state using a time-inhomogeneous Markov Process with two states: moves and pauses. Our…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
We analyze the asymptotic behavior for a system of fully nonlinear parabolic and elliptic quasi variational inequalities. These equations are related to robust switching control problems introduced in [3]. We prove that, as time horizon…
The probabilistic description of the time evolution of a physical system can take two conceptually distinct forms: a trajectory of probabilities, which specifies how probabilities evolve over time, and a probability on trajectories, which…
In this paper, we consider linear boundary port-Hamiltonian distributed parameter systems on a time-varying spatial domain. We derive the specific time-varying Dirac structure that these systems give rise to and use it to formally establish…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can…
We consider reinforcement learning in changing Markov Decision Processes where both the state-transition probabilities and the reward functions may vary over time. For this problem setting, we propose an algorithm using a sliding window…
Many records in environmental sciences exhibit asymmetric trajectories and there is a need for simple and tractable models which can reproduce such features. In this paper we explore an approach based on applying both a time change and a…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
We construct a general procedure for the Quasi Likelihood Analysis applied to a multivariate point process on the real half line in an ergodic framework. More precisely, we assume that the stochastic intensity of the underlying model…
In this paper we study the asymptotic behavior of the Random-Walk Metropolis algorithm on probability densities with two different `scales', where most of the probability mass is distributed along certain key directions with the…
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…
The exchange algorithm is one of the most popular extensions of the Metropolis--Hastings algorithm to sample from doubly-intractable distributions. However, the theoretical exploration of the exchange algorithm is very limited. For example,…
New algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces are presented. These algorithms are based on a special technique of sequential…
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…