相关论文: Distributed Algorithms in an Ergodic Markovian Env…

In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated…

Analysis of algorithms on time-varying networks (often called evolving graphs) is a modern challenge in theoretical computer science. The edge-Markovian is a relatively simple and comprehensive model of evolving graphs: every pair of…

Dynamic Programming suffers from the curse of dimensionality due to large state and action spaces, a challenge further compounded by uncertainties in the environment. To mitigate these issue, we explore an off-policy based Temporal…

We made a first attempt to associate a probabilistic description of stochastic processes like birth-death processes with spacetime geometry in the Schwarzschild metrics on distance scales from the macro- to the micro-domains. We idealize an…

The main goal of this paper is to investigate continuous-time distributed dynamic programming (DP) algorithms for networked multi-agent Markov decision problems (MAMDPs). In our study, we adopt a distributed multi-agent framework where…

This paper is devoted to the study of a stochastic process obtained by random switching between a finite collection of vector fields. Such processes have recently been the focus of much attention in the case where the switching times are…

We consider random walks on dynamical networks where edges appear and disappear during finite time intervals. The process is grounded on three independent stochastic processes determining the walker's waiting-time, the up-time and down-time…

We prove a quenched central limit theorem for random walks with bounded increments in a randomly evolving environment on $\mathbb{Z}^d$. We assume that the transition probabilities of the walk depend not too strongly on the environment and…

Multi-agent planning under stochastic dynamics is usually formalised using decentralized (partially observable) Markov decision processes ( MDPs) and reachability or expected reward specifications. In this paper, we propose a different…

Stochastic Spatio-Temporal processes are prevalent across domains ranging from modeling of plasma to the turbulence in fluids to the wave function of quantum systems. This letter studies a measure-theoretic description of such systems by…

The time evolution of a bounded quantum system is considered in the framework of the orthogonal, unitary and symplectic circular ensembles of random matrix theory. For an $N$ dimensional Hilbert space we prove that in the large $N$ limit…

We propose a method for analyzing the distributed random coordinate descent algorithm for solving separable resource allocation problems in the context of an open multiagent system, where agents can be replaced during the process. In…

We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…

In this paper, we analyze the dynamics of spreading processes taking place over time-varying networks. A common approach to model time-varying networks is via Markovian random graph processes. This modeling approach presents the following…

This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded…

In this paper, we present an overview of different types of random walk strategies with local and non-local transitions on undirected connected networks. We present a general approach to analyzing these strategies by defining the dynamics…

In this article we consider the ergodic risk-sensitive control problem for a large class of multidimensional controlled diffusions on the whole space. We study the minimization and maximization problems under either a blanket stability…

We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…

The methodology based on the random walk processes is adapted and applied to a comprehensive analysis of the statistical properties of the probability fluxes. To this aim we define a simple model of the Markovian stochastic dynamics on a…

We study a multi-armed bandit problem in a dynamic environment where arm rewards evolve in a correlated fashion according to a Markov chain. Different than much of the work on related problems, in our formulation a learning algorithm does…