Related papers: Absorption in Time-Varying Markov Chains: Graph-Ba…
We introduce the concept of group state transfer on graphs, summarize its relationship to other concepts in the theory of quantum walks, set up a basic theory, and discuss examples. Let $X$ be a graph with adjacency matrix $A$ and consider…
We conduct a systematic study of asynchronous models of distributed computing consisting of identical finite-state devices that cooperate in a network to decide if the network satisfies a given graph-theoretical property. Models discussed…
We study episodic reinforcement learning in Markov decision processes when the agent receives additional feedback per step in the form of several transition observations. Such additional observations are available in a range of tasks…
In this paper, we formulate and investigate a generalized consensus algorithm which makes an attempt to unify distributed averaging and maximizing algorithms considered in the literature. Each node iteratively updates its state as a…
In this work, we study the topological transition on the associated networks in a model proposed by Saeedian et al. (Scientific Reports 2019 9:9726), which considers a coupled dynamics of node and link states. Our goal was to better…
We explore conditions on the covariance matrices of a consistent chain of mean zero finite mode Gaussian states in order that the chain may be exchangeable or stationary. For an exchangeable chain our conditions are necessary and…
We consider here systems with piecewise linear dynamics that are periodically sampled with a given period {\tau} . At each sampling time, the mode of the system, i.e., the parameters of the linear dynamics, can be switched, according to a…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
This paper concentrates on the reliability of a discrete-time controlled Markov system with finite states and actions, and aims to give an efficient algorithm for obtaining an optimal (control) policy that makes the system have the maximal…
A combination of first principle molecular dynamics (MD) simulations with a rate equation model (MD-RE approach) is presented to study the trapping and the scattering of rare gas atoms from metal surfaces. The temporal evolution of the atom…
In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop…
Asymptotic hyperstability is achievable under certain switching laws if at least one of the feed-forward parameterization: 1) possesses a strictly positive real transfer function, 2) a minimum residence time interval is respected for each…
An important problem arising in the study of complex networks, for instance in community detection and motif finding, is the sampling of graphs with fixed degree sequence. The equivalent problem of generating random 0,1 matrices with fixed…
We consider a risk-sensitive optimization of consumption-utility on infinite time horizon where the one-period investment gain depends on an underlying economic state whose evolution over time is assumed to be described by a discrete-time,…
A granular system confined in a quasi two-dimensional box that is vertically vibrated can transit to an absorbing state in which all particles bounce vertically in phase with the box, with no horizontal motion. In principle, this state can…
Countable state Markov shifts are a natural generalization of the well-known subshifts of finite type. They are the subject of current research both for their own sake and as models for smooth dynamical systems. In this paper, we…
We give a Markov chain based algorithm for sampling almost uniform solutions of constraint satisfaction problems (CSPs). Assuming a canonical setting for the Lov\'asz local lemma, where each constraint is violated by a small number of…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…
Under certain conditions, the dynamics of coarse-grained models of solvated proteins can be described using a Markov state model, which tracks the evolution of populations of configurations. The transition rates among states that appear in…
Choosing a uniformly sampled simple directed graph realization of a degree sequence has many applications, in particular in social networks where self-loops are commonly not allowed. It has been shown in the past that one can perform a…