Related papers: Markov chain-based stability analysis of growing n…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…
In studying network growth, the conventional approach is to devise a growth mechanism, quantify the evolution of a statistic or distribution (such as the degree distribution), and then solve the equations in the steady state (the…
Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving minimization problems. In this paper, we provide a…
It is possible to represent each of a number of Markov chains as an evolving sequence of connected subsets of a directed acyclic graph that grow in the following way: initially, all vertices of the graph are unoccupied, particles are fed in…
We consider a discrete-time Markov chain $(X^t,Y^t)$, $t=0,1,2,...$, where the $X$-component forms a Markov chain itself. Assume that $(X^t)$ is Harris-ergodic and consider an auxiliary Markov chain ${\hat{Y}^t}$ whose transition…
Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs). Despite the predictive power of these systems, it has been…
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
The equilibrium distributions of a Markovian model describing the interaction of several classes of permanent connections in a network are analyzed. It has been introduced by Graham and Robert. For this model each of the connections has a…
We propose a class of evolutionary models that involves an arbitrary exchangeable process as the breeding process and different selection schemes. In those models, a new genome is born according to the breeding process, and then a genome is…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring…
We consider Markov jump processes on a graph described by a rate matrix that depends on various control parameters. We derive explicit expressions for the static responses of edge currents and steady-state probabilities. We show that they…
This paper presents a comprehensive review of stochastic processes, with a particular focus on Markov chains and jump processes. The main results related to queuing systems are analyzed. Additionally, conditions that ensure the stability,…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
Our work introduces an approach for estimating the contribution of attachment mechanisms to the formation of growing networks. We present a generic model in which growth is driven by the continuous attachment of new nodes according to…
For irreducible, time-homogeneous Markov networks, mutual linearity has recently been established for both occupation probabilities and network currents in the stationary regime as well as in the non-stationary regime in Laplace space. The…
Robustness of routing policies for networks is a central problem which is gaining increased attention with a growing awareness to safeguard critical infrastructure networks against natural and man-induced disruptions. Routing under limited…
Many real-world scale-free networks, such as neural networks and online communication networks, consist of a fixed number of nodes but exhibit dynamic edge fluctuations. However, traditional models frequently overlook scenarios where the…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…