Related papers: A linear programming approach to Markov reward err…

A switching random walk, commonly known under the misnomer `oscillating random walk', is a real-valued Markov chain whose distribution of increments is determined by the sign of the current position. We explicitly identify an invariant…

We have developed a steady state theory of complex transport networks used to model the flow of commodity, information, viruses, opinions, or traffic. Our approach is based on the use of the Markov chains defined on the graph…

In this paper we study the dynamics of nonlinear random walks. While typical random walks on networks consist of standard Markov chains whose static transition probabilities dictate the flow of random walkers through the network, nonlinear…

We obtain non-Gaussian limit laws for one-dimensional random walk in a random environment assuming that the environment is a function of a stationary Markov process. This is an extension of the work of Kesten, M. Kozlov and Spitzer for…

Random walks find applications in many areas of science and are the heart of essential network analytic tools. When defined on temporal networks, even basic random walk models may exhibit a rich spectrum of behaviours, due to the…

We study and develop the stochastic Markov reward model (sMRM), which extends the Markov chain where transition time/reward as modelled as random variables. Techniques are presented to enable computing first-passage time distributions (or…

We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…

Our model is a constrained homogeneous random walk in a nonnegative orthant Z_+^d. The convergence to stationarity for such a random walk can often be checked by constructing a Lyapunov function. The same Lyapunov function can also be used…

We consider random walks on dynamical networks where edges appear and disappear during finite time intervals. The process is grounded on three independent stochastic processes determining the walker's waiting-time, the up-time and down-time…

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…

Here, a new two-dimensional process, discrete in time and space, that yields the results of both a random walk and a quantum random walk, is introduced. This model describes the population distribution of four coin states |1>,-|1>, |0> -|0>…

We explore the use of machine-learning techniques to detect quantum speedup in random walks on graphs. Specifically, we investigate the performance of three different neural-network architectures (variations on fully connected and…

A one-dimensional confined Nonlinear Random Walk is a tuple of $N$ diffeomorphisms of the unit interval driven by a probabilistic Markov chain. For generic such walks, we obtain a geometric characterization of their ergodic stationary…

Queue networks describe complex stochastic systems of both theoretical and practical interest. They provide the means to assess alterations, diagnose poor performance and evaluate robustness across sets of interconnected resources. In the…

In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the…

We revisit a simple model class for machine learning on graphs, where a random walk on a graph produces a machine-readable record, and this record is processed by a deep neural network to directly make vertex-level or graph-level…

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…

We study a limit behavior of a sequence of Markov processes (or Markov chains) such that their distributions outside of any neighborhood of a "singular" point attract to some probability law. In any neighborhood of this point the behavior…

The characterization of the Hamiltonian parameters defining a quantum walk is of paramount importance when performing a variety of tasks, from quantum communication to computation. When dealing with physical implementations of quantum…

Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…