Related papers: A Markov Process Theory for Network Growth Process…
We consider the problem of computing reachability probabilities: given a Markov chain, an initial state of the Markov chain, and a set of goal states of the Markov chain, what is the probability of reaching any of the goal states from the…
Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular…
Understanding the strategic behavior of miners in a blockchain is of great importance for its proper operation. A common model for mining games considers an infinite time horizon, with players optimizing asymptotic average objectives.…
Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented…
Under a markovian evolutionary process, the expected number of substitutions per site (also called branch length) that have occurred when a sequence has evolved from another according to a transition matrix $P$ can be approximated by…
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good…
Blockchain technology has been revolutionizing many fields since last decade. Its true potential is not practically utilized yet. In a very short period of time, it has evolved twice - Smart contracts and Directed Acyclic Graph (DAG). DAG…
It is challenging for a security analyst to detect or defend against cyber-attacks. Moreover, traditional defense deployment methods require the security analyst to manually enforce the defenses in the presence of uncertainties about the…
We consider Markov jump processes on a graph described by a rate matrix that depends on various control parameters. We derive explicit expressions for the static responses of edge currents and steady-state probabilities. We show that they…
Many important stochastic counting models can be written as general birth-death processes (BDPs). BDPs are continuous-time Markov chains on the non-negative integers and can be used to easily parameterize a rich variety of probability…
Blockchain consensus protocols enable participants to agree on consistent views of the blockchain that may be ahead or behind relative to each other but do not fork into different chains. A number of recently popular…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
The classical embeddability problem asks whether a given stochastic matrix $T$, describing transition probabilities of a $d$-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the…
DAG-based protocols have been proposed as potential solutions to the latency and throughput limitations of traditional permissionless consensus protocols. However, their adoption has been hindered by security concerns and a lack of a solid…
We develop techniques to analyse the statistics of completion times of non-deterministic elements in quantum entanglement generation, and how they affect the overall performance as measured by the secret key rate. By considering such…
We provide a parameterization of the discrete nested Markov model, which is a supermodel that approximates DAG models (Bayesian network models) with latent variables. Such models are widely used in causal inference and machine learning. We…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…
Tree-based protocols are ubiquitous in distributed systems. They are flexible, they perform generally well, and, in static conditions, their analysis is mostly simple. Under churn, however, node joins and failures can have complex global…
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…