Related papers: Algorithmic Randomness in Continuous-Time Markov C…

We define the concept of an `open' Markov process, a continuous-time Markov chain equipped with specified boundary states through which probability can flow in and out of the system. External couplings which fix the probabilities of…

We make precise sense of the idea of "molecular chaos" through algorithmic randomness of microscopic trajectories, and ground macroscopic irreversibility in the lack of symmetry under time reversal of this property. This concept of…

The applicability of machine learning for predicting chaotic dynamics relies heavily upon the data used in the training stage. Chaotic time series obtained by numerically solving ordinary differential equations embed a complicated noise of…

Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…

In this paper a method based on a Markov chain Monte Carlo (MCMC) algorithm is proposed to compute the probability of a rare event. The conditional distribution of the underlying process given that the rare event occurs has the probability…

In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…

In this paper, we present criteria for non-exponential ergodicity of continuous-time Markov chains on a countable state space. These criteria can be verified by examining the ratio of transition rates over certain paths. We applied this…

Kinetic Monte Carlo methods such as the Gillespie algorithm model chemical reactions as random walks in particle number space. The inter-reaction times are exponentially distributed under the assumption that the system is well mixed. We…

This paper is motivated by examples from stochastic reaction network theory. The $Q$-matrix of a stochastic reaction network can be derived from the reaction graph, an edge-labelled directed graph encoding the jump vectors of an associated…

We present a numerical approximation technique for the analysis of continuous-time Markov chains that describe networks of biochemical reactions and play an important role in the stochastic modeling of biological systems. Our approach is…

This work considers the method of uniformisation for continuous-time Markov chains in the context of chemical reaction networks. Previous work in the literature has shown that uniformisation can be beneficial in the context of…

Multiplex networks are a common modeling framework for interconnected systems and multimodal data, yet we still lack fundamental insights for how multiplexity affects stochastic processes. We introduce a novel ``Markov chains of Markov…

This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct - only using the graph structure of the Markov chain and…

Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…

Machine learning algorithms have been applied to predict agent behaviors in real-world dynamic systems, such as advertiser behaviors in sponsored search and worker behaviors in crowdsourcing. The behavior data in these systems are generated…

Continuous Time Random Maxima (CTRM) are a generalization of classical extreme value theory: Instead of observing random events at regular intervals in time, the waiting times between the events are also random variables with arbitrary…

Cyclical MCMC is a novel MCMC framework recently proposed by Zhang et al. (2019) to address the challenge posed by high-dimensional multimodal posterior distributions like those arising in deep learning. The algorithm works by generating a…

In this paper we continue the study of conditional Markov chains (CMCs) with finite state spaces, that we initiated in Bielecki, Jakubowski and Niew\k{e}g{\l}owski (2014a) in an effort to enrich the theory of CMCs that was originated in…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…

Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…