Related papers: A Generalized Fundamental Matrix for Computing Fun…

The comparison of benchmark error sets is an essential tool for the evaluation of theories in computational chemistry. The standard ranking of methods by their Mean Unsigned Error is unsatisfactory for several reasons linked to the…

We continue development of the theory of Markov systems initiated in \cite{Wer1}. In this paper, we introduce fundamental Markov systems associated with random dynamical systems and show that the proof of the uniqueness and empiricalness of…

This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…

We extend Hoeffding's lemma to general-state-space and not necessarily reversible Markov chains. Let $\{X_i\}_{i \ge 1}$ be a stationary Markov chain with invariant measure $\pi$ and absolute spectral gap $1-\lambda$, where $\lambda$ is…

Inference, prediction and control of complex dynamical systems from time series is important in many areas, including financial markets, power grid management, climate and weather modeling, or molecular dynamics. The analysis of such highly…

The linear Markov Decision Process (MDP) framework offers a principled foundation for reinforcement learning (RL) with strong theoretical guarantees and sample efficiency. However, its restrictive assumption-that both transition dynamics…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

We consider Markov basis arising from fractional factorial designs with three-level factors. Once we have a Markov basis, $p$ values for various conditional tests are estimated by the Markov chain Monte Carlo procedure. For designed…

The vector autoregression (VAR) has been widely used in system identification, econometrics, natural science, and many other areas. However, when the state dimension becomes large the parameter dimension explodes. So rank reduced modelling…

Conditional distributions, as defined by the Markov category framework, are studied in the setting of matrix algebras (quantum systems). Their construction as linear unital maps are obtained via a categorical Bayesian inversion procedure.…

Deriving the time-dependent expected reward function associated with a continuous-time Markov chain involves the computation of its transient deviation matrix. In this paper we focus on the special case of a finite quasi-birth-and-death…

Markov chain Monte Carlo (MCMC) methods to sample from a probability distribution $\pi$ defined on a space $(\Theta,\mathcal{T})$ consist of the simulation of realisations of Markov chains $\{\theta_{n},n\geq1\}$ of invariant distribution…

Cross-sectional observations from a dynamical system can be modeled via steady-state distributions of Markov processes. The major challenge is then to determine whether the process parameters can be identified and estimated from the…

An expansion of row Markov matrices in terms of matrices related to permutations with repetitions, is introduced.It generalises the Birkhoff-von Neumann expansion of doubly stochastic matrices in terms of permutation matrices (without…

This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

In many autonomy applications, performance of perception algorithms is important for effective planning and control. In this paper, we introduce a framework for computing the probability of satisfaction of formal system specifications given…

We introduce the matrix-valued time-varying Main Effects Factor Model (MEFM). MEFM is a generalization to the traditional matrix-valued factor model (FM). We give rigorous definitions of MEFM and its identifications, and propose estimators…

Stochastic (Markovian) process algebra extend classical process algebra with probabilistic exponentially distributed time durations denoted by rates (the parameter of the exponential distribution). Defining a semantics for such an algebra,…

A generalization of the entropy production rate is proposed $\Pi_q$ in non-equilibrium systems by extending the formalism of classical stochastic thermodynamics to regimes with non-Gaussian fluctuations. Through the R\'enyi entropy $S_q$ ,…