Related papers: A Generalized Fundamental Matrix for Computing Fun…

Markov chains provide a foundational framework for modeling sequential stochastic processes, with the transition probability matrix characterizing the dynamics of state evolution. While classical estimation methods such as maximum…

The Markowitz model is still the cornerstone of modern portfolio theory. In particular, when focusing on the minimum-variance portfolio, the covariance matrix or better its inverse, the so-called precision matrix, is the only input…

Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…

Finding the stochastic equilibria for finite-state stochastic matrices amounts to solving an eigen\-vector problem $\pi = \pi P$. Various techniques for doing so are known, some extremely computationally intensive. Herein we shall aim to…

All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times…

The tensor properties of all the generators of Sp(12,R) - the group of dynamical symmetry of the Interacting Vector Boson Model (IVBM), are given with respect to the reduction chain Sp(12,R) $\supset $ U(6) $\supset $ U(3) x U(2) $\supset $…

Let $R$ be a continuous-time Markov process on the time interval $[0,1]$ with values in some state space $X$. We transform this reference process $R$ into $P:=f(X_0)\exp (-\int_0^1 V_t(X_t) dt) g(X_1)\,R$ where $f,g$ are nonnegative…

We consider conditional exact tests of factor effects in designed experiments for discrete response variables. Similarly to the analysis of contingency tables, Markov chain Monte Carlo methods can be used for performing exact tests,…

In this paper we present elementary computations for some Markov modulated counting processes, also called counting processes with regime switching. Regime switching has become an increasingly popular concept in many branches of science. In…

The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…

Value-at-risk (VaR), also known as quantile, is a crucial risk measure in finance and other fields. However, optimizing VaR metrics in Markov decision processes (MDPs) is challenging because VaR is non-additive and the traditional dynamic…

In tabular case, when the reward and environment dynamics are known, policy evaluation can be written as $\bm{V}_{\bm{\pi}} = (I - \gamma P_{\bm{\pi}})^{-1} \bm{r}_{\bm{\pi}}$, where $P_{\bm{\pi}}$ is the state transition matrix given…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

The time evolution of the one-point probability vector of stochastic processes and quantum processes for $N$-level systems have been unified. Hence, quantum states and quantum operations can be regarded as generalizations of the one-point…

A widely used approach to compute the action $f(A)v$ of a matrix function $f(A)$ on a vector $v$ is to use a rational approximation $r$ for $f$ and compute $r(A)v$ instead. If $r$ is not computed adaptively as in rational Krylov methods,…

Parametric Markov chains (pMC) are used to model probabilistic systems with unknown or partially known probabilities. Although (universal) pMC verification for reachability properties is known to be coETR-complete, there have been efforts…

With view to applications, we here give an explicit correspondence between the following two: (i) the set of symmetric and positive measures $\rho$ on one hand, and (ii) a certain family of generalized Markov transition measures $P$, with…

We consider a discrete-time Markov chain $\boldsymbol{\Phi}$ on a general state-space ${\sf X}$, whose transition probabilities are parameterized by a real-valued vector $\boldsymbol{\theta}$. Under the assumption that $\boldsymbol{\Phi}$…

We consider Markov decision processes (MDPs) which are a standard model for probabilistic systems. We focus on qualitative properties for MDPs that can express that desired behaviors of the system arise almost-surely (with probability 1) or…

Given a Markov Decision Process (MDP) with $n$ states and a totalnumber $m$ of actions, we study the number of iterations needed byPolicy Iteration (PI) algorithms to converge to the optimal$\gamma$-discounted policy. We consider two…