相关论文: A Generalized Fundamental Matrix for Computing Fun…

We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…

We develop a semi-parametric state-space model for time-series data with latent regime transitions. Classical Markov-switching models use fixed parametric transition functions, such as logistic or probit links, which restrict flexibility…

Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf…

In multi-period stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. We analyze how the expected value of this random variable changes…

Multiparametric statistical model providing stable reconstruction of parameters by observations is considered. The only general method of this kind is the root model based on the representation of the probability density as a squared…

The initial theoretical connections between Leontief input-output models and Markov chains were established back in 1950s. However, considering the wide variety of mathematical properties of Markov chains, there has not been a full…

We consider evaluation of proper posterior distributions obtained from improper prior distributions. Our context is estimating a bounded function $\phi$ of a parameter when the loss is quadratic. If the posterior mean of $\phi$ is…

A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems and to express different statistical operations such as measurement,…

Policy gradient methods are extensively used in reinforcement learning as a way to optimize expected return. In this paper, we explore the evolution of the policy parameters, for a special class of exactly solvable POMDPs, as a…

Markov state models (MSMs) are a widely used method for approximating the eigenspectrum of the molecular dynamics propagator, yielding insight into the long-timescale statistical kinetics and slow dynamical modes of biomolecular systems.…

We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…

Optimizing a partially observable Markov decision process (POMDP) policy is challenging. The policy graph improvement (PGI) algorithm for POMDPs represents the policy as a fixed size policy graph and improves the policy monotonically. Due…

This study proposes a statistically grounded framework for real-time win probability evaluation and player assessment in score-based team sports, based on minute-by-minute cumulative box-score data. We introduce a continuous dominance…

Optimal Markov Decision Process policies for problems with finite state and action space are identified through a partial ordering by comparing the value function across states. This is referred to as state-based optimality. This paper…

In batch reinforcement learning, there can be poorly explored state-action pairs resulting in poorly learned, inaccurate models and poorly performing associated policies. Various regularization methods can mitigate the problem of learning…

A semi-Markov process method for obtaining general counting statistics for open quantum systems is extended to the scenario of resetting. The simultaneous presence of random resets and wave function collapses means that the quantum jump…

Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented…

Positive-definite matrices materialize as state transition matrices of linear time-invariant gradient flows, and the composition of such materializes as the state transition after successive steps where the driving potential is suitably…

The Lamperti--Kiu transformation for real-valued self-similar Markov processes (rssMp) states that, associated to each rssMp via a space-time transformation, there is a Markov additive process (MAP). In the case that the rssMp is taken to…

We derive an estimate of statistical error in calculating the trace of a large matrix by using random vector, and show that {\em random phase vector} gives the results with the smallest statistical error for a given basis set. This result…