Related papers: Persistent-Transient Policy Evaluation for Markov …
In this paper, we develop a new approach to study gain operators built from the interconnection gains of infinite networks of dynamical systems. Our focus is on the construction of paths of strict decay which are used for building Lyapunov…
The transverse Ising Model (TIM) in one dimension is the simplest model which exhibits a quantum phase transition (QPT). Quantities related to quantum information theoretic measures like entanglement, quantum discord (QD) and fidelity are…
In this paper, we study first the problem of nonparametric estimation of the stationary density $f$ of a discrete-time Markov chain $(X_i)$. We consider a collection of projection estimators on finite dimensional linear spaces. We select an…
In this article, we propose a least squares method for the estimation of the transition density in bifurcating Markov models. Unlike the kernel estimation, this method do not use the quotient which can be a source of errors. In order to…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
We study merchant energy production modeled as a compound switching and timing option. The resulting Markov decision process is intractable. State-of-the-art approximate dynamic programming methods applied to realistic instances of this…
We give a closed form of the discrete-time evolution of a recombination transformation in population genetics. This decomposition allows to define a Markov chain in a natural way. We describe the geometric decay rate to the limit…
This paper is concerned with the problem of nonlinear (stochastic) filter stability of a hidden Markov model (HMM) with white noise observations. A contribution is the variance decay property which is used to conclude filter stability. For…
We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…
We present a rigorous analytic proof of a generalized continued fraction (GCF) identity for the transcendental constant $8/\pi^2$, a result recently conjectured via the algorithmic framework of the Ramanujan Machine. Distinct from canonical…
Many finite-state reversible Markov chains can be naturally decomposed into "projection" and "restriction" chains. In this paper we provide bounds on the total variation mixing times of the original chain in terms of the mixing properties…
Algorithmic decisions made by machine learning models in high-stakes domains may have lasting impacts over time. However, naive applications of standard fairness criterion in static settings over temporal domains may lead to delayed and…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
Consider a finite irreducible Markov chain with invariant distribution $\pi$. We use the inner product induced by $\pi$ and the associated heat operator to simplify and generalize some results related to graph partitioning and the small-set…
We present a variational principle for the extraction of a time-dependent orthonormal basis from random realizations of transient systems. The optimality condition of the variational principle leads to a closed-form evolution equation for…
This paper presents a hidden Markov model designed to investigate the complex nature of earnings persistence. The proposed model assumes that the residuals of log-earnings consist of a persistent component and a transitory component, both…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…
Pairwise Choice Markov Chains (PCMC) have been recently introduced to overcome limitations of choice models based on traditional axioms unable to express empirical observations from modern behavior economics like context effects occurring…
Standard Transformers have a fixed computational depth, fundamentally limiting their ability to generalize to tasks requiring variable-depth reasoning, such as multi-hop graph traversal or nested logic. We propose a depth-recurrent…
We introduce a new interpretation of the attention matrix as a discrete-time Markov chain. Our interpretation sheds light on common operations involving attention scores such as selection, summation, and averaging in a unified framework. It…