Related papers: Computing the minimal rebinding effect for non-rev…
Differential equations containing memory terms that depend nonlinearly on past states model a variety of non-Markovian processes. In this study, we present a Markovian embedding procedure for such equations with distributed delay by…
This paper presents a state representation framework for Markov decision processes (MDPs) that can be learned solely from state trajectories, requiring neither reward signals nor the actions executed by the agent. We propose learning the…
We address the problem of estimating the mixing time $t_{\mathsf{mix}}$ of an arbitrary ergodic finite-state Markov chain from a single trajectory of length $m$. The reversible case was addressed by Hsu et al. [2019], who left the general…
We introduce a Vasicek-type short rate model which has two additional parameters representing memory effect. This model presents better results in yield curve fitting than the classical Vasicek model. We derive closed-form expressions for…
The unknown inputs in a dynamical system may represent unknown external drivers, input uncertainty, state uncertainty, or instrument faults and thus unknown-input reconstruction has several wide-spread applications. In this paper we…
Lumping a Markov process introduces a coarser level of description that is useful in many contexts and applications. The dynamics on the coarse grained states is often approximated by its Markovian component. In this letter we derive…
For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the…
We develop a diffusion approximation for systems subject to fast random resetting by small amplitudes. Equivalently, this describes systems with frequent but small catastrophes. We demonstrate the validity of the approximation by computing…
We study finite memory belief approximation for partially observable (PO) stochastic optimal control (SOC) problems. While belief states are sufficient for SOC in partially observable Markov decision processes (POMDPs), they are generally…
We consider the problem of estimating the measure of subsets in very large networks. A prime tool for this purpose is the Markov Chain Monte Carlo (MCMC) algorithm. This algorithm, while extremely useful in many cases, still often suffers…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
We study the policy testing problem in discounted Markov decision processes (MDPs) in the fixed-confidence setting under a generative model with static sampling. The goal is to decide whether the value of a given policy exceeds a specified…
Many chemical reactions and molecular processes occur on timescales that are significantly longer than those accessible by direct simulation. One successful approach to estimating dynamical statistics for such processes is to use many short…
We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension…
The observation of light super-partners from a supersymmetric extension to the Standard Model is an intensely sought-after experimental outcome, providing an explanation for the stabilization of the electroweak scale and indicating the…
We propose a new multivariate time series model in which we assume that each component has a tendency to revert to the minimum of all components. Such a specification is useful to describe phenomena where each member in a population which…
Reinforcement learning algorithms are commonly analyzed (and designed) under the Markov assumption. This is unrealistic, as most environments encountered in practice are either partially observable, or require function approximation that…
Submodular function minimization is well studied, and existing algorithms solve it exactly or up to arbitrary accuracy. However, in many applications, such as structured sparse learning or batch Bayesian optimization, the objective function…
A theory of additive Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to describe statistical properties of long-range correlated systems. The convenient characteristics of such…
A key capability of intelligent agents is operating under partial observability: reasoning and acting effectively despite missing or incomplete state observations. While recurrent (memory-based) policies learned via reinforcement learning…