相关论文: Computing the minimal rebinding effect for non-rev…

We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…

Markov processes restarted or reset at random times to a fixed state or region in space have been actively studied recently in connection with random searches, foraging, and population dynamics. Here we study the large deviations of…

We develop Markov chain mixing time estimates for a class of Markov chains with restricted transitions. We assume transitions may occur along a cycle of $n$ nodes and on $n^\gamma$ additional edges, where $\gamma < 1$. We find that the…

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…

In this paper we study the semi-global (approximate) state feedback stabilization of an infinite dimensional quantum stochastic system towards a target state. A discrete-time Markov chain on an infinite-dimensional Hilbert space is used to…

Learning the influence graph G of a high-dimensional Markov process is central to many application domains, including social networks, neuroscience, and financial risk analysis. However, in many of these applications, future states of the…

In this work we address the problem of finding feasible policies for Constrained Markov Decision Processes under probability one constraints. We argue that stationary policies are not sufficient for solving this problem, and that a rich…

The efficient calculation of rare-event kinetics in complex dynamical systems, such as the rate and pathways of ligand dissociation from a protein, is a generally unsolved problem. Markov state models can systematically integrate ensembles…

Many probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely…

To make progress in understanding the issue of memory loss and history dependence in evolving complex systems, we consider the mixing rate that specifies how fast the future states become independent of the initial condition. We propose a…

Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such…

We study reinforcement learning methods with linear function approximation under non-Markov state and cost processes. We first consider the policy evaluation method and show that the algorithm converges under suitable ergodicity conditions…

We consider a control problem for a finite-state Markov system whose performance is evaluated by a coherent Markov risk measure. For each policy, the risk of a state is approximated by a function of its features, thus leading to a…

Memoryless processes are ubiquitous in nature, in contrast with the mathematics of open systems theory, which states that non-Markovian processes should be the norm. This discrepancy is usually addressed by subjectively making the…

We consider reinforcement learning in changing Markov Decision Processes where both the state-transition probabilities and the reward functions may vary over time. For this problem setting, we propose an algorithm using a sliding window…

Consider the problem of predicting the next symbol given a sample path of length n, whose joint distribution belongs to a distribution class that may have long-term memory. The goal is to compete with the conditional predictor that knows…

Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite…

Matrix product operators allow efficient descriptions (or realizations) of states on a 1D lattice. We consider the task of learning a realization of minimal dimension from copies of an unknown state, such that the resulting operator is…

Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones…

Equations governing the nonlinear dynamics of complex systems are usually unknown and indirect methods are used to reconstruct their manifolds. In turn, they depend on embedding parameters requiring other methods and long temporal sequences…