Related papers: Minimal positive Markov realizations
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…
The construction presented in this paper can be briefly described as follows: starting from any "finite-dimensional" Markov transition function p_t, on a measurable state space (E,B), we construct a strong Markov process on a certain…
Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…
We present metrics for measuring state similarity in Markov decision processes (MDPs) with infinitely many states, including MDPs with continuous state spaces. Such metrics provide a stable quantitative analogue of the notion of…
This paper develops a systematic approach to realising linear detectors with an optimised sensitivity, allowing for the detection of extremely weak signals. First, general constraints are derived on a specific class of input-output transfer…
We develop a framework for the compression of reversible Markov chains with rigorous error control. Given a subset of selected states, we construct reduced dynamics that can be lifted to an approximation of the full dynamics, and we prove…
In the past few years, the study of receptive field codes has been of large interest to mathematicians. Here we give a complete characterization of receptive field codes realizable by connected receptive fields and we give the minimal…
Consider a real line equipped with a (not necessarily intrinsic) distance. We deal with the minimum-weight perfect matching problem for a complete graph whose points are located on the line and whose edges have weights equal to distances…
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In…
Modeling unknown systems from data is a precursor of system optimization and sequential decision making. In this paper, we focus on learning a Markov model from a single trajectory of states. Suppose that the transition model has a small…
Consider a stationary discrete random process with alphabet size d, which is assumed to be the output process of an unknown stationary Hidden Markov Model (HMM). Given the joint probabilities of finite length strings of the process, we are…
Noncommutative rational functions, i.e., elements of the universal skew field of fractions of a free algebra, can be defined through evaluations of noncommutative rational expressions on tuples of matrices. This interpretation extends their…
This paper presents a state representation framework for Markov decision processes (MDPs) that can be learned solely from state trajectories, requiring neither reward signals nor the actions executed by the agent. We propose learning the…
We consider the problem: is the optimal expected total reward to reach a goal state in a partially observable Markov decision process (POMDP) below a given threshold? We tackle this -- generally undecidable -- problem by computing…
This paper investigates a class of optimal control problems associated with Markov processes with local state information. The decision-maker has only local access to a subset of a state vector information as often encountered in…
A multiplicative relative value iteration algorithm for solving the dynamic programming equation for the risk-sensitive control problem is studied for discrete time controlled Markov chains with a compact Polish state space, and controlled…
We consider dynamic programming problems with a large time horizon, and give sufficient conditions for the existence of the uniform value. As a consequence, we obtain an existence result when the state space is precompact, payoffs are…
It has previously been shown that response transformations can be very effective in improving dimension reduction outcomes for a continuous response. The choice of transformation used can make a big difference in the visualization of the…
We consider deterministic Markov decision processes (MDPs) and apply max-plus algebra tools to approximate the value iteration algorithm by a smaller-dimensional iteration based on a representation on dictionaries of value functions. The…
Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such…