Related papers: Evidential Decision Theory via Partial Markov Cate…

Comparison results for Markov processes w.r.t. function class induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach…

We study the problem of learning the Markov order in categorical sequences that represent paths in a network, i.e. sequences of variable lengths where transitions between states are constrained to a known graph. Such data pose challenges…

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…

We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…

The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in…

In this paper, we are interested in optimal decisions in a partially observable Markov universe. Our viewpoint departs from the dynamic programming viewpoint: we are directly approximating an optimal strategic tree depending on the…

This work presents a conceptual synthesis of causal discovery and inference frameworks, with a focus on how foundational assumptions -- causal sufficiency, causal faithfulness, and the causal Markov condition -- are formalized and…

In this paper we present a categorical version of the first and second fundamental theorems of the invariant theory for the quantized symplectic groups. Our methods depend on the theory of braided strict monoidal categories which are…

Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…

We consider the analysis of sets of categorical sequences consisting of piecewise homogeneous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each…

Generalizing Eisenbud's matrix factorizations, we define factorization categories. Following work of Positselski, we define their associated derived categories. We construct specific resolutions of factorizations built from a choice of…

We present a categorical construction for modelling causal structures within a general class of process theories that include the theory of classical probabilistic processes as well as quantum theory. Unlike prior constructions within…

I formulate an entropy-rate maximization problem at the observable level for stochastic processes observed through an information-reducing observation map. For a visible stationary law, the map determines an observational fiber of hidden…

For a class of piecewise deterministic Markov processes, the supports of the invariant measures are characterized. This is based on the analysis of controllability properties of an associated deterministic control system. Its invariant…

Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are…

The paper studies information-theoretic opacity, an information-flow privacy property, in a setting involving two agents: A planning agent who controls a stochastic system and an observer who partially observes the system states. The goal…

A new object of the probability theory, two-sided chain of events (symbols), is introduced. A theory of multi-steps Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to establish…

Markov processes are used in a wide range of disciplines, including finance. The transition densities of these processes are often unknown. However, the conditional characteristic functions are more likely to be available, especially for…

I propose a large class of stochastic Markov processes associated with probability distributions analogous to that of lattice gauge theory with dynamical fermions. The construction incorporates the idea of approximate spectral split of the…

We classify the rare events of structured, memoryful stochastic processes and use this to analyze sequential and parallel generators for these events. Given a stochastic process, we introduce a method to construct a new process whose…