Related papers: Markov-Modulated Affine Processes
Based on the theory of multivariate time changes for Markov processes, we show how to identify affine processes as solutions of certain time change equations. The result is a strong version of a theorem presented by J. Kallsen (2006) which…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…
We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as…
We study properties of a subclass of Markov processes that have all moments that are continuous functions of the time parameter and more importantly are characterized by the property that say their $n-$th conditional moment given the past…
Markov decision processes can be viewed as transformers of probability distributions. While this view is useful from a practical standpoint to reason about trajectories of distributions, basic reachability and safety problems are known to…
Coarse-graining is a standard method of extracting a simple Markov process from a more complicated one by identifying states. Here we extend coarse-graining to open Markov processes. An "open" Markov process is one where probability can…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
We introduce the problem of formally verifying properties of Markov processes where the parameters are given by the output of machine learning models. For a broad class of machine learning models, including linear models, tree-based models,…
Diffusion models are typically trained using score matching, a learning objective agnostic to the underlying noising process that guides the model. This paper argues that Markov noising processes enjoy an advantage over alternatives, as the…
The class of stochastic matrices that have a stochastic $c$-th root for infinitely many natural numbers $c$ is introduced and studied. Such matrices are called arbitrarily finely divisible, and generalise the class of infinitely divisible…
We construct a family of self-similar Markov martingales with given marginal distributions. This construction uses the self-similarity and Markov property of a reference process to produce a family of Markov processes that possess the same…
Universal machine-learned interatomic potentials (U-MLIPs) have demonstrated effectiveness across diverse atomistic systems but often require fine-tuning for task-specific accuracy. We investigate the fine-tuning of two MACE-based…
We introduce the Lyapunov approach to optimal control problems of average risk-sensitive Markov control processes with general risk maps. Motivated by applications in particular to behavioral economics, we consider possibly non-convex risk…
We consider Markov decision processes (MDPs) which are a standard model for probabilistic systems. We focus on qualitative properties for MDPs that can express that desired behaviors of the system arise almost-surely (with probability 1) or…
We have shown recently that a Markov process conditioned on rare events involving time-integrated random variables can be described in the long-time limit by an effective Markov process, called the driven process, which is given…
Machine learning interatomic potentials (MLIPs) have become powerful tools to extend molecular simulations beyond the limits of quantum methods, offering near-quantum accuracy at much lower computational cost. Yet, developing reliable MLIPs…
We exhibit conditions under which the flow of marginal distributions of a discontinuous semimartingale $\xi$ can be matched by a Markov process, whose infinitesimal generator is expressed in terms of the local characteristics of $\xi$. Our…
Planning problems where effects of actions are non-deterministic can be modeled as Markov decision processes. Planning problems are usually goal-directed. This paper proposes several techniques for exploiting the goal-directedness to…
We show that, for generative classifiers, conditional independence corresponds to linear constraints for the induced discrimination functions. Discrimination functions of undirected Markov network classifiers can thus be characterized by…
General purpose intelligent learning agents cycle through (complex,non-MDP) sequences of observations, actions, and rewards. On the other hand, reinforcement learning is well-developed for small finite state Markov Decision Processes…