Related papers: Foundation Inference Models for Markov Jump Proces…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
We study the fluctuations of systems modeled by Markov jump processes with periodic generators. We focus on observables defined through time-periodic functions of the system's states or transitions. Using large deviation theory, canonical…
Multi-agent systems can be successfully described by kinetic models, which allow one to explore the large scale aggregate trends resulting from elementary microscopic interactions. The latter may be formalised as collision-like rules, in…
Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…
Flowgraph models provide an alternative approach in modeling a multi-state stochastic process. One of the most widely used stochastic processes that have many real-world applications especially in actuarial models is the Markov jump process…
With the increasing ubiquity of safety-critical autonomous systems operating in uncertain environments, there is a need for mathematical methods for formal verification of stochastic models. Towards formally verifying properties of…
We propose a unified framework that extends the inference methods for classical hidden Markov models to continuous settings, where both the hidden states and observations occur in continuous time. Two different settings are analyzed: hidden…
We present new methodologies for Bayesian inference on the rate parameters of a discretely observed continuous-time Markov jump processes with a countably infinite state space. The usual method of choice for inference, particle Markov chain…
Automated synthesis of provably correct controllers for cyber-physical systems is crucial for deployment in safety-critical scenarios. However, hybrid features and stochastic or unknown behaviours make this problem challenging. We propose a…
We propose moment-based variational inference as a flexible framework for approximate smoothing of latent Markov jump processes. The main ingredient of our approach is to partition the set of all transitions of the latent process into…
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…
We study the synthesis of mode switching protocols for a class of discrete-time switched linear systems in which the mode jumps are governed by Markov decision processes (MDPs). We call such systems MDP-JLS for brevity. Each state of the…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
Switched systems are capable of modeling processes with underlying dynamics that may change abruptly over time. To achieve accurate modeling in practice, one may need a large number of modes, but this may in turn increase the model…
The velocity-jump model is a specific type of piecewise deterministic Markov process in which an individual's velocity is constant except at times that form the events of some point process. It represents an interpretable continuous-time…
We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…
Diffusive approximations of Markov jump processes often fail to accurately capture large fluctuations. This is confounding, as the rare events triggered by these large fluctuations, such as the failure of electronic memories, are often the…
Conventional imitation learning assumes access to the actions of demonstrators, but these motor signals are often non-observable in naturalistic settings. Additionally, sequential decision-making behaviors in these settings can deviate from…