Related papers: On the Modelling of Impulse Control with Random Ef…
Multi-state models are frequently applied for representing processes evolving through a discrete set of state. Important classes of multi-state models arise when transitions between states may depend on the time since entry into the current…
We study a class of Markov chains that model the evolution of a quantum system subject to repeated measurements. Each Markov chain in this class is defined by a measure on the space of matrices. It is then given by a random product of…
This paper considers the permissive supervisor synthesis for probabilistic systems modeled as Markov Decision Processes (MDP). Such systems are prevalent in power grids, transportation networks, communication networks and robotics. Unlike…
Markov decision process models and algorithms can be used to identify optimal policies for dispatching ambulances to spatially distributed customers, where the optimal policies indicate the ambulance to dispatch to each customer type in…
This paper considers a risk-constrained infinite-horizon optimal control problem and proposes to solve it in an iterative manner. Each iteration of the algorithm generates a trajectory from the starting point to the target equilibrium state…
Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to…
The task of maintaining a predefined level of effect in a dynamical plant by applying periodic control actions often arises in e.g. process control and medicine. When the state variables of the plant represent the concentrations of chemical…
The computation of the probability of the first-passage time through a given threshold of a stochastic process is a classic problem that appears in many branches of physics. When the stochastic dynamics is markovian, the probability admits…
Robotic systems must be able to quickly and robustly make decisions when operating in uncertain and dynamic environments. While Reinforcement Learning (RL) can be used to compute optimal policies with little prior knowledge about the…
Partially observed Markov process (POMP) models are powerful tools for time series modeling and analysis. Inherited the flexible framework of R package pomp, the is2 package extends some useful Monte Carlo statistical methodologies to…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
We consider control and inference problems where control protocols and internal dynamics are informed by two types of constraints. Our data consist of i) statistics on the ensemble and ii) trajectories or final disposition of selected…
This study in centered on models accounting for stochastic deformations of sample paths of random walks, embedded either in $\mathbb{Z}^2$ or in $\mathbb{Z}^3$. These models are immersed in multi-type particle systems with exclusion.…
In this paper, we consider a modified version of the control problem in a model free Markov decision process (MDP) setting with large state and action spaces. The control problem most commonly addressed in the contemporary literature is to…
In this paper, a sampling-based Stochastic Model Predictive Control algorithm is proposed for discrete-time linear systems subject to both parametric uncertainties and additive disturbances. One of the main drivers for the development of…
Stochastic policies (also known as relaxed controls) are widely used in continuous-time reinforcement learning algorithms. However, executing a stochastic policy and evaluating its performance in a continuous-time environment remain open…
In this paper, we consider an extension of the Poisson random measure for the formulation of continuous-time reinforcement learning, such that both the frequency and the width of the jumps depend on the path. Starting from a general point…
In this paper, we consider the problem of safety assessment for Markov decision processes without explicit knowledge of the model. We aim to learn probabilistic safety specifications associated with a given policy without compromising the…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…