Related papers: Abstraction-based Probabilistic Stability Analysis…

Markov Decision Processes (Mdps) form a versatile framework used to model a wide range of optimization problems. The Mdp model consists of sets of states, actions, time steps, rewards, and probability transitions. When in a given state and…

Synthesising verifiably correct controllers for dynamical systems is crucial for safety-critical problems. To achieve this, it is important to account for uncertainty in a robust manner, while at the same time it is often of interest to…

This paper is concerned with a compositional approach for constructing abstractions of interconnected discrete-time stochastic control systems. The abstraction framework is based on new notions of so-called stochastic simulation functions,…

Sufficiently accurate finite state models, also called symbolic models or discrete abstractions, allow one to apply fully automated methods, originally developed for purely discrete systems, to formally reason about continuous and hybrid…

Can we conclude the stability of an unknown dynamical system from the knowledge of a finite number of snapshots of trajectories? We tackle this black-box problem for switched linear systems. We show that, for any given random set of…

Multivariate Hawkes Processes (MHPs) are a class of point processes that can account for complex temporal dynamics among event sequences. In this work, we study the accuracy and computational efficiency of three classes of algorithms which,…

Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuators via probabilities in the transition function. However, in…

In this work, we derive conditions under which compositional abstractions of networks of stochastic hybrid systems can be constructed using the interconnection topology and joint dissipativity-type properties of subsystems and their…

We consider distribution-based objectives for Markov Decision Processes (MDP). This class of objectives gives rise to an interesting trade-off between full and partial information. As in full observation, the strategy in the MDP can depend…

In this paper, we prove the exponential stability property of a class of mechanical systems represented in the port-Hamiltonian framework. To this end, we propose a Lyapunov candidate function different from the Hamiltonian of the system.…

In this paper, we study the Poisson stability (in particular, stationarity, periodicity, quasi-periodicity, Bohr almost periodicity, almost automorphy, recurrence in the sense of Birkhoff, Levitan almost periodicity, pseudo periodicity,…

Online planning in Markov Decision Processes (MDPs) enables agents to make sequential decisions by simulating future trajectories from the current state, making it well-suited for large-scale or dynamic environments. Sample-based methods…

In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity conditions for POMDPs and their belief-MDP reductions, where…

The consistency of Moving Particle Semi-implicit (MPS) method in reproducing the gradient, divergence and Laplacian differential operators is discussed in the present paper. Its relation to the Smoothed Particle Hydrodynamics (SPH) method…

Impulsive systems are a very flexible class of systems that can be used to represent switched and sampled-data systems. We propose to extend here the previously obtained results on deterministic impulsive systems to the stochastic setting.…

We study the asymptotic optimality of abstraction-based control synthesis algorithms. Specifically, we consider uncertain MDP (UMDP) abstraction, and investigate whether refinement leads to optimal results, i.e., an optimal controller and…

In this work, we consider systems that are subjected to intermittent instabilities due to external stochastic excitation. These intermittent instabilities, though rare, have a large impact on the probabilistic response of the system and…

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty…

This paper presents an adaptive horizon multi-stage model-predictive control (MPC) algorithm. It establishes appropriate criteria for recursive feasibility and robust stability using the theory of input-to-state practical stability (ISpS).…

We study sufficient conditions for stability and recurrence in a class of singularly perturbed stochastic hybrid dynamical systems. The systems considered combine multi-time-scale deterministic continuous-time dynamics, modeled by…