Related papers: A Comprehensive Method for Solving Finite-State Se…
Mathematical modeling of cardiac function can provide augmented simulation-based diagnosis tool for complementing and extending human understanding of cardiac diseases which represent the most common cause of worldwide death. As the…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can…
Simulations with high accuracy are an essential part of scientific research to accelerate the innovation process. They are especially useful for finding novel approaches or optimizing existing methods. Today, powerful software tools are…
Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…
This paper attempts to study the optimal stopping time for semi-Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov…
Partially observed Markov process (POMP) models, also known as hidden Markov models or state space models, are ubiquitous tools for time series analysis. The R package pomp provides a very flexible framework for Monte Carlo statistical…
New algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces are presented. These algorithms are based on a special technique of sequential…
Markov Decision Processes (MDPs) are stochastic optimization problems that model situations where a decision maker controls a system based on its state. Partially observed Markov decision processes (POMDPs) are generalizations of MDPs where…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
Semidefinite programs (SDPs) are a class of optimisation problems that find application in numerous areas of physics, engineering and mathematics. Semidefinite programming is particularly suited to problems in quantum physics and quantum…
We study planning problems where autonomous agents operate inside environments that are subject to uncertainties and not fully observable. Partially observable Markov decision processes (POMDPs) are a natural formal model to capture such…
Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors. We study a setting in which the sensors are only…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
We consider parametric Markov decision processes (pMDPs) that are augmented with unknown probability distributions over parameter values. The problem is to compute the probability to satisfy a temporal logic specification with any concrete…
Markov decision problems (MDPs) provide the foundations for a number of problems of interest to AI researchers studying automated planning and reinforcement learning. In this paper, we summarize results regarding the complexity of solving…
Sampling-based methods are widely adopted solutions for robot motion planning. The methods are straightforward to implement, effective in practice for many robotic systems. It is often possible to prove that they have desirable properties,…
This tutorial aims to provide signal processing (SP) and machine learning (ML) practitioners with vital tools, in an accessible way, to answer the question: How to deal with missing data? There are many strategies to handle incomplete…
Cardiovascular disease affects millions of people worldwide and its social and economic cost clearly motivates scientific research. Computer simulation can lead to a better understanding of cardiac physiology, and for pathology presents…
Hybrid systems, and Piecewise Deterministic Markov Processes in particular, are widely used to model and numerically study systems exhibiting multiple time scales in biochemical reaction kinetics and related areas. In this paper an almost…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…