Related papers: A bivariate two-state Markov modulated Poisson pro…
This paper proposes an adaptive stochastic Model Predictive Control (MPC) strategy for stable linear time invariant systems in the presence of bounded disturbances. We consider multi-input multi-output systems that can be expressed by a…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
This paper deals with asymptotic expressions of the Mean Time To Failure (MTTF) and higher moments for large, recursive, and non-repairable systems in the context of two-terminal reliability. Our aim is to extend the well-known results of…
We review basic modeling approaches for failure and maintenance data from repairable systems. In particular we consider imperfect repair models, defined in terms of virtual age processes, and the trend-renewal process which extends the…
We introduce a probabilistic framework for two-sample comparison based on a nonparametric process taking the form of a Markov model that transitions between a "divide" and a "merge" state on a multi-resolution partition tree of the sample…
This paper considers a general class of parameter-driven models for time series of counts. A comprehensive simulation study is conducted to evaluate the accuracy and efficiency of three estimators: the maximum likelihood estimators of the…
We present new algorithms for computing and approximating bisimulation metrics in Markov Decision Processes (MDPs). Bisimulation metrics are an elegant formalism that capture behavioral equivalence between states and provide strong…
Interval-censored data solely records the aggregated counts of events during specific time intervals - such as the number of patients admitted to the hospital or the volume of vehicles passing traffic loop detectors - and not the exact…
Markov state models (MSMs) have been demonstrated to be a powerful method for computationally studying intramolecular processes such as protein folding and macromolecular conformational changes. In this article, we present a new approach to…
Many scientific and engineering problems require to perform Bayesian inferences in function spaces, in which the unknowns are of infinite dimension. In such problems, many standard Markov Chain Monte Carlo (MCMC) algorithms become arbitrary…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…
We study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of…
We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an…
One-dimensional run-and-tumble processes may converge towards some localized non-equilibrium steady state when the two velocities and/or the two switching rates are space-dependent. A long dynamical trajectory can be then analyzed via the…
This paper presents a novel approach to stochastic mortality modelling by using the Conway--Maxwell--Poisson (CMP) distribution to model death counts. Unlike standard Poisson or negative binomial distributions, the CMP is a more adaptable…
Partially Observable Markov Decision Processes (POMDPs) are fundamental to many real-world applications. Although reinforcement learning (RL) has shown success in fully observable domains, learning policies from traces in partially…
Many real-world phenomena are naturally bivariate. This includes blood pressure, which comprises systolic and diastolic levels. Here, we develop a Bayesian hierarchical model that estimates these values and their interactions…
In this paper we consider the problem of parameter inference for Markov jump process (MJP) representations of stochastic kinetic models. Since transition probabilities are intractable for most processes of interest yet forward simulation is…
Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…
Sequential optimization methods are often confronted with the curse of dimensionality in high-dimensional spaces. Current approaches under the Gaussian process framework are still burdened by the computational complexity of tracking…