Related papers: A new matrix-infinite-product-form solution for up…
In this paper we propose augmented interval Markov chains (AIMCs): a generalisation of the familiar interval Markov chains (IMCs) where uncertain transition probabilities are in addition allowed to depend on one another. This new model…
Markov state models (MSMs) have become a popular approach for investigating the conformational dynamics of proteins and other biomolecules. MSMs are typically built from numerous molecular dynamics simulations by dividing the sampled…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
A family $\{Q_{\beta}\}_{\beta \geq 0}$ of Markov chains is said to exhibit $\textit{metastable mixing}$ with $\textit{modes}$ $S_{\beta}^{(1)},\ldots,S_{\beta}^{(k)}$ if its spectral gap (or some other mixing property) is very close to the…
We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…
We present a compact mixed integer program (MIP) for the backhaul profit maximization problem in which a freight carrier seeks to generate profit from an empty delivery vehicle's backhaul trip from its last scheduled delivery to its depot…
We consider stochastically modeled chemical reaction systems with mass-action kinetics and prove that a product-form stationary distribution exists for each closed, irreducible subset of the state space if an analogous deterministically…
Several important properties of positive semidefinite processes of Ornstein--Uhlenbeck type are analysed. It is shown that linear operators of the form $X\mapsto AX+XA^{\mathrm{T}}$ with $A\in M_d(\mathbb{R})$ are the only ones that can be…
This paper considers continuous-time block-monotone Markov chains (BMMCs) and their block-augmented truncations. We first introduce the block monotonicity and block-wise dominance relation for continuous-time Markov chains, and then provide…
This work provides complete description of Quasistationary Distributions (QSDs) for Markov chains with a unique absorbing state and an irreducible set of non-absorbing states. As is well-known, every QSD has an associated absorption…
This paper investigates the robust stabilisation of a class of fractional-order non-linear systems via fixed-order dynamic output feedback controller in terms of linear matrix inequalities (LMIs). The systematic stabilisation algorithm…
This paper addresses the problem of planning under uncertainty in large Markov Decision Processes (MDPs). Factored MDPs represent a complex state space using state variables and the transition model using a dynamic Bayesian network. This…
We derive an algorithm to compute satisfiability bounds for arbitrary {\omega}-regular properties in an Interval-valued Markov Chain (IMC) interpreted in the adversarial sense. IMCs generalize regular Markov Chains by assigning a range of…
We propose the segmented iHMM (siHMM), a hierarchical infinite hidden Markov model (iHMM) that supports a simple, efficient inference scheme. The siHMM is well suited to segmentation problems, where the goal is to identify points at which a…
This paper considers a BMAP/M/$\infty$ queue with a batch Markovian arrival process (BMAP) and an exponential service time distribution. We first prove that the BMAP/M/$\infty$ queue is stable if and only if the expectation of the logarithm…
This paper proposes a novel robust Model Predictive Control (MPC) scheme for linear discrete-time systems affected by model uncertainty described by interval matrices. The key feature of the proposed method is a bound on the uncertainty…
Markov Chain Monte Carlo (MCMC) is a class of algorithms to sample complex and high-dimensional probability distributions. The Metropolis-Hastings (MH) algorithm, the workhorse of MCMC, provides a simple recipe to construct reversible…
The Dynamic Modified Stochastic p-Median Problem (DMS-p-MP) is an important problem in supply chain network design, as it deals with the optimal location of facilities and the allocation of demand in a dynamic and uncertain environment. In…
For nonlinear multi-agent systems with high relative degrees, achieving formation control and obstacle avoidance in a distributed manner remains a significant challenge. To address this issue, we propose a novel distributed safety-critical…
This paper investigates the robust stabilisation of a class of fractional-order non-linear systems via fixed-order dynamic output feedback controller in terms of linear matrix inequalities (LMIs). The systematic stabilisation algorithm…