Related papers: Explicit-Duration Markov Switching Models
Markov automata (MAs) extend labelled transition systems with random delays and probabilistic branching. Action-labelled transitions are instantaneous and yield a distribution over states, whereas timed transitions impose a random delay…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
Switching state-space models (SSSM) are a very popular class of time series models that have found many applications in statistics, econometrics and advanced signal processing. Bayesian inference for these models typically relies on Markov…
We study the long-term qualitative behavior of randomly perturbed dynamical systems. More specifically, we look at limit cycles of stochastic differential equations (SDE) with Markovian switching, in which the process switches at random…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
Robust Markov decision processes (MDPs) aim to handle changing or partially known system dynamics. To solve them, one typically resorts to robust optimization methods. However, this significantly increases computational complexity and…
In this paper, we consider a discrete time economy where we assume that the short term interest rate follows a quadratic term structure of a regime switching asset process. The possible non-linear structure and the fact that the interest…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
Structural results impose sufficient conditions on the model parameters of a Markov decision process (MDP) so that the optimal policy is an increasing function of the underlying state. The classical assumptions for MDP structural results…
We give a short overview of recent results on a specific class of Markov process: the Piecewise Deterministic Markov Processes (PDMPs). We first recall the definition of these processes and give some general results. On more specific cases…
Learning-based approaches to verifying unknown Markov decision processes (MDPs) often employ uncertain MDPs. These models use, for example, confidence intervals to capture transition uncertainty and allow synthesis of policies that are…
We propose a constructive approach to building temporal point processes that incorporate dependence on their history. The dependence is modeled through the conditional density of the duration, i.e., the interval between successive event…
Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…
A Markov network characterizes the conditional independence structure, or Markov property, among a set of random variables. Existing work focuses on specific families of distributions (e.g., exponential families) and/or certain structures…
In general terms, intermittency is the property for which time evolving systems alternate among two or more different regimes. Predicting the instance when the regime switch will occur is extremely challenging, often practically impossible.…
In medicine, comorbidities refer to the presence of multiple, co-occurring diseases. Due to their co-occurring nature, the course of one comorbidity is often highly dependent on the course of the other disease and, hence, treatments can…
Markov models of character substitution on phylogenies form the foundation of phylogenetic inference frameworks. Early models made the simplifying assumption that the substitution process is homogeneous over time and across sites in the…
Non-equilibrium Markov State Modeling (MSM) has recently been proposed [Phys. Rev. E 94, 053001 (2016)] as a possible route to construct a physical theory of sliding friction from a long steady state atomistic simulation: the approach…
Probabilistic model checking can provide formal guarantees on the behavior of stochastic models relating to a wide range of quantitative properties, such as runtime, energy consumption or cost. But decision making is typically with respect…
In this work, a data-driven modeling framework of switched dynamical systems under time-dependent switching is proposed. The learning technique utilized to model system dynamics is Extreme Learning Machine (ELM). First, a method is…