Related papers: Probabilistic Resistive Switching Device modeling …
We introduce a probabilistic framework for two-sample comparison based on a nonparametric process taking the form of a Markov model that transitions between a "divide" and a "merge" state on a multi-resolution partition tree of the sample…
Switched linear systems are time-varying nonlinear systems whose dynamics switch between different modes, where each mode corresponds to different linear dynamics. They arise naturally to model unexpected failures, environment uncertainties…
We study infinite-horizon robust Markov decision processes (MDPs) on continuous state spaces with structured rectangular ambiguity set. The proposed ambiguity set falls within the convex hull of unknown generating kernels. We utilize the…
Wildfires pose an increasing threat to the safety and reliability of power systems, particularly in distribution networks located in fire-prone regions. To mitigate ignition risk from electrical infrastructure, utilities often employ safety…
We study finite state random dynamical systems (RDS) and their induced Markov chains (MC) as stochastic models for complex dynamics. The linear representation of deterministic maps in RDS are matrix-valued random variables whose…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…
Markov chain Monte Carlo methods have become standard tools in statistics to sample from complex probability measures. Many available techniques rely on discrete-time reversible Markov chains whose transition kernels build up over the…
Regime-switching processes contain two components: continuous component and discrete component, which can be used to describe a continuous dynamical system in a random environment. Such processes have many different properties than general…
Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
Multi-agent systems can be successfully described by kinetic models, which allow one to explore the large scale aggregate trends resulting from elementary microscopic interactions. The latter may be formalised as collision-like rules, in…
Memristive systems and devices are potentially available for implementing reservoir computing (RC) systems applied to pattern recognition. However, the computational ability of memristive RC systems depends on intertwined factors such as…
In this paper, we consider an integrated MSP-MDP framework which captures features of Markov decision process (MDP) and multistage stochastic programming (MSP). The integrated framework allows one to study a dynamic decision-making process…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
In this paper, we explore lifting Markov Decision Processes (MDPs) to the space of probability measures and consider the so-called measurized MDPs: deterministic processes where states are probability measures on the original state space,…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…
We consider the problem of remanufacturing planning in the presence of statistical estimation errors. Determining the optimal remanufacturing timing, first and foremost, requires modeling of the state transitions of a system. The estimation…
A novel approach to simulate simple protein-ligand systems at large time- and length-scales is to couple Markov state models (MSMs) of molecular kinetics with particle-based reaction-diffusion (RD) simulations, MSM/RD. Currently, MSM/RD…