Related papers: Robust Parametric Inference for Finite Markov Chai…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
Distributed parameter systems (DPS) are formulated as partial differential equations (PDE). Especially, under time-varying boundary conditions, PDE introduce force coupling. In the case of the flexible stacker crane (STC), nonlinear…
We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax…
Accelerated life-tests (ALTs) are used for inferring lifetime characteristics of highly reliable products. In particular, step-stress ALTs increase the stress level at which units under test are subject at certain pre-fixed times, thus…
Markov Chain Monte Carlo (MCMC) techniques are now widely used for cosmological parameter estimation. Chains are generated to sample the posterior probability distribution obtained following the Bayesian approach. An important issue is how…
Moderate deviation principles for empirical measure processes associated with weakly interacting Markov processes are established. Two families of models are considered: the first corresponds to a system of interacting diffusions whereas…
In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
In this work, we study the problem of actively classifying the attributes of dynamical systems characterized as a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively interact with the…
We study safe, data-driven control of (Markov) jump linear systems with unknown transition probabilities, where both the discrete mode and the continuous state are to be inferred from output measurements. To this end, we develop a receding…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
Markov decision processes (MDPs) are a fundamental model in sequential decision making. Robust MDPs (RMDPs) extend this framework by allowing uncertainty in transition probabilities and optimizing against the worst-case realization of that…
Robust Markov Decision Processes (RMDPs) generalize classical MDPs that consider uncertainties in transition probabilities by defining a set of possible transition functions. An objective is a set of runs (or infinite trajectories) of the…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…
McKean-Vlasov stochastic differential equations (MVSDEs) describe systems whose dynamics depend on both individual states and the population distribution, and they arise widely in neuroscience, finance, and epidemiology. In many…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
Many modern products are highly reliable, often exhibiting long lifetimes. As a result, conducting experiments under normal operating conditions can be prohibitively time-consuming to collect sufficient failure data for robust statistical…
We develop a generic policy gradient method with the global optimality guarantee for robust Markov Decision Processes (MDPs). While policy gradient methods are widely used for solving dynamic decision problems due to their scalable and…
Stochastic Differential Equations (SDEs) serve as a powerful modeling tool in various scientific domains, including systems science, engineering, and ecological science. While the specific form of SDEs is typically known for a given…
This paper investigates a series of optimization problems for one-counter Markov decision processes (MDPs) and integer-weighted MDPs with finite state space. Specifically, it considers problems addressing termination probabilities and…