Related papers: State Aggregation Learning from Markov Transition …
We study dynamic discrete choice models, where a commonly studied problem involves estimating parameters of agent reward functions (also known as "structural" parameters), using agent behavioral data. Maximum likelihood estimation for such…
Analysis of Markov Decision Processes (MDP) is often hindered by state space explosion. Abstraction is a well-established technique in model checking to mitigate this issue. This paper presents a novel lazy abstraction method for MDP…
This paper introduces new efficient algorithms for two problems: sampling conditional on vertex degrees in unweighted graphs, and sampling conditional on vertex strengths in weighted graphs. The algorithms can sample conditional on the…
A mapping of the process on a continuous configuration space to the symbolic representation of the motion on a discrete state space will be combined with an iterative aggregation and disaggregation (IAD) procedure to obtain steady state…
We study the problem of estimating latent population flows from aggregated count data. This problem arises when individual trajectories are not available due to privacy issues or measurement fidelity. Instead, the aggregated observations…
We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions.…
The configuration model is a standard tool for uniformly generating random graphs with a specified degree sequence, and is often used as a null model to evaluate how much of an observed network's structure can be explained by its degree…
Performance-based engineering for natural hazards facilitates the design and appraisal of structures with rigorous evaluation of their uncertain structural behavior under potentially extreme stochastic loads expressed in terms of failure…
State aggregation aims to reduce the computational complexity of solving Markov Decision Processes (MDPs) while preserving the performance of the original system. A fundamental challenge lies in optimizing policies within the aggregated, or…
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a…
The problem of real-time remote tracking and reconstruction of a two-state Markov process is considered here. A transmitter sends samples from an observed information source to a remote monitor over an unreliable wireless channel. The…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
We propose a new sampling algorithm combining two quite powerful ideas in the Markov chain Monte Carlo literature -- adaptive Metropolis sampler and two-stage Metropolis-Hastings sampler. The proposed sampling method will be particularly…
Density tempering (also called density annealing) is a sequential Monte Carlo approach to Bayesian inference for general state models; it is an alternative to Markov chain Monte Carlo. When applied to state space models, it moves a…
A popular approach to solving large probabilistic systems relies on aggregating states based on a measure of similarity. Many approaches in the literature are heuristic. A number of recent methods rely instead on metrics based on the notion…
We propose a class of discrete state sampling algorithms based on Nesterov's accelerated gradient method, which extends the classical Metropolis-Hastings (MH) algorithm. The evolution of the discrete states probability distribution governed…
Modeling and simulating movement of vehicles in established transportation infrastructures, especially in large urban road networks is an important task. It helps with understanding and handling traffic problems, optimizing traffic…
To understand the long-run behavior of Markov population models, the computation of the stationary distribution is often a crucial part. We propose a truncation-based approximation that employs a state-space lumping scheme, aggregating…
We introduce Path-ZVA: an efficient simulation technique for estimating the probability of reaching a rare goal state before a regeneration state in a (discrete-time) Markov chain. Standard Monte Carlo simulation techniques do not work well…
We study the optimal transport problem for pairs of stationary finite-state Markov chains, with an emphasis on the computation of optimal transition couplings. Transition couplings are a constrained family of transport plans that capture…