Related papers: Factoring Exogenous State for Model-Free Monte Car…

We generalize the Time-Dependent Variational Principle (TDVP) to dissipative systems using Monte Carlo methods, allowing the application of existing variational classes for pure states, such as Matrix Product States (MPS), to the simulation…

Equation-free modeling aims at extracting low-dimensional macroscopic dynamics from complex high-dimensional systems that govern the evolution of microscopic states. This algorithm relies on lifting and restriction operators that map…

Quantitative theory of interbilayer interactions is essential to interpret x-ray scattering data and to elucidate these interactions for biologically relevant systems. For this purpose Monte Carlo simulations have been performed to obtain…

We present a new unbiased algorithm that estimates the expected value of f(U) via Monte Carlo simulation, where U is a vector of d independent random variables, and f is a function of d variables. We assume that f does not depend equally on…

Computational modeling of assembly is challenging for many systems because their timescales vastly exceed those accessible to simulations. This article describes the MultiMSM, which is a general framework that uses Markov state models…

You are a robot and you live in a Markov decision process (MDP) with a finite or an infinite number of transitions from state-action to next states. You got brains and so you plan before you act. Luckily, your roboparents equipped you with…

We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is…

Analog quantum simulation based on ultracold atoms in optical lattices has catalyzed significant breakthroughs in the study of quantum many-body systems. These simulations rely on the statistical sampling of electronic Fock states, which…

State-space models can be used to incorporate subject knowledge on the underlying dynamics of a time series by the introduction of a latent Markov state-process. A user can specify the dynamics of this process together with how the state…

We consider a robust approach to address uncertainty in model parameters in Markov Decision Processes (MDPs), which are widely used to model dynamic optimization in many applications. Most prior works consider the case where the uncertainty…

While general quantum many-body systems require exponential resources to be simulated on a classical computer, systems of non-interacting fermions can be simulated exactly using polynomially scaling resources. Such systems may be of…

Current model-free adaptive control (MFAC) can hardly deal with the time delay problem in multiple-input multiple-output (MIMO) systems. To solve this problem, a novel model-free adaptive predictive control (MFAPC) method is proposed.…

Planning under partial obervability is essential for autonomous robots. A principled way to address such planning problems is the Partially Observable Markov Decision Process (POMDP). Although solving POMDPs is computationally intractable,…

In 1950, Forsythe and Leibler (1950) introduced a statistical technique for finding the inverse of a matrix by characterizing the elements of the matrix inverse as expected values of a sequence of random walks. Barto and Duff (1994)…

In dynamic Monte Carlo simulations, using for example the Metropolis dynamic, it is often required to simulate for long times and to simulate large systems. We present an overview of advanced algorithms to simulate for larger times and to…

Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…

In this paper, we consider the problem of safety assessment for Markov decision processes without explicit knowledge of the model. We aim to learn probabilistic safety specifications associated with a given policy without compromising the…

Addressing the simultaneous identification of contributory variables while controlling the false discovery rate (FDR) in high-dimensional data is a crucial statistical challenge. In this paper, we propose a novel model-free variable…

In this paper we consider fully Bayesian inference in general state space models. Existing particle Markov chain Monte Carlo (MCMC) algorithms use an augmented model that takes into account all the variable sampled in a sequential Monte…

We report a new multicanonical Monte Carlo algorithm to obtain the density of states for physical systems with continuous state variables in statistical mechanics. Our algorithm is able to obtain a closed-form expression for the density of…