Related papers: Statistical behavior of adaptive multilevel splitt…
This paper considers online optimization for a system that performs a sequence of back-to-back tasks. Each task can be processed in one of multiple processing modes that affect the duration of the task, the reward earned, and an additional…
We present two algorithms by which a set of short, unbiased trajectories can be iteratively reweighted to obtain various observables. The first algorithm estimates the stationary (steady state) distribution of a system by iteratively…
Stochastic point processes with refractoriness appear frequently in the quantitative analysis of physical and biological systems, such as the generation of action potentials by nerve cells, the release and reuptake of vesicles at a synapse,…
Adaptive experiments, including efficient average treatment effect estimation and multi-armed bandit algorithms, have garnered attention in various applications, such as social experiments, clinical trials, and online advertisement…
The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
We present a new method that enables the identification and analysis of both transition and metastable conformational states from atomistic or coarse-grained molecular dynamics (MD) trajectories. Our algorithm is presented and studied by…
Reinforcement learning (RL) under changing environment models many real-world applications via nonstationary Markov Decision Processes (MDPs), and hence gains considerable interest. However, theoretical studies on nonstationary MDPs in the…
Motion trajectory planning is one crucial aspect for automated vehicles, as it governs the own future behavior in a dynamically changing environment. A good utilization of a vehicle's characteristics requires the consideration of the…
Computing accurate rate constants for catalytic events occurring at the surface of a given material represents a challenging task with multiple potential applications in chemistry. To address this question, we propose an approach based on a…
Rare events such as conformational changes in biomolecules, phase transitions, and chemical reactions are central to the behavior of many physical systems, yet they are extremely difficult to study computationally because unbiased…
Stochastic resetting, the procedure of stopping and re-initializing random processes, has recently emerged as a powerful tool for accelerating processes ranging from queuing systems to molecular simulations. However, its usefulness is…
Distribution-dependent stochastic dynamical systems arise widely in engineering and science. We consider a class of such systems which model the limit behaviors of interacting particles moving in a vector field with random fluctuations. We…
The concept of limiting step gives the limit simplification: the whole network behaves as a single step. However, in its simplest form this idea is applicable only to the simplest linear cycles in steady states. For such the simplest cycles…
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…
Importance sampling of trajectories has proved a uniquely successful strategy for exploring rare dynamical behaviors of complex systems in an unbiased way. Carrying out this sampling, however, requires an ability to propose changes to…
Multi-agent Markov Decision Processes (MMDPs) arise in a variety of applications including target tracking, control of multi-robot swarms, and multiplayer games. A key challenge in MMDPs occurs when the state and action spaces grow…
In this paper we demonstrate a simple heuristic adaptive restart technique that can dramatically improve the convergence rate of accelerated gradient schemes. The analysis of the technique relies on the observation that these schemes…
The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally…
We propose a new Monte Carlo method for sampling from multimodal distributions. The idea of this technique is based on splitting the task into two: finding the modes of a target distribution $\pi$ and sampling, given the knowledge of the…