Related papers: Dense periodic optimization for countable Markov s…
In this work we consider the unbiased estimation of expectations w.r.t.~probability measures that have non-negative Lebesgue density, and which are known point-wise up-to a normalizing constant. We focus upon developing an unbiased method…
We develop a quadratic regularization approach for the solution of high-dimensional multistage stochastic optimization problems characterized by a potentially large number of time periods/stages (e.g. hundreds), a high-dimensional resource…
We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional…
We study those measures whose doubling constant is the least possible among doubling measures on a given metric space. It is shown that such measures exist on every metric space supporting at least one doubling measure. In addition, a…
A new algorithm for one-dimensional minimization is described in detail and the results of some tests on practical cases are reported and illustrated. The method requires only punctual computation of the function, and is suitable to be…
Countable Markov shifts, denoted by $\Sigma_A$ for a 0-1 infinite matrix $A$, are central objects in symbolic dynamics and ergodic theory. R. Exel and M. Laca introduced the corresponding operator algebras, a generalization of the…
This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is…
In this article, we consider the weighted ergodic optimization problem of a class of dynamical systems $T:X\to X$ where $X$ is a compact metric space and $T$ is Lipschitz continuous. We show that once $T:X\to X$ satisfies both the {\em…
We present some streamlined proofs of some of the basic results in Aubry-Mather theory (existence of quasi-periodic minimizers, multiplicity results when there are gaps among minimizers) based on the study of hull functions. We present…
We study a broad class of polynomial optimization problems whose constraints and objective functions exhibit sparsity patterns. We give two characterizations of the number of critical points to these problems, one as a mixed volume and one…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
This paper presents sufficient conditions for the existence of stationary optimal policies for average-cost Markov Decision Processes with Borel state and action sets and with weakly continuous transition probabilities. The one-step cost…
Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…
In this Letter, we clarify the physical origin of effective transport in periodic and tilted periodic systems. When Brownian dynamics is examined on the scale of a single period, the particle displacement admits a natural separation into a…
We consider the piecewise-deterministic Markov process obtained by randomly switching between the flows generated by a finite set of smooth vector fields on a compact set. We obtain H\"ormander-type conditions on the vector fields…
We introduce a theoretical approach for designing generalizations of the approximate message passing (AMP) algorithm for compressed sensing which are valid for large observation matrices that are drawn from an invariant random matrix…
The Markovian dynamics of a qubit is investigated in the scheme of random unitary dynamics, where Kraus operators are changed by an extra noise. The behavior of Markovianity is explored in the perturbed scenario. We provide a new algorithm…
We introduce a new method to accurately and efficiently estimate the effective dynamics of collective variables in molecular simulations. Such reduced dynamics play an essential role in the study of a broad class of processes, ranging from…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
We propose solution methods for previously-unsolved constrained MDPs in which actions can continuously modify the transition probabilities within some acceptable sets. While many methods have been proposed to solve regular MDPs with large…