Related papers: Asymptotic optimality of the cross-entropy method …
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
We consider a general class of high-volume, fast-moving production-inventory systems based on both lost-sales and backorder inventory models. Such systems require a fundamental understanding of the asymptotic behavior of key performance…
We introduce the concept of asymptotic period for an irreducible and aperiodic, discrete-time Markov chain X on a countable state space, and develop the theory leading to its formal definition. The asymptotic period of X equals one - its…
The extremal behaviour of a Markov chain is typically characterized by its tail chain. For asymptotically dependent Markov chains existing formulations fail to capture the full evolution of the extreme event when the chain moves out of the…
We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…
The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
As is well known, average-cost optimality inequalities imply the existence of stationary optimal policies for Markov Decision Processes with average costs per unit time, and these inequalities hold under broad natural conditions. This paper…
We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms. Such algorithms provide unbiased estimators for equilibrium expectations associated with real- valued functionals defined on a Markov chain. We…
We consider minimizing the probability of falling below a target growth rate of the wealth process up to a time horizon $T$ in an incomplete market model, and then study the asymptotic behavior of minimizing probability as $T\to\infty$.…
MCMC methods are used in Bayesian statistics not only to sample from posterior distributions but also to estimate expectations. Underlying functions are most often defined on a continuous state space and can be unbounded. We consider a…
A problem from thermodynamic formalism for countable symbolic Markov chains is considered. It concerns asymptotic behavior of the equilibrium measures corresponding to increasing sequences of finite sub-matrices of an infinite nonnegative…
We investigate the problem of synthesizing optimal control policies for Markov decision processes (MDPs) with both qualitative and quantitative objectives. Specifically, our goal is to achieve a given linear temporal logic (LTL) task with…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
By means of two simple convexity arguments we are able to develop a general method for proving consistency and asymptotic normality of estimators that are defined by minimisation of convex criterion functions. This method is then applied to…
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…
Covariate balance is a conventional key diagnostic for methods used estimating causal effects from observational studies. Recently, there is an emerging interest in directly incorporating covariate balance in the estimation. We study a…
This paper considers a new approach to using Markov chain Monte Carlo (MCMC) in contexts where one may adopt multilevel (ML) Monte Carlo. The underlying problem is to approximate expectations w.r.t. an underlying probability measure that is…
In this paper we study the problem of computing max-entropy distributions over a discrete set of objects subject to observed marginals. Interest in such distributions arises due to their applicability in areas such as statistical physics,…
The Markovian evolution of an open quantum system is characterized by a positive entropy production, while the global entropy gets redistributed between the system and the environment degrees of freedom. Starting from these premises, we…