Related papers: A note on the gambling team method

In several standard models of dynamic programming (gambling houses, MDPs, POMDPs), we prove the existence of a very robust notion of value for the infinitely repeated problem, namely the pathwise uniform value. This solves two open…

We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…

We propose a new yet natural algorithm for learning the graph structure of general discrete graphical models (a.k.a. Markov random fields) from samples. Our algorithm finds the neighborhood of a node by sequentially adding nodes that…

Bailey showed that the general pointwise forecasting for stationary and ergodic time series has a negative solution. However, it is known that for Markov chains the problem can be solved. Morvai showed that there is a stopping time sequence…

The stochastic properties of a Langevin-type Markov process can be extracted from a given time series by a Markov analysis. Also processes that obey a stochastically forced second order differential equation can be analyzed this way by…

We introduce an information order on experiments based on weighted garbling, a generalization of the standard notion of garbling. In this order, an experiment is more informative than another if the latter is a weighted garbling of the…

We consider a continuous-time Markov chain model of SIR disease dynamics with two levels of mixing. For this so-called stochastic households model, we provide two methods for inferring the model parameters---governing within-household…

Matching games is a one-to-one two sided market model introduced by Garrido-Lucero and Laraki, in which coupled agents' utilities are endogenously determined as the outcome of a strategic game. They refine the classical pairwise stability…

Game-theoretic upper expectations are joint (global) probability models that mathematically describe the behaviour of uncertain processes in terms of supermartingales; capital processes corresponding to available betting strategies.…

We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…

We study "logit dynamics" [Blume, Games and Economic Behavior, 1993] for strategic games. This dynamics works as follows: at every stage of the game a player is selected uniformly at random and she plays according to a "noisy" best-response…

Markov chains are a convenient means of generating realizations of networks, since they require little more than a procedure for rewiring edges. If a rewiring procedure exists for generating new graphs with specified statistical properties,…

Simple stochastic exchange games are based on random allocation of finite resources. These games are Markov chains that can be studied either analytically or by Monte Carlo simulations. In particular, the equilibrium distribution can be…

In coordination games and speculative over-the-counter financial markets, solutions depend on higher-order average expectations: agents' expectations about what counterparties, on average, expect their counterparties to think, etc. We offer…

Individual cooperative strategy influences the surrounding dynamic population, which in turn affects cooperative strategy. To better model this phenomenon, we develop a Markov decision chain based game transitions model and examine the…

Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…

Characterizing whether a Markov process of discrete random variables has an homogeneous continuous-time realization is a hard problem. In practice, this problem reduces to deciding when a given Markov matrix can be written as the…

In this paper we prove a sharp quantitative version of the Kendall's Theorem. The Kendal Theorem states that under some mild conditions imposed on a probability distribution on positive integers (i.e. probabilistic sequence) one can prove…

In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…