Related papers: Late comment on Astumian's paradox
This note presents a simple proof of the monotonicity of the invariant distribution of a discrete Markov chain with a finite state space. This answers a question recently raised by David Siegmund.
Motivated by applications in Markov chain Monte Carlo, we discuss what it means for one Markov chain to be an approximation to another. Specifically included in that discussion are situations in which a Markov chain with continuous state…
A basic question for zero-sum repeated games consists in determining whether the mean payoff per time unit is independent of the initial state. In the special case of "zero-player" games, i.e., of Markov chains equipped with additive…
We consider a scenario where N users try to access a common base station. Associated with each user is its channel state and a finite queue which varies with time. Each user chooses his power and the admission control variable in a dynamic…
We study a novel control problem in the context of network coordination games: the individuation of the smallest set of players capable of driving the system, globally, from one Nash equilibrium to another one. Our main contribution is the…
Stochastic games are an important class of problems that generalize Markov decision processes to game theoretic scenarios. We consider finite state two-player zero-sum stochastic games over an infinite time horizon with discounted rewards.…
Consider a system evolving according to an absorbing discrete-time Markov chain with known transition matrix. The state of the system is observed at two points in time, separated by an unknown number of generations. We are interested in…
This study provides a model of cyber deception with asymmetric recognition represented by private beliefs. Signaling games, which are often used in existing works, are built on the implicit premise that the receiver's belief is public…
In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…
This paper considers the discounted criterion of nonzero-sum decentralized stochastic games with prospect players. The state and action spaces are finite. The state transition probability is nonstationary. Each player independently controls…
For an ascending correspondence $F:X\to 2^X$ with chain-complete values on a complete lattice $X$, we prove that the set of fixed points is a complete lattice. This strengthens Zhou's fixed point theorem. For chain-complete posets that are…
This paper introduces state abstraction for two-player zero-sum Markov games (TZMGs), where the payoffs for the two players are determined by the state representing the environment and their respective actions, with state transitions…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…
We study the limiting object of a sequence of Markov chains analogous to the limits of graphs, hypergraphs, and other objects which have been studied. Following a suggestion of Aldous, we assign to a sequence of finite Markov chains with…
Game theory is the standard tool used to model strategic interactions in evolutionary biology and social science. Traditional game theory studies the equilibria of simple games. But is traditional game theory applicable if the game is…
Gerber and Li in \cite{GeLi} formulated, using a Markov chain embedding, a system of equations that describes relations between generating functions of waiting time distributions for occurrences of patterns in a sequence of independent…
The partial sum of the states of a Markov chain or more generally a Markov source is asymptotically normally distributed under suitable conditions. One of these conditions is that the variance is unbounded. A simple combinatorial…
In this paper, we present a conceptual model game to examine the dynamics of asymmetric interactions in games with imperfect information. The game involves two agents with starkly contrasting capabilities: one agent can take actions but has…
It is shown that transient graphs for the simple random walk do not admit a nearest neighbor transient Markov chain (not necessarily a reversible one), that crosses all edges with positive probability, while there is such chain for the…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…