Related papers: Markov Chains for Collaboration
Card shuffling models have provided simple motivating examples for the mathematical theory of mixing times for Markov chains. As a complement, we introduce a more intricate realistic model of a certain observable real-world scheme for…
A well-studied randomized election algorithm proceeds as follows: In each round the remaining candidates each toss a coin and leave the competition if they obtain heads. Of interest is the number of rounds required and the number of…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
Consider a system evolving according to an absorbing discrete-time Markov chain with known transition matrix. The state of the system is observed at two points in time, separated by an unknown number of generations. We are interested in…
Individual cooperative strategy influences the surrounding dynamic population, which in turn affects cooperative strategy. To better model this phenomenon, we develop a Markov decision chain based game transitions model and examine the…
Gerber and Li in \cite{GeLi} formulated, using a Markov chain embedding, a system of equations that describes relations between generating functions of waiting time distributions for occurrences of patterns in a sequence of independent…
We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…
The paper introduces a modified version of the classical Coupon Collector's Problem entailing exchanges and cooperation between multiple players. Results of the development show that, within a proper Markov framework, the complexity of the…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
Understanding the strategic behavior of miners in a blockchain is of great importance for its proper operation. A common model for mining games considers an infinite time horizon, with players optimizing asymptotic average objectives.…
Markov chains are an important example for a course on stochastic processes because simple board games can be used to illustrate the fundamental concepts. For example, a looping board game (like Monopoly) consists of all recurrent states,…
A wide class of ``counting'' problems have been studied in Computer Science. Three typical examples are the estimation of - (i) the permanent of an $n\times n$ 0-1 matrix, (ii) the partition function of certain $n-$ particle Statistical…
We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of…
We consider a nonzero-sum N-player Markov game on an abstract measurable state space with compact metric action spaces. The payoff functions are bounded Carath\'eodory functions and the transitions of the system are assumed to have a…
We consider a scenario where N users try to access a common base station. Associated with each user is its channel state and a finite queue which varies with time. Each user chooses his power and the admission control variable in a dynamic…
We introduce a deterministic analogue of Markov chains that we call the hunger game. Like rotor-routing, the hunger game deterministically mimics the behavior of both recurrent Markov chains and absorbing Markov chains. In the case of…
We study a two-player, zero-sum, stochastic game with incomplete information on one side in which the players are allowed to play more and more frequently. The informed player observes the realization of a Markov chain on which the payoffs…
An interaction system has a finite set of agents that interact pairwise, depending on the current state of the system. Symmetric decomposition of the matrix of interaction coefficients yields the representation of states by self-adjoint…
We study the expected accumulated reward for a discrete-time Markov reward model with absorbing states. The rewards are impulse rewards, where a reward $\rho_{ij}$ is accumulated when transitioning from state $i$ to state $j$. We derive an…
In a zero-sum stochastic game with signals, at each stage, two adversary players take decisions and receive a stage payoff determined by these decisions and a variable called state. The state follows a Markov chain, that is controlled by…