Related papers: Markov Chains for Collaboration
We consider the problem of estimating the expected time to find a maximum degree node on a graph using a (parameterized) biased random walk. For assortative graphs the positive degree correlation serves as a local gradient for which a bias…
We consider multi-dimensional payoff functions in partially observable Markov decision processes. We study the structure of the set of expected payoff vectors of all strategies (policies) and study what kind are needed to achieve a given…
We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an…
Evolutionary spatial 2 x 2 games between heterogeneous agents are analyzed using different variants of cellular automata (CA). Agents play repeatedly against their nearest neighbors 2 x 2 games specified by a rescaled payoff matrix with two…
In this paper, we consider the Markov-Chain Monte Carlo (MCMC) approach for random sampling of combinatorial objects. The running time of such an algorithm depends on the total mixing time of the underlying Markov chain and is unknown in…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
In this paper, the problem of random access contention between machine type devices (MTDs) in the uplink of a wireless cellular network is studied. In particular, the possibility of forming cooperative groups to coordinate the MTDs'…
Acting as a goalkeeper in a video-game, a participant is asked to predict the successive choices of the penalty taker. The sequence of choices of the penalty taker is generated by a stochastic chain with memory of variable length. It has…
We study "logit dynamics" [Blume, Games and Economic Behavior, 1993] for strategic games. This dynamics works as follows: at every stage of the game a player is selected uniformly at random and she plays according to a "noisy" best-response…
Clustering algorithms frequently require the number of clusters to be chosen in advance, but it is usually not clear how to do this. To tackle this challenge when clustering within sequential data, we present a method for estimating the…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…
We consider a scenario where $N$ users send packets to a common access point. The receiver decodes the message of each user by treating the other user's signals as noise. Associated with each user is its channel state and a finite queue…
We model and study the game mechanisms and human behavior of the anarchy mode in Twitch Plays Pokemon with a pure-jump continuous-time Markov process. We computed the winning probability and expected game time for $1$ player and $N$ players…
The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior…
Optimization under uncertainty is a fundamental problem in learning and decision-making, particularly in multi-agent systems. Previously, Feldman, Kalai, and Tennenholtz [2010] demonstrated the ability to efficiently compete in repeated…
Various social dilemma games that follow different strategy updating rules have been studied on many networks.The reported results span the entire spectrum, from significantly boosting,to marginally affecting,to seriously decreasing the…
Distributed decision-makers are modeled as players in a game with two levels. High level decisions concern the game environment and determine the willingness of the players to form a coalition (or group). Low level decisions involve the…
In team-based invasion sports such as soccer and basketball, analytics is important for teams to understand their performance and for audiences to understand matches better. The present work focuses on performing visual analytics to…
We consider exchangeable Markov multi-state survival processes -- temporal processes taking values over a state-space$\mathcal{S}$ with at least one absorbing failure state $\flat \in \mathcal{S}$ that satisfy natural invariance properties…