Related papers: Markovian Embeddings for Coalitional Bargaining Ga…
We learn the structure of a Markov Network between two groups of random variables from joint observations. Since modelling and learning the full MN structure may be hard, learning the links between two groups directly may be a preferable…
The problem of selecting the right state-representation in a reinforcement learning problem is considered. Several models (functions mapping past observations to a finite set) of the observations are given, and it is known that for at least…
Gerber and Li in \cite{GeLi} formulated, using a Markov chain embedding, a system of equations that describes relations between generating functions of waiting time distributions for occurrences of patterns in a sequence of independent…
This paper introduces the class of cooperative games with generalized coalition configuration. This new class of games corresponds to cooperative games with coalition configuration and restricted cooperation. A coalition configuration is a…
Evolutionary spatial 2 x 2 games between heterogeneous agents are analyzed using different variants of cellular automata (CA). Agents play repeatedly against their nearest neighbors 2 x 2 games specified by a rescaled payoff matrix with two…
We study a general formulation of the classical two-player Dynkin game in a discrete time Markovian setting. We identify an appropriate class of mixed strategies -- \textit{Markovian randomized stopping times} -- in which players stop at…
We propose an original model for inferring team strengths using a Markov Random Field, which can be used to generate historical estimates of the offensive and defensive strengths of a team over time. This model was designed to be applied to…
Dynamics of non-Markovian systems is a classic problem yet it attracts an everlasting activity in physics and beyond. A powerful tool for modeling such setups is the Generalized Langevin Equation, however, its analysis typically poses a…
This paper studies cooperative games where coalitions are formed online and the value generated by the grand coalition must be irrevocably distributed among the players at each timestep. We investigate the fundamental issue of strategic…
This paper studies stochastic games on large graphs and their graphon limits. We propose a new formulation of graphon games based on a single typical player's label-state distribution. In contrast, other recently proposed models of graphon…
We have recently defined a weak Markovian bisimulation equivalence in an integrated-time setting, which reduces sequences of exponentially timed internal actions to individual exponentially timed internal actions having the same average…
Bayesian regression games are a special class of two-player general-sum Bayesian games in which the learner is partially informed about the adversary's objective through a Bayesian prior. This formulation captures the uncertainty in regard…
Markovian processes have long been used to model stochastic environments. Reinforcement learning has emerged as a framework to solve sequential planning and decision-making problems in such environments. In recent years, attempts were made…
This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to non-differentiable objective functions and trades off exploration and exploitation to reduce the number of potentially…
We provide performance guarantees for a variant of simulation-based policy iteration for controlling Markov decision processes that involves the use of stochastic approximation algorithms along with state-of-the-art techniques that are…
An interaction system has a finite set of agents that interact pairwise, depending on the current state of the system. Symmetric decomposition of the matrix of interaction coefficients yields the representation of states by self-adjoint…
The practically important classes of equal-input and of monotone Markov matrices are revisited, with special focus on embeddability, infinite divisibility, and mutual relations. Several uniqueness results for the classic Markov embedding…
We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state…
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which…
Markov chains are an important example for a course on stochastic processes because simple board games can be used to illustrate the fundamental concepts. For example, a looping board game (like Monopoly) consists of all recurrent states,…