Related papers: General Discounting versus Average Reward

We consider reinforcement learning for continuous-time Markov decision processes (MDPs) in the infinite-horizon, average-reward setting. In contrast to discrete-time MDPs, a continuous-time process moves to a state and stays there for a…

We consider an ordinary differential equation with a unique hyperbolic attractor at the origin, to which we add a small random perturbation. It is known that under general conditions, the solution of this stochastic differential equation…

In many professons employees are rewarded according to their relative performance. Corresponding economy can be modeled by taking $N$ independent agents who gain from the market with a rate which depends on their current gain. We argue that…

We show that, with indivisible goods, the existence of competitive equilibrium fundamentally depends on agents' substitution effects, not their income effects. Our Equilibrium Existence Duality allows us to transport results on the…

There are situations in which an agent should receive rewards only after having accomplished a series of previous tasks. In other words, the reward that the agent receives is non-Markovian. One natural and quite general way to represent…

In the framework of evolutionary games with institutional reciprocity, limited incentives are at disposal for rewarding cooperators and punishing defectors. In the simplest case, it can be assumed that, depending on their strategies, all…

Standard reinforcement learning (RL) assumes that an agent can observe a reward for each state-action pair. However, in practical applications, it is often difficult and costly to collect a reward for each state-action pair. While there…

Reinforcement learning (RL) agents have traditionally been tasked with maximizing the value function of a Markov decision process (MDP), either in continuous settings, with fixed discount factor $\gamma < 1$, or in episodic settings, with…

Consider a game where Alice generates an integer and Bob wins if he can factor that integer. Traditional game theory tells us that Bob will always win this game even though in practice Alice will win given our usual assumptions about the…

In this paper, we investigate the Merton portfolio management problem in the context of non-exponential discounting. This gives rise to time-inconsistency of the decision-maker. If the decision-maker at time t=0 can commit his/her…

We study $n$-dimensional contests between two players with heterogeneous effort costs, where each dimension (battle) is modeled as a Tullock contest. Prize-allocation rules are identity-independent, budget-balanced, and weakly increasing in…

The classical policy gradient method is the theoretical and conceptual foundation of modern policy-based reinforcement learning (RL) algorithms. Most rigorous analyses of such methods, particularly those establishing convergence guarantees,…

Exploiting others is beneficial individually but it could also be detrimental globally. The reverse is also true: a higher cooperation level may change the environment in a way that is beneficial for all competitors. To explore the possible…

Once failure is irreversible, continuation payoffs cannot be meaningfully aggregated across strategies that differ in their survival properties. Standard scalar evaluation sidesteps this by arbitrarily completing payoffs beyond termination,…

Several practical applications of reinforcement learning involve an agent learning from past data without the possibility of further exploration. Often these applications require us to 1) identify a near optimal policy or to 2) estimate the…

We study approachability theory in the presence of constraints. Given a repeated game with vector payoffs, we characterize the pairs of sets (A,D) in the payoff space such that Player 1 can guarantee that the long-run average payoff…

We consider a number of questions related to tradeoffs between reward and regret in repeated gameplay between two agents. To facilitate this, we introduce a notion of $\textit{generalized equilibrium}$ which allows for asymmetric regret…

We develop theory and algorithms for average-reward on-policy Reinforcement Learning (RL). We first consider bounding the difference of the long-term average reward for two policies. We show that previous work based on the discounted return…

We study the problem of Inverse Reinforcement Learning (IRL) with an average-reward criterion. The goal is to recover an unknown policy and a reward function when the agent only has samples of states and actions from an experienced agent.…

As is well known, average-cost optimality inequalities imply the existence of stationary optimal policies for Markov Decision Processes with average costs per unit time, and these inequalities hold under broad natural conditions. This paper…