Related papers: Counterexamples to "Transitive Regret"
We study preferences estimated from finite choice experiments and provide sufficient conditions for convergence to a unique underlying "true" preference. Our conditions are weak, and therefore valid in a wide range of economic environments.…
We consider the classical stochastic multi-armed bandit but where, from time to time and roughly with frequency $\epsilon$, an extra observation is gathered by the agent for free. We prove that, no matter how small $\epsilon$ is the agent…
In practical applications, data is used to make decisions in two steps: estimation and optimization. First, a machine learning model estimates parameters for a structural model relating decisions to outcomes. Second, a decision is chosen to…
We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an…
A relation extends another relation consistently if its symmetric, respectively its asymmetric, part contains the corresponding part of the smaller relation. It is shown that there exists no finite circular chain made from two transitive…
We consider undiscounted reinforcement learning in Markov decision processes (MDPs) where both the reward functions and the state-transition probabilities may vary (gradually or abruptly) over time. For this problem setting, we propose an…
Actual individual preferences are neither complete (=total) nor antisymmetric in general, so that at least every quasi-order must be an admissible input to a satisfactory choice rule. It is argued that the traditional notion of…
It is well-known that Choice and Regularity are independent of each other but have important common consequences of logical character (reflection principles, representations of classes by sets, etc.). We explain this phenomenon by isolating…
The problem of online prediction with sequential side information under logarithmic loss is studied, and general upper and lower bounds on the minimax regret incurred by the predictor is established. The upper bounds on the minimax regret…
We consider contextual bandit learning under distribution shift when reward vectors are ordered according to a given preference cone. We propose an adaptive-discretization and optimistic elimination based policy that self-tunes to the…
We study existence, uniqueness and computability of solutions for a class of discrete time recursive utilities models. By combining two streams of the recent literature on recursive preferences---one that analyzes principal eigenvalues of…
Many prediction tasks can admit multiple models that can perform almost equally well. This phenomenon can can undermine interpretability and safety when competing models assign conflicting predictions to individuals. In this work, we study…
We extend well-known comparative results under expected utility to models of non-expected utility by providing novel conditions on local utility functions. We illustrate how our results parallel, and are distinct from, existing results for…
For decision making under uncertainty, min-max regret has been established as a popular methodology to find robust solutions. In this approach, we compare the performance of our solution against the best possible performance had we known…
We prove that a classic sub-Gaussian mixture proposed by Robbins in a stochastic setting actually satisfies a path-wise (deterministic) regret bound. For every path in a natural ``Ville event'' $\mathcal E_\alpha$, this regret till time $T$…
We present an online learning analysis of minimax adaptive control for the case where the uncertainty includes a finite set of linear dynamical systems. Precisely, for each system inside the uncertainty set, we define the model-based regret…
Multi-objective optimization studies the process of seeking multiple competing desiderata in some operation. Solution techniques highlight marginal tradeoffs associated with weighing one objective over others. In this paper, we consider…
Information-theoretic Bayesian regret bounds of Russo and Van Roy capture the dependence of regret on prior uncertainty. However, this dependence is through entropy, which can become arbitrarily large as the number of actions increases. We…
We revisit lower bounds on the regret in the case of multi-armed bandit problems. We obtain non-asymptotic, distribution-dependent bounds and provide straightforward proofs based only on well-known properties of Kullback-Leibler…
We investigate contextual bandits in the presence of side-observations across arms in order to design recommendation algorithms for users connected via social networks. Users in social networks respond to their friends' activity, and hence…