Related papers: Why Should I Trust You, Bellman? The Bellman Error…

Off-policy Reinforcement Learning (RL) holds the promise of better data efficiency as it allows sample reuse and potentially enables safe interaction with the environment. Current off-policy policy gradient methods either suffer from high…

We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence…

While Bellman equations for basic reach, avoid, and reach-avoid problems are well studied, the relationship between value optimality and policy optimality becomes subtle in the undiscounted infinite-horizon setting, particularly for more…

This paper studies value iteration for infinite horizon contracting Markov decision processes under convexity assumptions and when the state space is uncountable. The original value iteration is replaced with a more tractable form and the…

This paper revisits the recently proposed reward centering algorithms including simple reward centering (SRC) and value-based reward centering (VRC), and points out that SRC is indeed the reward centering, while VRC is essentially Bellman…

The experimentally verified violation of Bell's inequalities apparently implies that at least one of two intuitive beliefs must be false: that effects propagating at infinite velocity do not exist, and that natural phenomena occur…

We remind the viewpoint that violation of Bell's inequality might be interpreted not only as an evidence of the alternative -- either nonlocality or ``death of reality'' (under the assumption the quantum mechanics is incomplete). Violation…

Cyber-physical systems are found in many applications such as power networks, manufacturing processes, and air and ground transportation systems. Maintaining security of these systems under cyber attacks is an important and challenging…

Bell's Theorem was developed on the basis of considerations involving a linear combination of spin correlation functions, each of which has a distinct pair of arguments. The simultaneous presence of these different pairs of arguments in the…

We consider the problem of reinforcement learning using function approximation, where the approximating basis can change dynamically while interacting with the environment. A motivation for such an approach is maximizing the value function…

We study risk-sensitive reinforcement learning (RL) based on the entropic risk measure. Although existing works have established non-asymptotic regret guarantees for this problem, they leave open an exponential gap between the upper and…

Bell's theorem admits several interpretations or 'solutions', the standard interpretation being 'indeterminism', a next one 'nonlocality'. In this article two further solutions are investigated, termed here 'superdeterminism' and…

In this short note we derive a relationship between the Bregman divergence from the current policy to the optimal policy and the suboptimality of the current value function in a regularized Markov decision process. This result has…

This paper considers batch Reinforcement Learning (RL) with general value function approximation. Our study investigates the minimal assumptions to reliably estimate/minimize Bellman error, and characterizes the generalization performance…

Machine learning (ML) models show strong promise for new biomedical prediction tasks, but concerns about trustworthiness have hindered their clinical adoption. In particular, it is often unclear whether a model relies on true clinical cues…

In this paper we argue for the fundamental importance of the value distribution: the distribution of the random return received by a reinforcement learning agent. This is in contrast to the common approach to reinforcement learning which…

Offline policy evaluation (OPE) is considered a fundamental and challenging problem in reinforcement learning (RL). This paper focuses on the value estimation of a target policy based on pre-collected data generated from a possibly…

The weak value approximation has been in use for thirty-five years, but it has not as of yet received a truly complete derivation, leaving its mathematical validity in a state of limbo. Herein, I fill this gap, deriving the weak value…

Offline reinforcement learning (RL) promises the ability to learn effective policies solely using existing, static datasets, without any costly online interaction. To do so, offline RL methods must handle distributional shift between the…

Bell's theorem supposedly demonstrates an irreconcilable conflict between quantum mechanics and local, realistic hidden variable theories. In this paper we show that all experiments that aim to prove Bell's theorem do not actually achieve…