Related papers: Capturing Misalignment
Artificial intelligence safety research focuses on aligning individual language models with human values, yet deployed AI systems increasingly operate as interacting populations where social influence may override individual alignment. Here…
We propose a formalism to model and reason about reconfigurable multi-agent systems. In our formalism, agents interact and communicate in different modes so that they can pursue joint tasks; agents may dynamically synchronize, exchange…
When should we delegate decisions to AI systems? While the value alignment literature has developed techniques for shaping AI values, less attention has been paid to how to determine, under uncertainty, when imperfect alignment is good…
This chapter develops a unified framework for studying misspecified learning situations in which agents optimize and update beliefs within an incorrect model of their environment. We review the statistical foundations of learning from…
Model-based multi-agent control requires agents to possess a model of the behavior of others to make strategic decisions. Solution concepts from game theory are often used to model the emergent collective behavior of self-interested agents…
Differences in perception, information asymmetries, and bounded rationality lead game-theoretic players to derive a private, subjective view of the game that may diverge from the underlying ground-truth scenario and may be misaligned with…
In action domains where agents may have erroneous beliefs, reasoning about the effects of actions involves reasoning about belief change. In this paper, we use a transition system approach to reason about the evolution of an agents beliefs…
Alignment is a social phenomenon wherein individuals share a common goal or perspective. Mirroring, or mimicking the behaviors and opinions of another individual, is one mechanism by which individuals can become aligned. Large scale…
The main approach to evaluating communication is by assessing how well it facilitates coordination. If two or more individuals can coordinate through communication, it is generally assumed that they understand one another. We investigate…
Traditional economic models typically treat private information, or signals, as generated from some underlying state. Recent work has explicated alternative models, where signals correspond to interpretations of available information. We…
Agent-based models are versatile tools for studying how societal opinion change, including political polarization and cultural diffusion, emerges from individual behavior. This study expands agents' psychological realism using…
We consider communication when there is no agreement about symbols and meanings. We treat it within the framework of reinforcement learning. We apply different reinforcement learning models in our studies and simplify the problem as much as…
We extend the indirect evolutionary approach to the selection of (possibly misspecified) models. Agents with different models match in pairs to play a stage game, where models define feasible beliefs about game parameters and about others'…
Agent-based modeling is a powerful simulation technique to understand the collective behavior and microscopic interaction in complex financial systems. Recently, the concept for determining the key parameters of the agent-based models from…
This paper presents a simple agent-based model of an economic system, populated by agents playing different games according to their different view about social cohesion and tax payment. After a first set of simulations, correctly…
Understanding the evolution of human social systems requires flexible formalisms for the emergence of institutions. Although game theory is normally used to model interactions individually, larger spaces of games can be helpful for modeling…
Beliefs are not facts, but they are factive - they feel like facts. This property is what can make misinformation dangerous. Being able to deliberately navigate through a landscape of often conflicting factive statements is difficult when…
The problem of achieving common understanding between agents that use different vocabularies has been mainly addressed by designing techniques that explicitly negotiate mappings between their vocabularies, requiring agents to share a…
We examine settings in which agents choose behaviors and care about their neighbors' behaviors, but have incomplete information about the network in which they are embedded. We develop a model in which agents use local knowledge of their…
As AI models become ever more complex and intertwined in humans' daily lives, greater levels of interactivity of explainable AI (XAI) methods are needed. In this paper, we propose the use of belief change theory as a formal foundation for…