Related papers: Capturing Misalignment
We present an extensive study of the joint effects of heterogeneous social agents and their heterogeneous social links in a bounded confidence opinion dynamics model. The full phase diagram of the model is explored for two different…
Modeling social interactions based on individual behavior has always been an area of interest, but prior literature generally presumes rational behavior. Thus, such models may miss out on capturing the effects of biases humans are…
Modelling the behaviours of other agents is essential for understanding how agents interact and making effective decisions. Existing methods for agent modelling commonly assume knowledge of the local observations and chosen actions of the…
Recent advances in AI research make it increasingly plausible that artificial agents with consequential real-world impact will soon operate beyond tightly controlled environments. Ensuring that these agents are not only safe but that they…
This paper develops a dynamic equilibrium model where agents exhibit a strong form of belief heterogeneity: they disagree about zero probability events. It is shown that, somewhat surprisingly, equilibrium exists in this setting, and that…
The value alignment problem for artificial intelligence (AI) is often framed as a purely technical or normative challenge, sometimes focused on hypothetical future systems. I argue that the problem is better understood as a structural…
Social acceptability is an important consideration for HCI designers who develop technologies for social contexts. However, the current theoretical foundations of social acceptability research do not account for the complex interactions…
Value alignment is essential for building AI systems that can safely and reliably interact with people. However, what a person values -- and is even capable of valuing -- depends on the concepts that they are currently using to understand…
Multi-agent scenarios, like Wigner's friend and Frauchiger-Renner scenarios, can show contradictory results when a non-classical formalism must deal with the knowledge between agents. Such paradoxes are described with multi-modal logic as…
Intelligent agents such as robots are increasingly deployed in real-world, safety-critical settings. It is vital that these agents are able to explain the reasoning behind their decisions to human counterparts, however, their behavior is…
Auctions in which agents' payoffs are random variables have received increased attention in recent years. In particular, recent work in algorithmic mechanism design has produced mechanisms employing internal randomization, partly in…
Imitation learning, which learns agent policy by mimicking expert demonstration, has shown promising results in many applications such as medical treatment regimes and self-driving vehicles. However, it remains a difficult task to interpret…
Despite investments in improving model safety, studies show that misaligned capabilities remain latent in safety-tuned models. In this work, we shed light on the mechanics of this phenomenon. First, we show that even when model generations…
In various economic environments, people observe other people with whom they strategically interact. We can model such information-sharing relations as an information network, and the strategic interactions as a game on the network. When…
Human behavior in interactive settings is shaped not only by individual objectives but also by shared constraints with others, such as safety. Understanding how people allocate responsibility, i.e., how much one deviates from their desired…
We introduce a novel setting, wherein an agent needs to learn a task from a demonstration of a related task with the difference between the tasks communicated in natural language. The proposed setting allows reusing demonstrations from…
We examine the tuning of cooperative behavior in repeated multi-agent games using an analytically tractable, continuous-time, nonlinear model of opinion dynamics. Each modeled agent updates its real-valued opinion about each available…
In dynamic settings each economic agent's choices can be revealing of her private information. This elicitation via the rationalization of observable behavior depends each agent's perception of which payoff-relevant contingencies other…
The mental models that humans form of other agents---encapsulating human beliefs about agent goals, intentions, capabilities, and more---create an underlying basis for interaction. These mental models have the potential to affect both the…
We show that in delegation problems, a principal benefits from belief misalignment vis-\`a-vis an agent when the latter can flexibly acquire costly information. The agent optimally succumbs to confirmatory learning, leading him to favor the…