We evaluate how visualizations can influence the judgment of MLLMs about the presence or absence of bridges in a network. We show that the inclusion of visualization improves confidence over a structured text-based input that could theoretically be helpful for answering the question. On the other hand, we observe that standard visualization techniques create a strong bias towards accepting or refuting the presence of a bridge -- independently of whether or not a bridge actually exists in the network. While our results indicate that the inclusion of visualization techniques can effectively influence the MLLM's judgment without compromising its self-reported confidence, they also imply that practitioners must be careful of allowing users to include visualizations in generative AI applications so as to avoid undesired hallucinations.
@article{arxiv.2511.03617,
title = {Visualization Biases MLLM's Decision Making in Network Data Tasks},
author = {Timo Brand and Henry Förster and Stephen G. Kobourov and Jacob Miller},
journal= {arXiv preprint arXiv:2511.03617},
year = {2025}
}
Comments
This manuscript was presented at VIS x GenAI, a workshop co-located with IEEE VIS 2025