Supporting Reflection and Forward-Looking Reasoning With Data-Driven Questions
Abstract
Many generative AI systems as well as decision-support systems (DSSs) provide operators with predictions or recommendations. Various studies show, however, that people can mistakenly adopt the erroneous results presented by those systems. Hence, it is crucial to promote critical thinking and reflection during interaction. One approach we are focusing on involves encouraging reflection during machine-assisted decision-making by presenting decision-makers with data-driven questions. In this short paper, we provide a brief overview of our work in that regard, namely: 1) the development of a question taxonomy, 2) the development of a prototype in the medical domain and the feedback received from clinicians, 3) a method for generating questions using a large language model, and 4) a proposed scale for measuring cognitive engagement in human-AI decision-making. In doing so, we contribute to the discussion about the design, development, and evaluation of tools for thought, i.e., AI systems that provoke critical thinking and enable novel ways of sense-making.
Cite
@article{arxiv.2603.27318,
title = {Supporting Reflection and Forward-Looking Reasoning With Data-Driven Questions},
author = {Simon WS Fischer and Hanna Schraffenberger and Serge Thill and Pim Haselager},
journal= {arXiv preprint arXiv:2603.27318},
year = {2026}
}
Comments
Accepted at CHI2026 Workshop on Tools for Thought, April 16, 2026, in Barcelona, Spain