Related papers: Validation of a Conceptual Assessment Tool in E&M …
We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e.g. Flamingo, SeViLA, or GPT-4). Compared to existing benchmarks that focus on…
Chain-of-Thought (CoT) prompting has marked a significant advancement in enhancing the reasoning capabilities of large language models (LLMs). Previous studies have developed various extensions of CoT, which focus primarily on enhancing…
Prompt design is a primary control interface for large language models (LLMs), yet standard evaluations largely reduce performance to answer correctness, obscuring why a prompt succeeds or fails and providing little actionable guidance. We…
We have developed the Physics Inventory of Quantitative Literacy (PIQL) as a tool to measure students' quantitative literacy in the context of introductory physics topics. We present the results from various quantitative analyses used to…
As artificial intelligence systems become increasingly prevalent in education, a fundamental challenge emerges: how can we verify if an AI truly understands how students think and reason? Traditional evaluation methods like measuring…
Recent studies demonstrate that multimodal large language models (MLLMs) can proficiently evaluate visual quality through interpretable assessments. However, existing approaches typically treat quality scoring and reasoning descriptions as…
Existing text-driven infrared and visible image fusion approaches often rely on textual information at the sentence level, which can lead to semantic noise from redundant text and fail to fully exploit the deeper semantic value of textual…
Humans can learn and reason under substantial uncertainty in a space of infinitely many concepts, including structured relational concepts ("a scene with objects that have the same color") and ad-hoc categories defined through goals…
Project-Based Learning (PBL) involves a variety of highly correlated multimodal data, making it a vital educational approach within STEM disciplines. With the rapid development of multimodal large language models (MLLMs), researchers have…
Calibration of mean estimates for predictions is a crucial property in many applications, particularly in the fields of financial and actuarial decision-making. In this paper, we first review classical approaches for validating…
Research-validated clicker questions as instructional tools for formative assessment are relatively easy to implement and can provide effective scaffolding when developed and implemented in a sequence. We present findings from the…
Science is an inherently quantitative endeavor, and general education science courses are taken by a majority of college students. As such, they are a powerful venue for advancing students' skills and attitudes toward mathematics. This…
Concept Bottleneck Models (CBMs) decompose image classification into a process governed by interpretable, human-readable concepts. Recent advances in CBMs have used Large Language Models (LLMs) to generate candidate concepts. However, a…
The deployment of Large Language Models (LLMs) in high-stakes clinical settings demands rigorous and reliable evaluation. However, existing medical benchmarks remain static, suffering from two critical limitations: (1) data contamination,…
We present a study of active learning pedagogies in an upper division physics course. This work was guided by the principle of deliberate practice for the development of expertise, and this principle was used in the design of the materials…
The aim of this study is to investigate the decisions and reasoning of undergraduate students when choosing simple measurement instruments in an introductory physics laboratory course. For this study, we have developed a questionnaire and…
A common learning goal for modern physics instructors is for students to recognize a difference between the experimental uncertainty of classical physics and the fundamental uncertainty of quantum mechanics. Our prior work has shown that…
Item Response Theory (IRT) is a popular assessment method used in education measurement, which builds on an assumption of a probability framework connecting students' innate ability and their actual performances on test items. The model…
Computer-aided diagnosis systems must make critical decisions from medical images that are often noisy, ambiguous, or conflicting, yet today's models are trained on overly simplistic labels that ignore diagnostic uncertainty. One-hot labels…
The assessment of evaluation metrics (meta-evaluation) is crucial for determining the suitability of existing metrics in text-to-image (T2I) generation tasks. Human-based meta-evaluation is costly and time-intensive, and automated…