Related papers: Analyzing Force Concept Inventory with Item Respon…
Item Response Theory (IRT) is a powerful statistical approach for evaluating test items and determining test taker abilities through response analysis. An IRT model that better fits the data leads to more accurate latent trait estimates. In…
This study investigates differences in student responses to in-class and online administrations of the Force Concept Inventory (FCI), Conceptual Survey of Electricity and Magnetism (CSEM), and the Colorado Learning Attitudes about Science…
Analyses of heterogeneous treatment effects (HTE) are common in applied causal inference research. However, when outcomes are latent variables assessed via psychometric instruments such as educational tests, standard methods ignore the…
Humans can progressively learn visual concepts from easy to hard questions. To mimic this efficient learning ability, we propose a competence-aware curriculum for visual concept learning in a question-answering manner. Specifically, we…
The increased use of multiple-choice (MC) questions in introductory-level physics final exams is largely hindered by reservations about its ability to test the broad cognitive domain that is routinely accessed with typical…
Item Response Theory (IRT) models have received growing interest in health science for analyzing latent constructs such as depression, anxiety, quality of life, or cognitive functioning from the information provided by each individual's…
Item (question) difficulties play a crucial role in educational assessments, enabling accurate and efficient assessment of student abilities and personalization to maximize learning outcomes. Traditionally, estimating item difficulties can…
Item response theory (IRT) models explain an observed item response as a function of a respondent's latent trait and the item's property. IRT is one of the most widely utilized tools for item response analysis; however, local item and…
The force concept inventory (FCI) is one of the research-based assessments (RBAs) established by the physics education research (PER) community to measure students' understanding of Newtonian mechanics. Former works have often recorded the…
We propose a class of Item Response Theory models for items with ordinal polytomous responses, which extends an existing class of multidimensional models for dichotomously-scored items measuring more than one latent trait. In the proposed…
Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale. Existing evaluation protocols rely on average accuracy over fixed item sets,…
Many statisticians regularly teach large lecture courses on statistics, probability, or mathematics for students from other fields such as business and economics, social sciences and psychology, etc. The corresponding exams often use a…
We present a study that translates the Force Concept Inventory (FCI) using OpenAI GPT-4o and assess the specific difficulties of translating a scientific-focused topic using Large Language Models (LLMs). The FCI is a physics exam meant to…
This is the first in a series of papers that analyze college student beliefs in realms where common astronomy misconceptions are prevalent. Data was collected through administration of an inventory distributed at the end of an introductory…
Educational assessments are valuable tools for measuring student knowledge and skills, but their validity can be compromised when test takers exhibit changes in response behavior due to factors such as time pressure. To address this issue,…
Item response theory aims to estimate respondent's latent skills from their responses in tests composed of items with different levels of difficulty. Several models of item response theory have been proposed for different types of tasks,…
Item Response Theory (IRT) is widely applied in the human sciences to model persons' responses on a set of items measuring one or more latent constructs. While several R packages have been developed that implement IRT models, they tend to…
Large language models (LLMs) achieve high performance on mathematical reasoning, but these results can be inflated by training data leakage or superficial pattern matching rather than genuine reasoning. To this end, an adversarial…
The aim of this study is to investigate the effectiveness of ChatGPT 3.5 in developing algorithms for data generation within the framework of Item Response Theory (IRT) using the R programming language. In this context, validity…
It is widely believed that a joint factor analysis of item responses and response time (RT) may yield more precise ability scores that are conventionally predicted from responses only. For this purpose, a simple-structure factor model is…