Related papers: Rubric Design for Separating the Roles of Open-End…
Expert problem solvers are characterized by continuous evaluation of their progress towards a solution. One characteristic of expertise is self-diagnosis directed towards elaboration of the solvers' conceptual understanding, knowledge…
Teaching assistants (TAs) are often responsible for grading in introductory physics courses at large research universities. Their grading practices can shape students' approaches to problem solving and learning. Physics education research…
This study attempted to develop fair, relevant, and content-valid assessment tools for capstone project courses. Toward this goal, new rating instruments based on the concept of rubrics were proposed. To ensure that the new instruments were…
Rubrics and oral feedback are approaches to help students improve performance and meet learning outcomes. However, their effect on the actual improvement achieved is inconclusive. This paper evaluates the effect of rubrics and oral feedback…
Standardized assessment tests that allow researchers to compare the performance of students under various curricula are highly desirable. There are several research-based conceptual tests that serve as instruments to assess and identify…
As systematic inequities in higher education and society have been brought to the forefront, graduate programs are interested in increasing the diversity of their applicants and enrollees. Yet, structures in place to evaluate applicants may…
Student responses in STEM assessments are often handwritten and combine symbolic expressions, calculations, and diagrams, creating substantial variation in format and interpretation. Despite their importance for evaluating students'…
A recent paper evaluating a new rubric-based graduate admissions approach using generic methods tentatively suggested that its decisions differed noticeably from the previous approach in an unspecified way. Using prior knowledge that the…
Explication and reflection on expert vs. novice considerations within the problem-solving process characterize a cognitive apprenticeship approach for the development of expert-like problem solving practices. In the context of grading, a…
Rubric-based evaluation has become a prevailing paradigm for evaluating instruction following in large language models (LLMs). Despite its widespread use, the reliability of these rubric-level evaluations remains unclear, calling for…
Many instructors choose to assess their students using open-ended written exam items that require students to show their understanding of physics by solving a problem and/or explaining a concept. Grading these items is fairly time…
Rubrics have been extensively utilized for evaluating unverifiable, open-ended tasks, with recent research incorporating them into reward systems for reinforcement learning. However, existing frameworks typically treat rubrics only as…
Rubric-based admissions are claimed to help make the graduate admissions process more equitable, possibly helping to address the historical and ongoing inequities in the U.S. physics graduate school admissions process that have often…
Open-ended grading is central to equitable and personalized education, yet manual grading remains time-consuming and costly, underscoring the need for automated grading systems. Although recent neural and large language model (LLM) based…
The research presented in this thesis was motivated by the need to improve introductory physics courses. Introductory physics courses are generally the first courses in which students learn to create models to solve complex problems.…
Traditional high-stakes summative assessments--timed, in-class exams accounting for a large percentage of the term's overall grade--have often received criticism from the educational community. Such assessments tend to prize a particular…
Rubrics are being used in a wide variety of disciplines in higher education to evaluate assessments and provide feedback to students. Rubrics are traditionally implemented as paper-based table format to grade assessments and provide…
Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level…
We discuss first experiences with a new variant of self-assessment in higher mathematics education. In our setting, the students of the course have to mark a part of their homework assignments themselves and they receive the corresponding…
Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and generating rubric-based reward signals substantially improves…