Related papers: Improving Students With Rubric-Based Self-Assessme…
Background: Proof Blocks is a software tool that enables students to construct proofs by assembling prewritten lines and gives them automated feedback. Prior work on learning gains from Proof Blocks has focused on comparing learning gains…
User feedback is becoming an increasingly important source of information for requirements engineering, user interface design, and software engineering in general. Nowadays, user feedback is largely available and easily accessible in social…
As systematic inequities in higher education and society have been brought to the forefront, graduate programs are interested in increasing the diversity of their applicants and enrollees. Yet, structures in place to evaluate applicants may…
Large Language Models (LLMs) have become indispensable for evaluating writing. However, text feedback they provide is often unintelligible, generic, and not specific to user criteria. Inspired by structured rubrics in education and…
This paper describes results from a project in an undergraduate engineering physics course that coupled classroom use of interactive computer simulations with the collection of real-time formative assessment using pen-enabled mobile…
Feedback is essential for learning, but its effectiveness relies heavily on how well it engages students in the educational process. Generative AI offers novel opportunities to efficiently produce rich, formative feedback, ranging from…
In this paper, we analyse how learning is measured and optimized in Educational Recommender Systems (ERS). In particular, we examine the target metrics and evaluation methods used in the existing ERS research, with a particular focus on the…
This experiment research study examines how traditional assessment methods such as written tests and presentations compared to the new online tests in higher education We want to know how the use of the Internet for assessment affects how…
The quality of assessment determines the quality of learning, and is characterized by validity, reliability and difficulty. Mastery of learning is generally represented by the difficulty levels of assessment items. A very large number of…
Human feedback has become the de facto standard for evaluating the performance of Large Language Models, and is increasingly being used as a training objective. However, it is not clear which properties of a generated output this single…
In computer science, students could benefit from more opportunities to learn important, high-level concepts and to improve their learning skills. Peer review is one method to encourage this by providing students with the opportunity to…
Refactoring is the process of improving the design of existing code by changing its internal structure without affecting its external behaviour, with the main aims of improving the quality of software product. Therefore, there is a belief…
Research suggests "write-to-learn" tasks improve learning outcomes, yet constructed-response methods of formative assessment become unwieldy with large class sizes. This study evaluates natural language processing algorithms to assist this…
Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences. A common framework is to iteratively query the user about which of two presented robot trajectories they prefer. While…
Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and generating rubric-based reward signals substantially improves…
Techniques for reliable rubric-based LLM evaluation -- ensemble judging, bias mitigation, few-shot calibration -- are scattered across papers with inconsistent terminology and partial implementations. We introduce Autorubric, an open-source…
Expert problem solvers are characterized by continuous evaluation of their progress towards a solution. One characteristic of expertise is self-diagnosis directed towards elaboration of the solvers' conceptual understanding, knowledge…
Automated writing evaluation systems can improve students' writing insofar as students attend to the feedback provided and revise their essay drafts in ways aligned with such feedback. Existing research on revision of argumentative writing…
Despite growing interest in using Large Language Models (LLMs) for educational assessment, it remains unclear how closely they align with human scoring. We present a systematic evaluation of instruction-tuned LLMs across three open…
Data-driven programming feedback systems can help novices to program in the absence of a human tutor. Prior evaluations showed that these systems improve learning in terms of test scores, or task completion efficiency. However, crucial…