Related papers: An Application of HodgeRank to Online Peer Assessm…
We propose the PeerRank method for peer assessment. This constructs a grade for an agent based on the grades proposed by the agents evaluating the agent. Since the grade of an agent is a measure of their ability to grade correctly, the…
Peer assessment has established itself as a critical pedagogical tool in academic settings, offering students timely, high-quality feedback to enhance learning outcomes. However, the efficacy of this approach depends on two factors: (1) the…
The prevalence of e-learning systems and on-line courses has made educational material widely accessible to students of varying abilities and backgrounds. There is thus a growing need to accommodate for individual differences in e-learning…
Iterative peer grading activities may keep students engaged during in-class project presentations. Effective methods for collecting and aggregating peer assessment data are essential. Students tend to grade projects favorably. So, while…
Peer assessment is an efficient and effective learning assessment method that has been used widely in diverse fields in higher education. Despite its many benefits, a fundamental problem in peer assessment is that participants lack the…
Peer review is the primary means of quality control in academia; as an outcome of a peer review process, program and area chairs make acceptance decisions for each paper based on the review reports and scores they received. Quality of…
Evaluating large language models typically relies on human-authored benchmarks, reference answers, and human or single-model judgments, approaches that scale poorly, become quickly outdated, and mismatch open-world deployments that depend…
HodgeRank generalizes ranking algorithms, e.g. Google PageRank, to rank alternatives based on real-world (often incomplete) data using graphs and discrete exterior calculus. It analyzes multipartite interactions on high-dimensional networks…
Peer Assessment is a task of analysis and commenting on student's writing by peers, is core of all educational components both in campus and in MOOC's. However, with the sheer scale of MOOC's & its inherent personalised open ended learning,…
Massive Open Online Courses (MOOCs) use peer assessment to grade open ended questions at scale, allowing students to provide feedback. Relative to teacher based grading, peer assessment on MOOCs traditionally delivers lower quality feedback…
One of the most widespread multi-criteria decision-making methods is the Analytic Hierarchy Process (AHP). AHP successfully combines the pairwise comparisons method and the hierarchical approach. It allows the decision-maker to set…
Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible. Most current systems either take the mean or median of reported grades; others aim to estimate students' grading…
We investigate the potential of using ordinal peer grading for the evaluation of students in massive online open courses (MOOCs). According to such grading schemes, each student receives a few assignments (by other students) which she has…
Online learning to rank is a sequential decision-making problem where in each round the learning agent chooses a list of items and receives feedback in the form of clicks from the user. Many sample-efficient algorithms have been proposed…
Multi-criteria recommender systems can improve the quality of recommendations by considering user preferences on multiple criteria. One promising approach proposed recently is multi-criteria ranking, which uses Pareto ranking to assign a…
Heuristic Rating Estimation (HRE) is a newly proposed method supporting decisions analysis based on the use of pairwise comparisons. It allows that the ranking values of some alternatives (herein referred to as concepts) are initially…
We examine three methods for ranking by pairwise comparison: Principal Eigenvector, HodgeRank and Tropical Eigenvector. It is shown that the choice of method can produce arbitrarily different rank order.To be precise, for any two of the…
Pairwise comparisons are a well-known method for modelling of the subjective preferences of a decision maker. A popular implementation of the method is based on solving an eigenvalue problem for M - the matrix of pairwise comparisons. This…
Peer grading is an educational system in which students assess each other's work. It is commonly applied under Massive Open Online Course (MOOC) and offline classroom settings. With this system, instructors receive a reduced grading…
In massive open online courses (MOOCs), peer grading serves as a critical tool for scaling the grading of complex, open-ended assignments to courses with tens or hundreds of thousands of students. But despite promising initial trials, it…