相关论文: Predictive validities: figures of merit or veils o…

In this work, I point out the negative implications for diversity in graduate school resulting from the use of cutoff scores on the GRE in the admissions process. In light of the data presented, as well as a swelling body of evidence…

A recent paper in Sci. Adv. by Miller et al. concludes that GREs do not help predict whether physics grad students will get Ph.D.s. The paper makes numerous elementary statistics errors, including introduction of unnecessary collider-like…

Recent and influential critiques of standardized testing have noted the existence of non-trivial numbers of successful scientists who received low scores on the GRE. Here we use computer simulations to show that the prevalence of such…

Generalization error predictors (GEPs) aim to predict model performance on unseen distributions by deriving dataset-level error estimates from sample-level scores. However, GEPs often utilize disparate mechanisms (e.g., regressors,…

The meteoric rise in text generation capability has been accompanied by parallel growth in interest in machine-generated text detection: the capability to identify whether a given text was generated using a model or written by a person.…

U.S. state education agencies mark schools displaying achievement gaps between demographic subgroups as needing improvement. Some schools may have few students in these subgroups, such that average end-of-year test scores only noisily…

Social contexts -- such as families, schools, and neighborhoods -- shape life outcomes. The key question is not simply whether they matter, but rather for whom and under what conditions. Here, we argue that prediction gaps -- differences in…

This paper introduces a novel revealed-preference approach to ranking colleges and professional schools based on applicants' choices and standardized test scores. Unlike traditional rankings that rely on data supplied by institutions or…

A recent paper evaluating a new rubric-based graduate admissions approach using generic methods tentatively suggested that its decisions differed noticeably from the previous approach in an unspecified way. Using prior knowledge that the…

An analysis of 1,955 physics graduate students from 19 PhD programs shows that undergraduate grade point average predicts graduate grades and PhD completion more effectively than GRE scores. Students' undergraduate GPA (UGPA) and GRE…

Predictive modeling uncovers knowledge and insights regarding a hypothesized data generating mechanism (DGM). Results from different studies on a complex DGM, derived from different data sets, and using complicated models and algorithms,…

Pretraining-based (PT-based) automatic evaluation metrics (e.g., BERTScore and BARTScore) have been widely used in several sentence generation tasks (e.g., machine translation and text summarization) due to their better correlation with…

Every year, over one million EU students choose a secondary school track based on teacher recommendations, yet little evidence shows this yields optimal assignments. Using Dutch data, we examine whether access to standardized test scores…

Context: Software engineering has a problem in that when we empirically evaluate competing prediction systems we obtain conflicting results. Objective: To reduce the inconsistency amongst validation study results and provide a more formal…

When the same LLM generates assessment items, simulates student responses, and scores them, the validation loop is self-referential. We introduce Generative-Evaluative Agreement (GEA), a validity criterion measuring whether an LLM's scoring…

Machine learning is about forecasting. When the forecasts come with an evaluation metric the forecasts become useful. What are reasonable evaluation metrics? How do existing evaluation metrics relate? In this work, we provide a general…

Automatic Essay Scoring (AES) is a well-established educational pursuit that employs machine learning to evaluate student-authored essays. While much effort has been made in this area, current research primarily focuses on either (i)…

Predicting students' academic performance is one of the key tasks of educational data mining (EDM). Traditionally, the high forecasting quality of such models was deemed critical. More recently, the issues of fairness and discrimination…

Colleges and universities use predictive analytics in a variety of ways to increase student success rates. Despite the potential for predictive analytics, two major barriers exist to their adoption in higher education: (a) the lack of…

Equity of educational outcome and fairness of AI with respect to race have been topics of increasing importance in education. In this work, we address both with empirical evaluations of grade prediction in higher education, an important…