Detecting Problem Statements in Peer Assessments
Abstract
Effective peer assessment requires students to be attentive to the deficiencies in the work they rate. Thus, their reviews should identify problems. But what ways are there to check that they do? We attempt to automate the process of deciding whether a review comment detects a problem. We use over 18,000 review comments that were labeled by the reviewees as either detecting or not detecting a problem with the work. We deploy several traditional machine-learning models, as well as neural-network models using GloVe and BERT embeddings. We find that the best performer is the Hierarchical Attention Network classifier, followed by the Bidirectional Gated Recurrent Units (GRU) Attention and Capsule model with scores of 93.1% and 90.5% respectively. The best non-neural network model was the support vector machine with a score of 89.71%. This is followed by the Stochastic Gradient Descent model and the Logistic Regression model with 89.70% and 88.98%.
Keywords
Cite
@article{arxiv.2006.04532,
title = {Detecting Problem Statements in Peer Assessments},
author = {Yunkai Xiao and Gabriel Zingle and Qinjin Jia and Harsh R. Shah and Yi Zhang and Tianyi Li and Mohsin Karovaliya and Weixiang Zhao and Yang Song and Jie Ji and Ashwin Balasubramaniam and Harshit Patel and Priyankha Bhalasubbramanian and Vikram Patel and Edward F. Gehringer},
journal= {arXiv preprint arXiv:2006.04532},
year = {2020}
}
Comments
8 pages, 9 images. Extended version of a paper published at EDM 2020, 13th International Conference on Educational Data Mining