Related papers: Analyzing the Effects of Two-Stage Peer Evaluation
Decision-makers in high-stakes selection processes often face a fundamental choice: whether to make decisions themselves or to delegate authority to another entity whose incentives may only be partially aligned with their own. Such…
Two-stage recommender systems are widely adopted in industry due to their scalability and maintainability. These systems produce recommendations in two steps: (i) multiple nominators preselect a small number of items from a large pool using…
Peer grading is an educational system in which students assess each other's work. It is commonly applied under Massive Open Online Course (MOOC) and offline classroom settings. With this system, instructors receive a reduced grading…
Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible. Most current systems either take the mean or median of reported grades; others aim to estimate students' grading…
Consider an actor making selection decisions using a series of classifiers, which we term a sequential screening process. The early stages filter out some applicants, and in the final stage an expensive but accurate test is applied to the…
When making a decision as a group, there are two primary paradigms: aggregating preferences (e.g. voting, mechanism design) and aggregating information (e.g. discussion, consulting, forecasting). Almost all formally-studied group…
One of the basic tasks which is responded for head of each university department, is employing lecturers based on some default factors such as experience, evidences, qualifies and etc. In this respect, to help the heads, some automatic…
Effective group decision-making is critical in Multi-Agent Systems (MAS). Yet, how different mechanisms for reaching consensus impact collaboration quality and efficiency remains understudied. We conduct a systematic study on group…
We consider an agent community wishing to decide on several binary issues by means of issue-by-issue majority voting. For each issue and each agent, one of the two options is better than the other. However, some of the agents may be…
Comparing with lecturer marked assessments, peer assessment is a more comprehensive learning process and many of the associated problems have occurred. In this research work, we study the peer-assessment impact on group learning activities…
We consider five different peer-to-peer file sharing systems with two chunks, with the aim of finding chunk selection algorithms that have provably stable performance with any input rate and assuming non-altruistic peers who leave the…
Epidemiologic and medical studies often rely on evaluators to obtain measurements of exposures or outcomes for study participants, and valid estimates of associations depends on the quality of data. Even though statistical methods have been…
Peer prediction mechanisms incentivize agents to truthfully report their signals even in the absence of verification by comparing agents' reports with those of their peers. In the detail-free multi-task setting, agents respond to multiple…
A recommender system learns to predict the user-specific preference or intention over many items simultaneously for all users, making personalized recommendations based on a relatively small number of observations. One central issue is how…
Multi-stage stochastic programming is a well-established framework for sequential decision making under uncertainty by seeking policies that are fully adapted to the uncertainty. Often such flexible policies are not desirable, and the…
Peer review is a process designed to produce a fair assessment of research quality before the publication of scholarly work in a journal. Demographics, nepotism, and seniority have been all shown to affect reviewer behavior suggesting the…
Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions. Given that human decision making is known to be…
Modern multi-stage retrieval systems are comprised of a candidate generation stage followed by one or more reranking stages. In such an architecture, the quality of the final ranked list may not be sensitive to the quality of initial…
In this paper, we describe a two-stage method for solving optimization problems with bound constraints. It combines the active-set estimate described in [Facchinei and Lucidi, 1995] with a modification of the non-monotone line search…
Peer assessment is a popular technique for a more fine-grained evaluation of individual students in group projects. Its effect on the evaluation is well studied. However, its effects on the learning abilities of students are often…