Related papers: Using the Data Agreement Criterion to Rank Experts…

Textbooks on statistics emphasize care and precision, via concepts such as reliability and validity in measurement, random sampling and treatment assignment in data collection, and causal identification and bias in estimation. But how do…

This paper describes recent work on an ongoing project in medical diagnosis at the University of Guelph. A domain on which experts are not very good at pinpointing a single disease outcome is explored. On-line medical data is available over…

We investigate whether experts possess differential expertise when making predictions. We note that this would make it possible to aggregate multiple predictions into a result that is more accurate than their consensus average, and that the…

An intelligent agent will often be uncertain about various properties of its environment, and when acting in that environment it will frequently need to quantify its uncertainty. For example, if the agent wishes to employ the…

The survey is concerned with the issue of information transmission from experts to non-experts. Two main approaches to the use of experts can be traced. According to the game-theoretic approach expertise is a case of asymmetric information…

To obtain reliable results of expertise, which usually use individual and group expert pairwise comparisons, it is important to summarize (aggregate) expert estimates provided that they are sufficiently consistent. There are several ways to…

We present a new method for probabilistic elicitation of expert knowledge using binary responses of human experts assessing simulated data from a statistical model, where the parameters are subject to uncertainty. The binary responses…

In various situations, decision makers face experts that may provide conflicting advice. This advice may be in the form of probabilistic forecasts over critical future events. We consider a setting where the two forecasters provide their…

Although the usefulness of belief networks for reasoning under uncertainty is widely accepted, obtaining numerical probabilities that they require is still perceived a major obstacle. Often not enough statistical data is available to allow…

Expertise is a loosely defined concept that is hard to formalize. Much research has focused on designing efficient algorithms for expert finding in large databases in various application domains. The evaluation of such recommender systems…

We consider the problem of ranking n experts based on their performances on d tasks. We make a monotonicity assumption stating that for each pair of experts, one outperforms the other on all tasks. We consider the sequential setting where…

The problem of assessing the value of a candidate is viewed here as a multiple combination problem. On the one hand a candidate can be evaluated according to different criteria, and on the other hand several experts are supposed to assess…

We are designing an automated technique to find and recommend experts for helping in Requirements Engineering tasks, which can be done by ranking the available people by level of expertise. For evaluating the correctness of the rankings…

Ranking populations such as institutions based on certain characteristics is often of interest, and these ranks are typically estimated using samples drawn from the populations. Due to sample randomness, it is important to quantify the…

We consider a setting where in a known future time, a certain continuous random variable will be realized. There is a public prediction that gradually converges to its realized value, and an expert that has access to a more accurate…

Forensic experts use specialized training and knowledge to enable other members of the judicial system to make better informed and more just decisions. Factfinders, in particular, are tasked with judging how much weight to give to experts'…

Machine learning classification tasks often benefit from predicting a set of possible labels with confidence scores to capture uncertainty. However, existing methods struggle with the high-dimensional nature of the data and the lack of…

This paper presents an approach for developing the explanation capabilities of rule-based expert systems managing imprecise and uncertain knowledge. The treatment of uncertainty takes place in the framework of possibility theory where the…

The ranking problem is to order a collection of units by some unobserved parameter, based on observations from the associated distribution. This problem arises naturally in a number of contexts, such as business, where we may want to rank…

In large organisations, identifying experts on a given topic is crucial in leveraging the internal knowledge spread across teams and departments. So-called enterprise expert retrieval systems automatically discover and structure employees'…