Related papers: Trustworthiness in Enterprise Crowdsourcing: a Tax…
Quality improvement methods are essential to gathering high-quality crowdsourced data, both for research and industry applications. A popular and broadly applicable method is task assignment that dynamically adjusts crowd workflow…
The recent boom in crowdsourcing has opened up a new avenue for utilizing human intelligence in the realm of data analysis. This innovative approach provides a powerful means for connecting online workers to tasks that cannot effectively be…
Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people are used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused…
We conduct an experimental analysis of a dataset comprising over 27 million microtasks performed by over 70,000 workers issued to a large crowdsourcing marketplace between 2012-2016. Using this data---never before analyzed in an academic…
The concept of social trust has attracted an attention of information processors/data scientists and information consumers / business firms. One of the main reasons for acquiring the value of SBD is to provide frameworks and methodologies…
Crowdsourcing is defined as the outsourcing of tasks to a crowd of contributors. The crowd is very diverse on these platforms and includes malicious contributors attracted by the remuneration of tasks and not conscientiously performing…
Recent years have seen a surge in online collaboration between experts and amateurs on scientific research. In this article, we analyse the epistemological implications of these crowdsourced projects, with a focus on Zooniverse, the world's…
New techniques leveraging IT-mediated crowds such as Crowdsensing, Situated Crowdsourcing, Spatial Crowdsourcing, and Wearables Crowdsourcing have now materially emerged. These techniques, here termed next generation Crowdsourcing, serve to…
Crowdsourcing systems often have crowd workers that perform unreliable work on the task they are assigned. In this paper, we propose the use of error-control codes and decoding algorithms to design crowdsourcing systems for reliable…
Many software developments projects fail due to quality problems. Software testing enables the creation of high quality software products. Since it is a cumbersome and expensive task, and often hard to manage, both its technical background…
Crowdsourcing is beginning to be used for policymaking. The wisdom of crowds [Surowiecki 2005], and crowdsourcing [Brabham 2008], are seen as new avenues that can shape all kinds of policy, from transportation policy [Nash 2009] to urban…
We propose a novel generic trust management framework for crowdsourced IoT services. The framework exploits a multi-perspective trust model that captures the inherent characteristics of crowdsourced IoT services. Each perspective is defined…
In any given collaboration, information needs to flow from one participant to another. While participants may be interested in sharing information with one another, it is often necessary for them to establish the impact of sharing certain…
A key challenge of big data analytics is how to collect a large volume of (labeled) data. Crowdsourcing aims to address this challenge via aggregating and estimating high-quality data (e.g., sentiment label for text) from pervasive…
While machine learning (ML) technology affects diverse stakeholders, there is no one-size-fits-all metric to evaluate the quality of outputs, including performance and fairness. Using predetermined metrics without soliciting stakeholder…
Human computation is an approach to solving problems that prove difficult using AI only, and involves the cooperation of many humans. Because human computation requires close engagement with both "human populations as users" and "human…
The widespread utilization of AI systems has drawn attention to the potential impacts of such systems on society. Of particular concern are the consequences that prediction errors may have on real-world scenarios, and the trust humanity…
With the development of mobile social networks, more and more crowdsourced data are generated on the Web or collected from real-world sensing. The fragment, heterogeneous, and noisy nature of online/offline crowdsourced data, however, makes…
This paper investigates the shift in crowdsourcing towards self-managed enterprises of crowdworkers (SMECs), diverging from traditional platform-controlled models. It reviews the literature to understand the foundational aspects of this…
The present research examines two problems inherent to the creation of crowdsourcing games: how to give feedback when the right answer is not always known by the game and how much time to give players without sacrificing data quality. Taken…