Related papers: Experiments on Crowdsourcing Policy Assessment
Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people are used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused…
Machine Learning models have many potentially beneficial applications in education settings, but a key barrier to their development is securing enough data to train these models. Labelling educational data has traditionally relied on highly…
This paper reports on the challenges and lessons we learned while running controlled experiments in crowdsourcing platforms. Crowdsourcing is becoming an attractive technique to engage a diverse and large pool of subjects in experimental…
Crowdsourcing information constitutes an important aspect of human-in-the-loop learning for researchers across multiple disciplines such as AI, HCI, and social science. While using crowdsourced data for subjective tasks is not new,…
Generating models from large data sets -- and determining which subsets of data to mine -- is becoming increasingly automated. However choosing what data to collect in the first place requires human intuition or experience, usually supplied…
In this work we seek to understand how differences in location affect participation outcomes in IT-mediated crowds. To do so, we operationalize Crowd Capital Theory with data from a popular international creative crowdsourcing site, to…
In this study we investigate to which degree experts and non-experts agree on questions of difficulty in a crowdsourcing experiment. We ask non-experts (second language learners of Swedish) and two groups of experts (teachers of Swedish as…
In the last decade, crowdsourcing has become a popular method for conducting quantitative empirical studies in human-machine interaction. The remote work on a given task in crowdworking settings suits the character of typical…
Crowdsourcing is beginning to be used for policymaking. The wisdom of crowds [Surowiecki 2005], and crowdsourcing [Brabham 2008], are seen as new avenues that can shape all kinds of policy, from transportation policy [Nash 2009] to urban…
The increasing practice of engaging crowds, where organizations use IT to connect with dispersed individuals for explicit resource creation purposes, has precipitated the need to measure the precise processes and benefits of these…
We consider unsupervised crowdsourcing performance based on the model wherein the responses of end-users are essentially rated according to how their responses correlate with the majority of other responses to the same subtasks/questions.…
Crowdsourcing refers to the arrangement in which contributions are solicited from a large group of unrelated people. Due to this nature, crowdsourcers (or task requesters) often face uncertainty about the workers' capabilities which, in…
Crowdsourcing has been successfully employed in the past as an effective and cheap way to execute classification tasks and has therefore attracted the attention of the research community. However, we still lack a theoretical understanding…
To begin to understand the implications of the implementation of IT-mediated Crowds for Politics and Policy purposes, this research builds the first-known dataset of IT-mediated Crowd applications currently in use in the governance context.…
With the increasing pervasiveness of algorithms across industry and government, a growing body of work has grappled with how to understand their societal impact and ethical implications. Various methods have been used at different stages of…
Crowd simulation is a research area widely used in diverse fields, including gaming and security, assessing virtual agent movements through metrics like time to reach their goals, speed, trajectories, and densities. This is relevant for…
Crowdsourced data supports real-time decision-making but faces challenges like misinformation, errors, and contributor power concentration. This study systematically examines trust management practices across platforms categorised as…
Fact-checking is one of the effective solutions in fighting online misinformation. However, traditional fact-checking is a process requiring scarce expert human resources, and thus does not scale well on social media because of the…
Misinformation about critical issues such as climate change and vaccine safety is oftentimes amplified on online social and search platforms. The crowdsourcing of content credibility assessment by laypeople has been proposed as one strategy…
Social Media provides researchers with an unprecedented opportunity to gain insight into various facets of human life. Health practitioners put a great emphasis on pinpointing socioeconomic status (SES) of individuals as they can use to it…