Related papers: Ten simple rules for measuring the impact of works…
In recent years, evaluation of the quality of academic research has become an increasingly important and influential business. It determines, often to a large extent, the amount of research funding flowing into universities and similar…
We present CrowdHub, a tool for running systematic evaluations of task designs on top of crowdsourcing platforms. The goal is to support the evaluation process, avoiding potential experimental biases that, according to our empirical…
Citation metrics are becoming pervasive in the quantitative evaluation of scholars, journals and institutions. More then ever before, hiring, promotion, and funding decisions rely on a variety of impact metrics that cannot disentangle…
Mixed-effect models are flexible tools for researchers in a myriad of fields, but that flexibility comes at the cost of complexity and if users are not careful in how their model is specified, they could be making faulty inferences from…
In an ever-changing world, even software that fulfils its requirements may have un-envisioned aftereffects with significant impacts. We explored how such impacts can be better understood at the pre-design phase in support of organisational…
Testing knowledge is an integral part of summative assessment at schools. It can be performed in many different ways. In this study we propose assessment of physics knowledge by using a tournament approach. Prior to a statistical analysis…
We describe and contrast two distinct problem areas for statistical causality: studying the likely effects of an intervention ("effects of causes"), and studying whether there is a causal link between the observed exposure and outcome in an…
This study evaluates the effectiveness of incorporating makerspace workshops into computer science education by assessing 21st century skills critical thinking, collaboration, communication, and creativity before and after the intervention.…
The collaborative development methods pioneered by the open source software community offer a way to create lessons that are open, accessible, and sustainable. This paper presents ten simple rules for doing this drawn from our experience…
Building of tools--from simple prototypes to industrial-strength applications--is a pervasive activity in academic research. When proposing a new technique for software maintenance, effective tool support is typically required to…
Measuring meaning is a central problem in cultural sociology and word embeddings may offer powerful new tools to do so. But like any tool, they build on and exert theoretical assumptions. In this paper I theorize the ways in which word…
A primary difficulty with unsupervised discovery of structure in large data sets is a lack of quantitative evaluation criteria. In this work, we propose and investigate several metrics for evaluating and comparing generative models of…
Quantum mechanics is a field often considered very mathematical, abstract, and unintuitive. One way some instructors are hoping to help familiarize their students with these complex topics is to have the students see quantum effects in…
The influence maximization is the problem of finding a set of social network users, called influencers, that can trigger a large cascade of propagation. Influencers are very beneficial to make a marketing campaign goes viral through social…
We study ways of evaluating the performance of losing projects in participatory budgeting (PB) elections by seeking actions that would have led to their victory. We focus on lowering the projects' costs, obtaining additional approvals for…
The digital transformation of work presents new opportunities to understand how informal workgroups organize around the dynamic needs of organizations, potentially in contrast to the formal, static, and idealized hierarchies depicted by org…
Context: Forgetting is defined as a gradual process of losing information. Even though there are many studies demonstrating the effect of forgetting in software development, to the best of our knowledge, no study explores the impact of…
The need for high-throughput, precise, and meaningful methods for measuring behavior has been amplified by our recent successes in measuring and manipulating neural circuitry. The largest challenges associated with moving in this direction,…
Measurement is a complicated but very necessary task. Many indices have been created in an effort to define the quality of knowledge produced but they have attracted strong criticism, having become synonymous with individualism, competition…
Our work aims to study tools offered to students and tutors involved in face-to-face or blended project- based learning activities. Project-based learning is often applied in the case of complex learning (i.e. which aims at making learners…