Related papers: Democratizing online controlled experiments at Boo…
We consider the problem of conducting large experimental campaigns in programming languages research. Most research efforts require a certain level of bookkeeping of results. This is manageable via quick, on-the-fly infrastructure…
Computational experiments have become essential for scientific discovery, allowing researchers to test hypotheses, analyze complex datasets, and validate findings. However, as computational experiments grow in scale and complexity, ensuring…
There is a growing interest in utilizing digital services, such as software apps and cloud-based software services. The utilization of digital services is increasing more rapidly than any other segment of world trade. The availability of…
During acceptance testing customers assess whether a system meets their expectations and often identify issues that should be improved. These findings have to be communicated to the developers a task we observed to be error prone,…
The last 30 years have seen the creation of a variety of electronic collaboration tools for science and business. Some of the best-known collaboration tools support text editing (e.g., wikis). Wikipedia's success shows that large-scale…
Decentralization initiatives like Solid enable data owners to control who has access to their data and to stimulate innovation by creating both application and data markets. Once data owners share their data with others, though, it is no…
A/B testing, or controlled experiments, is the gold standard approach to causally compare the performance of algorithms on online platforms. However, conventional Bernoulli randomization in A/B testing faces many challenges such as…
The goal of experiment design is to select the inputs of a dynamical system in such a way that the resulting data contain sufficient information for system identification and data-driven control. This paper investigates the problem of…
Decentralization is a fundamental design element of the Web3 economy. Blockchains and distributed consensus mechanisms are touted as fault-tolerant, attack-resistant, and collusion-proof because they are decentralized. Recent analyses,…
Online experimentation platforms abstract away many of the details of experimental design, ensuring experimenters do not have to worry about sampling, randomisation, subject tracking, data collection, metric definition and interpretation of…
Data democratization is an ongoing process that broadens access to data and facilitates employees to find, access, self-analyze, and share data without additional support. This data access management process enables organizations to make…
A data commons is a cloud-based data platform with a governance structure that allows a community to manage, analyze and share its data. Data commons provide a research community with the ability to manage and analyze large datasets using…
Many online experiments exhibit dependence between users and items. For example, in online advertising, observations that have a user or an ad in common are likely to be associated. Because of this, even in experiments involving millions of…
While online services emerge in all areas of life, the voting procedure in many democracies remains paper-based as the security of current online voting technology is highly disputed. We address the issue of trustworthy online voting…
Over the past 40 years, database management systems (DBMSs) have evolved to provide a sophisticated variety of data management capabilities. At the same time, tools for managing queries over the data have remained relatively primitive. One…
Participatory design in digital civics aims to foster mutual learning and co-creation between public services and citizens. However, rarely do we collectively explore the challenges and failures we experience within PD and digital civics,…
Governance in online communities is an increasingly high-stakes challenge, and yet many basic features of offline governance legacies--juries, political parties, term limits, and formal debates, to name a few--are not in the feature-sets of…
Data can be collected in scientific studies via a controlled experiment or passive observation. Big data is often collected in a passive way, e.g. from social media. In studies of causation great efforts are made to guard against bias and…
Digitisation is often viewed as beneficial to a user. Whereas traditionally, people would physically have to identify to a service, pay for a ticket in cash, or go into a library to access a book, people can now achieve all of this through…
Recommender systems have become increasingly important with the rise of the web as a medium for electronic and business transactions. One of the key drivers of this technology is the ease with which users can provide feedback about their…