Related papers: Democratizing online controlled experiments at Boo…
Performing controlled user experiments on small devices in naturalistic mobile settings has always proved to be a difficult undertaking for many Human Factors researchers. Difficulties exist, not least, because mimicking natural small…
The ability to assess the reputation of a member in a web community is a need addressed in many different ways according to the many different stages in which the nature of communities has evolved over time. In the case of reputation of…
Decentralisation is one of the promises introduced by blockchain technologies: fair and secure interaction amongst peers with no dominant positions, single points of failure or censorship. Decentralisation, however, appears difficult to be…
Booking.com is a virtual two-sided marketplace where guests and accommodation providers are the two distinct stakeholders. They meet to satisfy their respective and different goals. Guests want to be able to choose accommodations from a…
Online controlled experiments (A/B tests) have become the gold standard for learning the impact of new product features in technology companies. Randomization enables the inference of causality from an A/B test. The randomized assignment…
Cooperation is one of the behavioral traits that define human beings, however we are still trying to understand why humans cooperate. Behavioral experiments have been largely conducted to shed light into the mechanisms behind cooperation…
Recommendation based on user preferences is a common task for e-commerce websites. New recommendation algorithms are often evaluated by offline comparison to baseline algorithms such as recommending random or the most popular items. Here,…
Recommender systems have become an integral part of online platforms, providing personalized recommendations for purchases, content consumption, and interpersonal connections. These systems consist of two sides: the producer side comprises…
Development of the majority of the leading web services and software products today is generally guided by data-driven decisions based on evaluation that ensures a steady stream of updates, both in terms of quality and quantity. Large…
We demonstrate the effectiveness of democratization and efficient computation as key concepts of our experimentation platform (XP) by presenting four new models supported by the platform: 1) Weighted least squares, 2) Quantile…
A common concern in experimental research is the auditability and reproducibility of experiments. Experiments are usually designed, provisioned, managed, and analyzed by diverse teams of specialists (e.g., researchers, technicians and…
Proponents of blockchains believe that this technology will revolutionize e-commerce. To evaluate this belief, we invite several groups of students to transact on a decentralized peer-to-peer marketplace built on the platform provided by…
Online portals include an increasing amount of user feedback in form of ratings and reviews. Recent research highlighted the importance of this feedback and confirmed that positive feedback improves product sales figures and thus its…
The reproducibility of scientific experiment is vital for the advancement of disciplines based on previous work. To achieve this goal, many researchers focus on complex methodology and self-invented tools which have difficulty in practical…
When it comes to industrial organizations, current collaboration efforts in software engineering research are very often kept in-house, depriving these organizations off the skills necessary to build independent collaborative research. The…
While automated experiments and high-throughput methods are becoming more mainstream in the age of data, empowering individual researchers to capture, collate, and contextualize their data faster and more reproducibly still remains a…
Online labor markets have great potential as platforms for conducting experiments, as they provide immediate access to a large and diverse subject pool and allow researchers to conduct randomized controlled trials. We argue that online…
Blockchain, like any other complex technology, needs a strong testing methodology to support its evolution in both research and development contexts. Setting up meaningful tests for permissionless blockchain technology is a notoriously…
This chapter reviews empirical evidence bearing on the design of online forums for deliberative civic engagement. Dimensions of design are defined for different aspects of the deliberation: its purpose, the target population, the…
Participatory design initiatives, especially within the realm of digital civics, are often integrated and co-developed with the very citizens and communities they intend to assist. Digital civics research aims to create positive social…