Related papers: Questionnaires and Qualitative Feedback Methods to…
During acceptance testing customers assess whether a system meets their expectations and often identify issues that should be improved. These findings have to be communicated to the developers a task we observed to be error prone,…
Cognitive Dimensions is a framework for analyzing human-computer interaction. It is used for meta-analysis, that is, for talking about characteristics of systems without getting bogged down in details of a particular implementation. In this…
Human feedback is increasingly used to steer the behaviours of Large Language Models (LLMs). However, it is unclear how to collect and incorporate feedback in a way that is efficient, effective and unbiased, especially for highly subjective…
We study user sentiment (reported via optional surveys) as a metric for fully randomized A/B tests. Both user-level covariates and treatment assignment can impact response propensity. We propose a set of consistent estimators for the…
The cognitive load can be used to assess if someone is struggling while performing a task. It can be used in many different situations such as in driving, piloting, studying, playing, working, etc. This information can help to design better…
We propose the following question: what game-like interactive system would provide a good environment for measuring the impact and success of a co-creative, cooperative agent? Creativity is often formulated in terms of novelty, value,…
Quantum phase estimation is fundamental to advancing quantum science and technology. While much of the research has concentrated on estimating a single phase, the simultaneous estimation of multiple phases can yield significantly enhanced…
With the advancement in technology, raw event data generated by the digital world have grown tremendously. However, such data tend to be insufficient and noisy when it comes to measuring user intention or satisfaction. One effective way to…
Providing an equitable and inclusive user experience (UX) for people with disabilities (PWD) is a central goal of accessible design. In the specific case of Deaf users, whose hearing impairments impact language development and…
The search engine evaluation research has quite a lot metrics available to it. Only recently, the question of the significance of individual metrics started being raised, as these metrics' correlations to real-world user experiences or…
Virtual Reality (VR) techniques, serving as the bridge between the real and virtual worlds, have boomed and are widely used in manufacturing, remote healthcare, gaming, etc. Specifically, VR systems offer users immersive experiences that…
Gameplay research about experiential phenomena is a challenging undertaking, given the variety of experiences that gamers encounter when playing and which currently do not have a formal taxonomy, such as flow, immersion, boredom, and fun.…
Quantum physics holds the promise of enabling certain tasks with better performance than possible when only classical resources are employed. The quantum phenomena present in many experiments signify nonclassical behavior, but do not always…
Subjective assessment tests are often employed to evaluate image processing systems, notably image and video compression, super-resolution among others and have been used as an indisputable way to provide evidence of the performance of an…
Product review websites provide an incredible lens into the wide variety of opinions and experiences of different people, and play a critical role in helping users discover products that match their personal needs and preferences. To help…
Evaluating and optimising human comfort within the built environment is challenging due to the large number of physiological, psychological and environmental variables that affect occupant comfort preference. Human perception could be…
Listening to user's requirements is crucial to building and maintaining high quality software. Online software user feedback has been shown to contain large amounts of information useful to requirements engineering (RE). Previous studies…
Though it has been recognized that recommending serendipitous (i.e., surprising and relevant) items can be helpful for increasing users' satisfaction and behavioral intention, how to measure serendipity in the offline environment is still…
Evaluating LLMs is challenging, as benchmark scores often fail to capture models' real-world usefulness. Instead, users often rely on ``vibe-testing'': informal experience-based evaluation, such as comparing models on coding tasks related…
With growing success in experimental implementations it is critical to identify a "gold standard" for quantum information processing, a single measure of distance that can be used to compare and contrast different experiments. We enumerate…