相关论文: Do Differences in Values Influence Disagreements i…
The explosion in the use of software in important sociotechnical systems has renewed focus on the study of the way technical constructs reflect policies, norms, and human values. This effort requires the engagement of scholars and…
The rapid development of online media has significantly facilitated the public's information consumption, knowledge acquisition, and opinion exchange. However, it has also led to more violent conflicts in online discussions. Therefore,…
Does talking to others make people more accurate or less accurate on numeric estimates such as quantitative evaluations or probabilistic forecasts? Research on peer-to-peer communication suggests that discussion between people will usually…
In online review sites, the analysis of user feedback for assessing its helpfulness for decision-making is usually carried out by locally studying the properties of individual reviews. However, global properties should be considered as well…
Studies on interpersonal conflict have a long history and contain many suggestions for conflict typology. We use this as the basis of a novel annotation scheme and release a new dataset of situations and conflict aspect annotations. We then…
This paper studies the effect of boundary value conditions on consensus networks. Consider a network where some nodes keep their estimates constant while other nodes average their estimates with that of their neighbors. We analyze such…
Benchmarks underpin how progress in large language models (LLMs) is measured and trusted. Yet our analyses reveal that apparent convergence in benchmark accuracy can conceal deep epistemic divergence. Using two major reasoning benchmarks -…
Public discourse and opinions stem from multiple social groups. Each group has beliefs about a topic (such as vaccination, abortion, gay marriage, etc.), and opinions are exchanged and blended to produce consensus. A particular measure of…
This paper presents models for detecting agreement/disagreement in online discussions. In this work we show that by using a Siamese inspired architecture to encode the discussions, we no longer need to rely on hand-crafted features to…
Research from a variety of fields including psychology and linguistics have found correlations and patterns in personal attributes and behavior, but efforts to understand the broader heterogeneity in human behavior have not yet integrated…
The field of information retrieval often works with limited and noisy data in an attempt to classify documents into subjective categories, e.g., relevance, sentiment and controversy. We typically quantify a notion of agreement to understand…
Cognitively plausible visual dialogue models should keep a mental scoreboard of shared established facts in the dialogue context. We propose a theory-based evaluation method for investigating to what degree models pretrained on the VisDial…
Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers. Automatic metrics such as BLEU correlate weakly with human annotations, resulting in a significant bias across different models and…
Opinion dynamics aims to understand how individuals' opinions evolve through local interactions. Recently, opinion dynamics have been modeled as network games, where individuals update their opinions in order to minimize the social pressure…
We study the problem of agreement and disagreement detection in online discussions. An isotonic Conditional Random Fields (isotonic CRF) based sequential model is proposed to make predictions on sentence- or segment-level. We automatically…
Disagreement is an essential element of science and life in general. The language of probabilities and statistics is often used to describe disagreements quantitatively. In practice, however, we want much more than that. We want…
Interactions with AI assistants are increasingly personalized to individual users. As AI personalization is dynamic and machine-learning-driven, we have limited understanding of how personalization affects interaction outcomes and user…
Large language models (LLMs) are increasingly used to assign document relevance labels in information retrieval pipelines, especially in domains lacking human-labeled data. However, different models often disagree on borderline cases,…
Due to the correlational structure in our traits such as identities, cultures, and political attitudes, seemingly innocuous preferences like following a band or using a specific slang can reveal private traits. This possibility, especially…
Human annotation plays a core role in machine learning -- annotations for supervised models, safety guardrails for generative models, and human feedback for reinforcement learning, to cite a few avenues. However, the fact that many of these…