English
Related papers

Related papers: Evaluating Language Models for Knowledge Base Comp…

200 papers

In this paper we introduce the notion of Demand-Weighted Completeness, allowing estimation of the completeness of a knowledge base with respect to how it is used. Defining an entity by its classes, we employ usage data to predict the…

Artificial Intelligence · Computer Science 2018-05-01 Andrew Hopkinson , Amit Gurdasani , Dave Palfrey , Arpit Mittal

Large language models (LLMs) have raised hopes for automated end-to-end fact-checking, but prior studies report mixed results. As mainstream chatbots increasingly ship with reasoning capabilities and web search tools -- and millions of…

Computation and Language · Computer Science 2025-11-25 Matthew R. DeVerna , Kai-Cheng Yang , Harry Yaojun Yan , Filippo Menczer

Concept Bottleneck Models (CBMs) have been proposed as a compromise between white-box and black-box models, aiming to achieve interpretability without sacrificing accuracy. The standard training procedure for CBMs is to predefine a…

Machine Learning · Computer Science 2025-12-05 Jean Feng , Avni Kothari , Luke Zier , Chandan Singh , Yan Shuo Tan

Contextual word representations, typically trained on unstructured, unlabeled text, do not contain any explicit grounding to real world entities and are often unable to remember facts about those entities. We propose a general method to…

Computation and Language · Computer Science 2019-11-01 Matthew E. Peters , Mark Neumann , Robert L. Logan , Roy Schwartz , Vidur Joshi , Sameer Singh , Noah A. Smith

The extensive utilization of large language models (LLMs) underscores the crucial necessity for precise and contemporary knowledge embedded within their intrinsic parameters. Existing research on knowledge editing primarily concentrates on…

Computation and Language · Computer Science 2025-02-20 Zihao Wei , Jingcheng Deng , Liang Pang , Hanxing Ding , Huawei Shen , Xueqi Cheng

Knowledge Base Question Answering (KBQA) challenges models to bridge the gap between natural language and strict knowledge graph schemas by generating executable logical forms. While Large Language Models (LLMs) have advanced this field,…

Computation and Language · Computer Science 2026-01-12 Xin Sun , Zhongqi Chen , Xing Zheng , Qiang Liu , Shu Wu , Bowen Song , Zilei Wang , Weiqiang Wang , Liang Wang

Question answering over knowledge bases (KBQA) aims to answer factoid questions with a given knowledge base (KB). Due to the large scale of KB, annotated data is impossible to cover all fact schemas in KB, which poses a challenge to the…

Computation and Language · Computer Science 2023-05-24 Chuanyuan Tan , Yuehe Chen , Wenbiao Shao , Wenliang Chen

Large language models hallucinate factual claims and struggle to ground their outputs in retrievable evidence, particularly in non-English languages. Existing resources impose a trade-off: structured knowledge bases lack textual grounding,…

Computation and Language · Computer Science 2026-05-15 Yingli Shen , Wen Lai , Jie Zhou , Xueren Zhang , Yudong Wang , Kangyang Luo , Shuo Wang , Ge Gao , Alexander Fraser , Maosong Sun

During the past few decades, knowledge bases (KBs) have experienced rapid growth. Nevertheless, most KBs still suffer from serious incompletion. Researchers proposed many tasks such as knowledge base completion and relation prediction to…

Computation and Language · Computer Science 2019-04-23 Zihao Fu , Yankai Lin , Zhiyuan Liu , Wai Lam

The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or…

Computation and Language · Computer Science 2021-09-10 Nicola De Cao , Wilker Aziz , Ivan Titov

State of the art large language models (LLMs) have shown impressive performance on a variety of benchmark tasks and are increasingly used as components in larger applications, where LLM-based predictions serve as proxies for human…

Computation and Language · Computer Science 2024-06-14 Michael Franke , Polina Tsvilodub , Fausto Carcassi

Large Language Models (LLMs) excel at tackling various natural language tasks. However, due to the significant costs involved in re-training or fine-tuning them, they remain largely static and difficult to personalize. Nevertheless, a…

Information Retrieval · Computer Science 2024-02-20 Jinheon Baek , Nirupama Chandrasekaran , Silviu Cucerzan , Allen herring , Sujay Kumar Jauhar

Large language models (LLMs) are increasingly used to meet user information needs, but their effectiveness in dealing with user queries that contain various types of ambiguity remains unknown, ultimately risking user trust and satisfaction.…

Computation and Language · Computer Science 2024-06-04 Tong Zhang , Peixin Qin , Yang Deng , Chen Huang , Wenqiang Lei , Junhong Liu , Dingnan Jin , Hongru Liang , Tat-Seng Chua

Knowledge-enhanced Pre-trained Language Model (PLM) has recently received significant attention, which aims to incorporate factual knowledge into PLMs. However, most existing methods modify the internal structures of fixed types of PLMs by…

Computation and Language · Computer Science 2022-10-18 Jianing Wang , Wenkang Huang , Qiuhui Shi , Hongbin Wang , Minghui Qiu , Xiang Li , Ming Gao

Problem-solving has been a fundamental driver of human progress in numerous domains. With advancements in artificial intelligence, Large Language Models (LLMs) have emerged as powerful tools capable of tackling complex problems across…

Machine Learning · Computer Science 2025-05-07 Da Zheng , Lun Du , Junwei Su , Yuchen Tian , Yuqi Zhu , Jintian Zhang , Lanning Wei , Ningyu Zhang , Huajun Chen

Language Models (LMs) continue to advance, improving response quality and coherence. Given Internet-scale training datasets, LMs have likely encountered much of what users may ask them to generate in some form during their training. A…

Artificial Intelligence · Computer Science 2026-01-27 Michael Majurski , Cynthia Matuszek

Existing work investigates the reasoning capabilities of large language models (LLMs) to uncover their limitations, human-like biases and underlying processes. Such studies include evaluations of base LLMs (pre-trained on unlabeled corpora…

Computation and Language · Computer Science 2025-11-14 Jason Chan , Zhixue Zhao , Robert Gaizauskas

The rapid growth of biomedical knowledge has outpaced our ability to efficiently extract insights and generate novel hypotheses. Large language models (LLMs) have emerged as a promising tool to revolutionize knowledge interaction and…

Computation and Language · Computer Science 2024-07-16 Biqing Qi , Kaiyan Zhang , Kai Tian , Haoxiang Li , Zhang-Ren Chen , Sihang Zeng , Ermo Hua , Hu Jinfang , Bowen Zhou

Schema matching (SM) and entity matching (EM) tasks are crucial for data integration. While large language models (LLMs) have shown promising results in these tasks, they suffer from hallucinations and confusion about task instructions.…

Computation and Language · Computer Science 2025-02-18 Yongqin Xu , Huan Li , Ke Chen , Lidan Shou

The usefulness and usability of knowledge bases (KBs) is often limited by quality issues. One common issue is the presence of erroneous assertions, often caused by lexical or semantic confusion. We study the problem of correcting such…

Artificial Intelligence · Computer Science 2020-01-22 Jiaoyan Chen , Xi Chen , Ian Horrocks , Ernesto Jimenez-Ruiz , Erik B. Myklebus