English
Related papers

Related papers: Multi-LLM QA with Embodied Exploration

200 papers

Multimodal Large Language Models (MLLMs) have shown significant advancements, providing a promising future for embodied agents. Existing benchmarks for evaluating MLLMs primarily utilize static images or videos, limiting assessments to…

Computer Vision and Pattern Recognition · Computer Science 2025-04-14 Zhili Cheng , Yuge Tu , Ran Li , Shiqi Dai , Jinyi Hu , Shengding Hu , Jiahao Li , Yang Shi , Tianyu Yu , Weize Chen , Lei Shi , Maosong Sun

We examine the capability of Multimodal Large Language Models (MLLMs) to tackle diverse domains that extend beyond the traditional language and vision tasks these models are typically trained on. Specifically, our focus lies in areas such…

Machine Learning · Computer Science 2024-12-12 Andrew Szot , Bogdan Mazoure , Omar Attia , Aleksei Timofeev , Harsh Agrawal , Devon Hjelm , Zhe Gan , Zsolt Kira , Alexander Toshev

Recent advances in Large Language Models (LLMs) have shown inspiring achievements in constructing autonomous agents that rely on language descriptions as inputs. However, it remains unclear how well LLMs can function as few-shot or…

Computation and Language · Computer Science 2024-06-25 Zixia Jia , Mengmeng Wang , Baichen Tong , Song-Chun Zhu , Zilong Zheng

Question Answering (QA) is the task of automatically answering questions posed by humans in natural languages. There are different settings to answer a question, such as abstractive, extractive, boolean, and multiple-choice QA. As a popular…

Computation and Language · Computer Science 2023-04-07 Zhichao Duan , Xiuxing Li , Zhengyan Zhang , Zhenyu Li , Ning Liu , Jianyong Wang

Recent works have shown how the reasoning capabilities of Large Language Models (LLMs) can be applied to domains beyond natural language processing, such as planning and interaction for robots. These embodied problems require an agent to…

Deploying Multimodal Large Language Models as the brain of embodied agents remains challenging, particularly under long-horizon observations and limited context budgets. Existing memory assisted methods often rely on textual summaries,…

Robotics · Computer Science 2026-03-03 Ji Li , Bo Wang , Jing Xia , Mingyi Li , Shiyan Hu

Embodied multimodal large models (EMLMs) have gained significant attention in recent years due to their potential to bridge the gap between perception, cognition, and action in complex, real-world environments. This comprehensive review…

Robotics · Computer Science 2025-02-24 Shoubin Chen , Zehao Wu , Kai Zhang , Chunyu Li , Baiyang Zhang , Fei Ma , Fei Richard Yu , Qingquan Li

Large Language Models (LLMs) have emerged as integral tools for reasoning, planning, and decision-making, drawing upon their extensive world knowledge and proficiency in language-related tasks. LLMs thus hold tremendous potential for…

Artificial Intelligence · Computer Science 2024-05-24 Xudong Guo , Kaixuan Huang , Jiale Liu , Wenhui Fan , Natalia Vélez , Qingyun Wu , Huazheng Wang , Thomas L. Griffiths , Mengdi Wang

Multimodal large language models (MLLMs) have advanced vision-language reasoning and are increasingly deployed in embodied agents. However, significant limitations remain: MLLMs generalize poorly across digital-physical spaces and…

Computational experiments have emerged as a valuable method for studying complex systems, involving the algorithmization of counterfactuals. However, accurately representing real social systems in Agent-based Modeling (ABM) is challenging…

Artificial Intelligence · Computer Science 2024-02-02 Qun Ma , Xiao Xue , Deyu Zhou , Xiangning Yu , Donghua Liu , Xuwen Zhang , Zihan Zhao , Yifan Shen , Peilin Ji , Juanjuan Li , Gang Wang , Wanpeng Ma

Large Language Models (LLMs) are becoming increasingly popular in pervasive computing due to their versatility and strong performance. However, despite their ubiquitous use, the exact mechanisms underlying their outstanding performance…

Computation and Language · Computer Science 2026-02-02 Alhassan Abdelhalim , Janick Edinger , Sören Laue , Michaela Regneri

Due to privacy concerns, open dialogue datasets for mental health are primarily generated through human or AI synthesis methods. However, the inherent implicit nature of psychological processes, particularly those of clients, poses…

Human-Computer Interaction · Computer Science 2025-11-18 Lixiu Wu , Yuanrong Tang , Qisen Pan , Xianyang Zhan , Yucheng Han , Lanxi Xiao , Tianhong Wang , Chen Zhong , Jiangtao Gong

Embodied artificial intelligence (Embodied AI) plays a pivotal role in the application of advanced technologies in the intelligent era, where AI systems are integrated with physical bodies that enable them to perceive, reason, and interact…

Artificial Intelligence · Computer Science 2025-06-24 Zhaohan Feng , Ruiqi Xue , Lei Yuan , Yang Yu , Ning Ding , Meiqin Liu , Bingzhao Gao , Jian Sun , Xinhu Zheng , Gang Wang

Despite advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), their integration into language-grounded, human-like embodied agents remains incomplete, hindering complex real-life task performance in physical…

Computation and Language · Computer Science 2024-08-20 Zhili Cheng , Zhitong Wang , Jinyi Hu , Shengding Hu , An Liu , Yuge Tu , Pengkai Li , Lei Shi , Zhiyuan Liu , Maosong Sun

In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and solving mathematical problems, leading to advancements in various fields. We propose an LLM-embodied path planning framework for…

Robotics · Computer Science 2024-07-08 Xiangrui Kong , Wenxiao Zhang , Jin Hong , Thomas Braunl

Multi-agent large language models (MA-LLMs) are a rapidly growing research area that leverages multiple interacting language agents to tackle complex tasks, outperforming single-agent large language models. This literature review…

Multiagent Systems · Computer Science 2025-06-03 Arne Tillmann

Accurate and efficient question-answering systems are essential for delivering high-quality patient care in the medical field. While Large Language Models (LLMs) have made remarkable strides across various domains, they continue to face…

Computation and Language · Computer Science 2025-01-22 Hang Yang , Hao Chen , Hui Guo , Yineng Chen , Ching-Sheng Lin , Shu Hu , Jinrong Hu , Xi Wu , Xin Wang

With the surge in the development of large language models, embodied intelligence has attracted increasing attention. Nevertheless, prior works on embodied intelligence typically encode scene or historical memory in an unimodal manner,…

Computer Vision and Pattern Recognition · Computer Science 2024-07-30 Yang Liu , Xinshuai Song , Kaixuan Jiang , Weixing Chen , Jingzhou Luo , Guanbin Li , Liang Lin

Large Language Models (LLMs) have demonstrated remarkable capabilities in conversational tasks. Embodying an LLM as a virtual human allows users to engage in face-to-face social interactions in Virtual Reality. However, the influence of…

Human-Computer Interaction · Computer Science 2025-08-06 Leon O. H. Kroczek , Alexander May , Selina Hettenkofer , Andreas Ruider , Bernd Ludwig , Andreas Mühlberger

This paper reviews the state-of-the-art of large language models (LLM) architectures and strategies for "complex" question-answering with a focus on hybrid architectures. LLM based chatbot services have allowed anyone to grasp the potential…

Computation and Language · Computer Science 2025-11-04 Xavier Daull , Patrice Bellot , Emmanuel Bruno , Vincent Martin , Elisabeth Murisasco