English
Related papers

Related papers: Three Ways of Using Large Language Models to Evalu…

200 papers

Dialogue State Tracking (DST) is of paramount importance in ensuring accurate tracking of user goals and system actions within task-oriented dialogue systems. The emergence of large language models (LLMs) such as GPT3 and ChatGPT has…

Computation and Language · Computer Science 2023-10-24 Yujie Feng , Zexin Lu , Bo Liu , Liming Zhan , Xiao-Ming Wu

In 2025, Large Language Model (LLM) services have launched a new feature -- AI video chat -- allowing users to interact with AI agents via real-time video communication (RTC), just like chatting with real people. Despite its significance,…

Networking and Internet Architecture · Computer Science 2025-10-02 Jiayang Xu , Xiangjie Huang , Zijie Li , Zili Meng

Large Language Models (LLMs) have demonstrated remarkable performance across various information-seeking and reasoning tasks. These computational systems drive state-of-the-art dialogue systems, such as ChatGPT and Bard. They also carry…

Computation and Language · Computer Science 2023-10-13 Siyuan Brandon Loh , Aravind Sesagiri Raamkumar

This work introduces LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training. Leveraging a taxonomy-guided…

Computation and Language · Computer Science 2024-05-01 Shivchander Sudalairaj , Abhishek Bhandwaldar , Aldo Pareja , Kai Xu , David D. Cox , Akash Srivastava

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the…

Artificial Intelligence · Computer Science 2023-09-04 Natalie Kiesler , Dominic Lohr , Hieke Keuning

In this paper, we propose a feature pioneering method using Large Language Models (LLMs). In the proposed method, we use Chat-GPT 1 to find new sensor locations and new features. Then we evaluate the machine learning model which uses the…

Human-Computer Interaction · Computer Science 2023-10-10 Haru Kaneko , Sozo Inoue

Real-time, intelligent, and natural speech interaction is an essential part of the next-generation human-computer interaction. Recent advancements have showcased the potential of building intelligent spoken chatbots based on large language…

Computation and Language · Computer Science 2025-05-06 Qingkai Fang , Yan Zhou , Shoutao Guo , Shaolei Zhang , Yang Feng

Recent efforts have evaluated large language models (LLMs) in areas such as commonsense reasoning, mathematical reasoning, and code generation. However, to the best of our knowledge, no work has specifically investigated the performance of…

Computation and Language · Computer Science 2024-05-17 Xuanfan Ni , Piji Li

Since the breakthrough of ChatGPT, large language models (LLMs) have garnered significant attention in the research community. With the development of LLMs, the question of text style transfer for conversational models has emerged as a…

Computation and Language · Computer Science 2024-03-15 Jianlin Chen

Automatic evaluation is an integral aspect of dialogue system research. The traditional reference-based NLG metrics are generally found to be unsuitable for dialogue assessment. Consequently, recent studies have suggested various unique,…

Computation and Language · Computer Science 2024-01-23 Chen Zhang , Luis Fernando D'Haro , Yiming Chen , Malu Zhang , Haizhou Li

In this technical report, we present TeleChat, a collection of large language models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It includes pretrained language models as well as fine-tuned chat models that is aligned…

Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Therefore, it is of great importance to evaluate their emerging abilities. In this study, we show that LLMs…

Computation and Language · Computer Science 2023-10-10 Thilo Hagendorff , Sarah Fabi , Michal Kosinski

Large language models (LLMs) are a special class of pretrained language models obtained by scaling model size, pretraining corpus and computation. LLMs, because of their large size and pretraining on large volumes of text data, exhibit…

Computation and Language · Computer Science 2023-10-20 Katikapalli Subramanyam Kalyan

Unlocking the potential of Large Language Models (LLMs) in data classification represents a promising frontier in natural language processing. In this work, we evaluate the performance of different LLMs in comparison with state-of-the-art…

Computation and Language · Computer Science 2025-01-16 Arina Kostina , Marios D. Dikaiakos , Dimosthenis Stefanidis , George Pallis

This research explores the integration of chatbot technology powered by GPT-3.5 and GPT-4 Turbo into higher education to enhance internationalization and leverage digital transformation. It delves into the design, implementation, and…

Computers and Society · Computer Science 2024-03-25 Achraf Hsain , Hamza El Housni

This paper presents ChatCounselor, a large language model (LLM) solution designed to provide mental health support. Unlike generic chatbots, ChatCounselor is distinguished by its foundation in real conversations between consulting clients…

Computation and Language · Computer Science 2023-09-28 June M. Liu , Donghao Li , He Cao , Tianhe Ren , Zeyi Liao , Jiamin Wu

Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages and challenges of LLMs for MMT by answering two questions:…

Computation and Language · Computer Science 2024-06-17 Wenhao Zhu , Hongyi Liu , Qingxiu Dong , Jingjing Xu , Shujian Huang , Lingpeng Kong , Jiajun Chen , Lei Li

Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as…

Human-Computer Interaction · Computer Science 2024-05-08 Jing Wei , Sungdong Kim , Hyunhoon Jung , Young-Ho Kim

Each day, individuals set behavioral goals such as eating healthier, exercising regularly, or increasing productivity. While psychological frameworks (i.e., goal setting and implementation intentions) can be helpful, they often need…

Human-Computer Interaction · Computer Science 2026-02-10 Michel Schimpf , Sebastian Maier , Anton Wyrowski , Lara Christoforakos , Stefan Feuerriegel , Thomas Bohné

Providing scaffolding through educational chatbots built on Large Language Models (LLM) has potential risks and benefits that remain an open area of research. When students navigate impasses, they ask for help by formulating impasse-driven…

Human-Computer Interaction · Computer Science 2026-02-23 Alexandra Neagu , Marcus Messer , Peter Johnson , Rhodri Nelson