Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous Systems
Abstract
Recent advances in machine learning, particularly deep learning, have enabled autonomous systems to perceive and comprehend objects and their environments in a perceptual subsymbolic manner. These systems can now perform object detection, sensor data fusion, and language understanding tasks. However, there is a growing need to enhance these systems to understand objects and their environments more conceptually and symbolically. It is essential to consider both the explicit teaching provided by humans (e.g., describing a situation or explaining how to act) and the implicit teaching obtained by observing human behavior (e.g., through the system's sensors) to achieve this level of powerful artificial intelligence. Thus, the system must be designed with multimodal input and output capabilities to support implicit and explicit interaction models. In this position paper, we argue for considering both types of inputs, as well as human-in-the-loop and incremental learning techniques, for advancing the field of artificial intelligence and enabling autonomous systems to learn like humans. We propose several hypotheses and design guidelines and highlight a use case from related work to achieve this goal.
Cite
@article{arxiv.2309.05787,
title = {Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous Systems},
author = {Amr Gomaa and Michael Feld},
journal= {arXiv preprint arXiv:2309.05787},
year = {2023}
}
Comments
AI&HCI Workshop accepted paper at ICML2023 and accepted at ICMI2023 Blue Sky Papers. arXiv admin note: text overlap with arXiv:2211.03539