English

Directed Structural Adaptation to Overcome Statistical Conflicts and Enable Continual Learning

Machine Learning 2024-12-06 v1 Artificial Intelligence

Abstract

Adaptive networks today rely on overparameterized fixed topologies that cannot break through the statistical conflicts they encounter in the data they are exposed to, and are prone to "catastrophic forgetting" as the network attempts to reuse the existing structures to learn new task. We propose a structural adaptation method, DIRAD, that can complexify as needed and in a directed manner without being limited by statistical conflicts within a dataset. We then extend this method and present the PREVAL framework, designed to prevent "catastrophic forgetting" in continual learning by detection of new data and assigning encountered data to suitable models adapted to process them, without needing task labels anywhere in the workflow. We show the reliability of the DIRAD in growing a network with high performance and orders-of-magnitude simpler than fixed topology networks; and demonstrate the proof-of-concept operation of PREVAL, in which continual adaptation to new tasks is observed while being able to detect and discern previously-encountered tasks.

Keywords

Cite

@article{arxiv.2412.04190,
  title  = {Directed Structural Adaptation to Overcome Statistical Conflicts and Enable Continual Learning},
  author = {Zeki Doruk Erden and Boi Faltings},
  journal= {arXiv preprint arXiv:2412.04190},
  year   = {2024}
}

Comments

Presented in Deployable AI (DAI) workshop at AAAI-2024

R2 v1 2026-06-28T20:24:15.962Z