Related papers: Cycle is All You Need: More Is Different
A widely used strategy to discover and understand language model mechanisms is circuit analysis. A circuit is a minimal subgraph of a model's computation graph that executes a specific task. We identify a gap in existing circuit discovery…
In this work, we introduce Contextual Analog Logic with Multimodality (CALM). CALM unites symbolic reasoning with neural generation, enabling systems to make context-sensitive decisions grounded in real-world multi-modal data. Background:…
Due to existence of periodic windows, chaotic systems undergo numerous bifurcations as system parameters vary, rendering it hard to employ an analytic continuation, which constitutes a major obstacle for its effective analysis or…
The irreducible complexity of natural phenomena has led Graph Neural Networks to be employed as a standard model to perform representation learning tasks on graph-structured data. While their capacity to capture local and global patterns is…
The abundance of different species in a community often follows the log series distribution. Other ecological patterns also have simple forms. Why does the complexity and variability of ecological systems reduce to such simplicity? Common…
We present a novel framework for iterative visual reasoning. Our framework goes beyond current recognition systems that lack the capability to reason beyond stack of convolutions. The framework consists of two core modules: a local module…
Memory is an important cognitive function for humans. How a brain with such a small power can complete such a complex memory function, the working mechanism behind this is undoubtedly fascinating. Engram theory views memory as the…
I investigate a stronger form of regularization by deactivating neurons for extended periods, a departure from the temporary changes of methods like Dropout. However, this long-term dynamism introduces a critical challenge: severe training…
Sequence learning, prediction and replay have been proposed to constitute the universal computations performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes these forms of computation. It learns sequences in an…
A key challenge to understanding self-awareness has been a principled way of quantifying whether an intelligent system has a concept of a "self", and if so how to differentiate the "self" from other cognitive structures. We propose that the…
The generalization properties of an attractive network of non monotonic neurons which infers concepts from samples are studied. The macroscopic dynamics for the overlap between the state of the neurons with the concepts, well as the…
Simple cycles on a digraph form a trace monoid under the rule that two such cycles commute if and only if they are vertex disjoint. This rule describes the spatial configuration of simple cycles on the digraph. Cartier and Foata have showed…
Neural network training is typically viewed as gradient descent on a loss surface. We propose a fundamentally different perspective: learning is a structure-preserving transformation (a functor L) between the space of network parameters…
In biological systems, the capacity of anticipation--that is, entertaining a model of the system within the system--can be considered as naturally given. Human languages enable psychological systems to construct and exchange mental models…
Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many real-world inference problems, the typical decomposition has a large…
What is it about human brains that allows us to reason symbolically whereas most other animals cannot? There is evidence that dynamic binding, the ability to combine neurons into groups on the fly, is necessary for symbolic thought, but…
We study a discrete-time asynchronous midpoint dynamics on the circle in which, at each step, a uniformly chosen neighboring pair moves to the midpoint along the shortest arc. Although the update rule is locally contractive, we show that…
Artificial autonomous agents and robots interacting in complex environments are required to continually acquire and fine-tune knowledge over sustained periods of time. The ability to learn from continuous streams of information is referred…
Complex, oscillatory data arises from a large variety of biological, physical, and social systems. However, the inherent oscillation and ubiquitous noise pose great challenges to current methodology such as linear and nonlinear time series…
Two rate code models -- a reconstruction network model and a control model -- of the hippocampal-entorhinal loop are merged. The hippocampal-entorhinal loop plays a double role in the unified model, it is part of a reconstruction network…