Related papers: Hessian-information geometric formulation of a cla…

Datasets such as images, text, or movies are embedded in high-dimensional spaces. However, in important cases such as images of objects, the statistical structure in the data constrains samples to a manifold of dramatically lower…

Bridging information geometry with machine learning, this paper presents a method for constructing neural networks intrinsically on statistical manifolds. We demonstrate this approach by formulating a neural network architecture directly on…

In this paper a class of classical Hamiltonian systems is geometrically formulated. This class is such that a Hamiltonian can be written as the sum of a kinetic energy function and a potential energy function. In addition, these energy…

In the present work, a hyperelastic constitutive model based on neural networks is proposed which fulfills all common constitutive conditions by construction, and in particular, is applicable to compressible material behavior. Using…

Neural population activity in cortical and hippocampal circuits can be flexibly reorganized by context, suggesting that cognition relies on dynamic manifolds rather than static representations. However, how such dynamic organization can be…

The ever-increasing parameter counts of deep learning models necessitate effective compression techniques for deployment on resource-constrained devices. This paper explores the application of information geometry, the study of…

We propose a dynamical neural network model with a hierarchical and modular structure. The network architecture can be derived by minimizing an energy function that is originally designed based on two kinds of neurons with quite different…

In this paper we develop a novel mathematical formalism for the modeling of neural information networks endowed with additional structure in the form of assignments of resources, either computational or metabolic or informational. The…

We introduce a new information-geometric structure associated with the dynamics on discrete objects such as graphs and hypergraphs. The presented setup consists of two dually flat structures built on the vertex and edge spaces,…

The space of possible behaviors complex biological systems may exhibit is unimaginably vast, and these systems often appear to be stochastic, whether due to variable noisy environmental inputs or intrinsically generated chaos. The brain is…

We propose a hierarchically modular, dynamical neural network model whose architecture minimizes a specifically designed energy function and defines its temporal characteristics. The model has an internal and an external space that are…

The Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. It is a fundamental object of study, closely tied to various problems in deep learning, including model design, optimization, and…

We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping…

We investigate recurrent neural networks with asymmetric interactions and demonstrate that the inclusion of self-couplings or sparse excitatory inter-module connections leads to the emergence of a densely connected manifold of dynamically…

The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negative eigenvalues. We observe that for wider networks,…

Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advances encompass a wide range of conceptual, mathematical, and…

Hopfield models, originally developed to study memory retrieval in neural networks, have become versatile tools for modeling diverse biological systems in which function emerges from collective dynamics. In this review, we provide a…

In this paper we apply the method of Lagrangian descriptors to explore the geometrical structures in phase space that govern the dynamics of dissipative systems. We demonstrate through many classical examples taken from the nonlinear…

In this paper a class of dynamical systems describing expectation variables exactly derived from continuous-time master equations is introduced and studied from the viewpoint of differential geometry, where such master equations consist of…

In this paper, we investigate the geometric structure of activation spaces of fully connected layers in neural networks and then show applications of this study. We propose an efficient approximation algorithm to characterize the convex…