Related papers: Neural Collapse with Cross-Entropy Loss

An implicit Euler finite-volume scheme for a nonlocal cross-diffusion system on the one-dimensional torus, arising in population dynamics, is proposed and analyzed. The kernels are assumed to be in detailed balance and satisfy a weak…

We determine the universal part of pseudoentropy for small shape deformations of spherical entangling surfaces in the context of de Sitter/conformal field theory (dS/CFT) correspondence. The leading correction at quadratic order in the…

We investigate the entanglement entropy of a two-dimensional disordered system holographically. In particular, we study the evolution of the entanglement entropy along renormalization group flows for a conformal theory at the UV fixed…

We study the loss surface of a feed-forward neural network with ReLU non-linearities, regularized with weight decay. We show that the regularized loss function is piecewise strongly convex on an important open set which contains, under some…

Neural scaling laws govern the prediction power-law improvement of test loss with respect to model capacity ($N$), datasize ($D$), and compute ($C$). However, existing theoretical explanations often rely on specific architectures or complex…

Multi-view learning accomplishes the task objectives of classification by leverag-ing the relationships between different views of the same object. Most existing methods usually focus on consistency and complementarity between multiple…

One of the central challenges in modern machine learning is understanding how neural networks generalize knowledge learned from training data to unseen test data. While numerous empirical techniques have been proposed to improve…

Extreme neural network sparsification (90% activation reduction) presents a critical challenge for mechanistic interpretability: understanding whether interpretable features survive aggressive compression. This work investigates feature…

This paper presents a neural network filter method based on contraction operators to address model collapse in recursive training of generative models. Unlike \cite{xu2024probabilistic}, which requires superlinear sample growth…

One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…

Brain function and connectivity is a pressing mystery in medicine related to many diseases. Neural connectomes have been studied as graphs with graph theory methods including topological methods. Work has started on hypergraph models and…

Catastrophic failures are complete and sudden collapses in the activity of large networks such as economics, electrical power grids and computer networks, which typically require a manual recovery process. Here we experimentally show that…

A one-dimensional cross-diffusion system modeling the transport of vesicles in neurites is analyzed. The equations are coupled via nonlinear Robin boundary conditions to ordinary differential equations for the number of vesicles in the…

The minimum-norm interpolator (MNI) framework has recently attracted considerable attention as a tool for understanding generalization in overparameterized models, such as neural networks. In this work, we study the MNI under a $2$-uniform…

Cross-entropy loss is the standard metric used to train classification models in deep learning and gradient boosting. It is well-known that this loss function fails to account for similarities between the different values of the target. We…

We extend the empirical results from the structural equation model (SEM) published in the paper Assortment Planning for Retail Buying, Retail Store Operations, and Firm Performance [1] by implementing the directed acyclic graph as a causal…

We consider the optimization problem associated with fitting two-layer ReLU networks with $k$ hidden neurons, where labels are assumed to be generated by a (teacher) neural network. We leverage the rich symmetry exhibited by such models to…

We study a number of (3+1)- and (2+1)-dimensional defect and boundary conformal field theories holographically dual to supergravity theories. In all cases the defects or boundaries are planar, and the defects are codimension-one. Using…

In this paper we develop a framework to study observability for uniform hypergraphs. Hypergraphs, being extensions of graphs, allow edges to connect multiple nodes and unambiguously represent multi-way relationships which are ubiquitous in…

This study introduces a novel approach to ensure the existence and uniqueness of optimal parameters in neural networks. The paper details how a recurrent neural networks (RNN) can be transformed into a contraction in a domain where its…