Related papers: ReLU Code Space: A Basis for Rating Network Qualit…

Recent findings suggest that consecutive layers of neural networks with the ReLU activation function \emph{fold} the input space during the learning process. While many works hint at this phenomenon, an approach to quantify the folding was…

Recent findings suggest that the consecutive layers of ReLU neural networks can be understood geometrically as space folding transformations of the input space, revealing patterns of self-similarity. In this paper, we present the first…

Finding best architectures of learning machines, such as deep neural networks, is a well-known technical and theoretical challenge. Recent work by Mellor et al (2021) showed that there may exist correlations between the accuracies of…

The weighted-Hamming metric generalizes the Hamming metric by assigning different weights to blocks of coordinates. It is well-suited for applications such as coding over independent parallel channels, each of which has a different level of…

A ReLU neural network leads to a finite polyhedral decomposition of input space and a corresponding finite dual graph. We show that while this dual graph is a coarse quantization of input space, it is sufficiently robust that it can be…

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit…

Parameter space is not function space for neural network architectures. This fact, investigated as early as the 1990s under terms such as ``reverse engineering," or ``parameter identifiability", has led to the natural question of parameter…

We present a powerful new loss function and training scheme for learning binary hash codes with any differentiable model and similarity function. Our loss function improves over prior methods by using log likelihood loss on top of an…

Random network coding recently attracts attention as a technique to disseminate information in a network. This paper considers a non-coherent multi-shot network, where the unknown and time-variant network is used several times. In order to…

We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these standard symmetries, we introduce a framework based on…

As modern deep learning architectures grow in complexity, representational ambiguity emerges as a critical barrier to their interpretability and reliable merging. For ReLU networks, identical functional mappings can be achieved through…

We introduce a novel approach for analyzing the training dynamics of ReLU networks by examining the characteristic activation boundaries of individual ReLU neurons. Our proposed analysis reveals a critical instability in common neural…

It has been widely observed that there exists a fundamental trade-off between the minimum (Hamming) distance properties and the iterative decoding convergence behavior of turbo-like codes. While capacity achieving code ensembles typically…

Neural networks have been widely used to solve complex real-world problems. Due to the complicate, nonlinear, non-convex nature of neural networks, formal safety guarantees for the output behaviors of neural networks will be crucial for…

Reed Muller (RM) codes are known for their good minimum distance. One can use their structure to construct polar-like codes with good distance properties by choosing the information set as the rows of the polarization matrix with the…

An equidistant code is a code in the Hamming space such that two distinct codewords have the same Hamming distance. This paper investigates the bounds for equidistant codes in Hamming spaces.

We present a method for computing exact reachable sets for deep neural networks with rectified linear unit (ReLU) activation. Our method is well-suited for use in rigorous safety analysis of robotic perception and control systems with deep…

In this letter we consider the ensemble of codes formed by the serial concatenation of a Hamming code and two accumulate codes. We show that this ensemble is asymptotically good, in the sense that most codes in the ensemble have minimum…

We introduce the sum-rank metric analogue of Reed--Muller codes, which we called linearized Reed--Muller codes, using multivariate Ore polynomials. We study the parameters of these codes, compute their dimension and give a lower bound for…

We investigate the asymptotic density of error-correcting codes with good distance properties and prescribed linearity degree, including sublinear and nonlinear codes. We focus on the general setting of finite translation-invariant metric…