Related papers: Non Commutative Convolutional Signal Models in Neu…

We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region boundaries. These regions emerge during training and become…

Network filtering is an important form of dimension reduction to isolate the core constituents of large and interconnected complex systems. We introduce a new technique to filter large dimensional networks arising out of dynamical behavior…

Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically,…

We fill the two main remaining gaps in the full classification of non-degenerate planar traveling waves of scalar balance laws from the point of view of spectral and nonlinear stability/instability under smooth perturbations. On one hand we…

In this paper, we consider nonsubsampled graph filter banks (NSGFBs) to process data on a graph in a distributed manner. Given an analysis filter bank with small bandwidth, we propose algebraic and optimization methods of constructing…

Nature inspired neuromorphic architectures are being explored as an alternative to imminent limitations of conventional complementary metal-oxide semiconductor (CMOS) architectures. Utilization of such architectures for practical…

Convolutional neural networks have achieved great success in various vision tasks; however, they incur heavy resource costs. By using deeper and wider networks, network accuracy can be improved rapidly. However, in an environment with…

Inverse problems are key issues in several scientific areas, including signal processing and medical imaging. Since inverse problems typically suffer from instability with respect to data perturbations, a variety of regularization…

Augmenting transformation knowledge onto a convolutional neural network's weights has often yielded significant improvements in performance. For rotational transformation augmentation, an important element to recent approaches has been the…

The use of covariance kernels is ubiquitous in the field of spatial statistics. Kernels allow data to be mapped into high-dimensional feature spaces and can thus extend simple linear additive methods to nonlinear methods with higher order…

We propose the introduction of nonlinear operation into the feature generation process in convolutional neural networks. This nonlinearity can be implemented in various ways. First we discuss the use of nonlinearities in the process of data…

Even though image signals are typically defined on a regular two-dimensional grid, there also exist many scenarios where this is not the case and the amplitude of the image signal only is available for a non-regular subset of pixel…

Applying feature dependent network weights have been proved to be effective in many fields. However, in practice, restricted by the enormous size of model parameters and memory footprints, scalable and versatile dynamic convolutions with…

This paper studies whether numerically preserving monotonic properties can offer modelling advantages in data assimilation, particularly when the signal or data is a realization of a stochastic partial differential equation (SPDE) or…

Convolutional neural networks (CNNs) are being applied to an increasing number of problems and fields due to their superior performance in classification and regression tasks. Since two of the key operations that CNNs implement are…

Any representation of data involves arbitrary investigator choices. Because those choices are external to the data-generating process, each choice leads to an exact symmetry, corresponding to the group of transformations that takes one…

We discuss the effects of non-linear structure formation on the signature of acoustic oscillations in the late-time galaxy distribution. We argue that the dominant non-linear effect is the differential motion of pairs of tracers separated…

In this paper, we give a new covariation spectral representation of some non stationary symmetric $\alpha$-stable processes (S$\alpha$S). This representation is based on a weaker covariation pseudo additivity condition which is more general…

Data assimilation methodologies are designed to incorporate noisy observations of a physical system into an underlying model in order to infer the properties of the state of the system. Filters refer to a class of data assimilation…

Recently, the stability of graph filters has been studied as one of the key theoretical properties driving the highly successful graph convolutional neural networks (GCNs). The stability of a graph filter characterizes the effect of…