Related papers: Nonstandard principles for generalized functions

We study theoretical guarantees for solving linear systems in-context using a linear transformer architecture. For in-domain generalization, we provide neural scaling laws that bound the generalization error in terms of the number of tasks…

In this paper we give a brief review of semiparametric theory, using as a running example the common problem of estimating an average causal effect. Semiparametric models allow at least part of the data-generating process to be unspecified…

Networks are a powerful tool to model complex systems, and the definition of many Graph Neural Networks (GNN), Deep Learning algorithms that can handle networks, has opened a new way to approach many real-world problems that would be hardly…

Detecting structure in noisy time series is a difficult task. One intuitive feature is the notion of trend. From theoretical hints and using simulated time series, we empirically investigate the efficiency of standard recurrent neural…

Understanding the behavior of a trained network and finding explanations for its outputs is important for improving the network's performance and generalization ability, and for ensuring trust in automated systems. Several approaches have…

In order to apply thermodynamics to systems in which entropy is not extensive, it has become customary to define generalized entropies. While this approach has been effective, it is not the only possible approach. We suggest that some…

A quasi-product on the normed space is defined. In addition, the notions of the eigenvectors of a linear operator can be extended for the nonlinear operator. Based on the quasi-product and the generalized eigenvectors, the spectral theorems…

We suggest to include the density of electron charge explicitly in the electron potential of density functional theory, rather than implicitly via exchange-correlation functionals. The advantages of the approach are conceptual and…

We consider nonlinear networks as perturbations of linear ones. Based on this approach, we present novel generalization bounds that become non-vacuous for networks that are close to being linear. The main advantage over the previous works…

We will give an elementary nonstandard proof that the family of generalized blancmange functions are nowhere differentiable. The proof follows from the intuitive characterization of differentiability at a point as almost $\delta$ affine…

Selection of 25 examples from extensive nontrivial families for different types of nonlinear PDEs and their formal general solutions are given. The main goal here is to show on examples the types of solvable PDEs and what their general…

Deep neural networks (DNNs) have become increasingly important due to their excellent empirical performance on a wide range of problems. However, regularization is generally achieved by indirect means, largely due to the complex set of…

Gaussian processes have become a popular tool for nonparametric regression because of their flexibility and uncertainty quantification. However, they often use stationary kernels, which limit the expressiveness of the model and may be…

Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the…

In this paper we investigate maximum principles for functionals defined on solutions to special partial differential equations of elliptic type, extending results by Payne and Philippin. We apply such maximum principles to investigate one…

Based on the matrix expression of general nonlinear numerical analogues presented by the present author, this paper proposes a novel philosophy of nonlinear computation and analysis. The nonlinear problems are considered an ill-posed linear…

Deep neural networks are widely used for nonlinear function approximation with applications ranging from computer vision to control. Although these networks involve the composition of simple arithmetic operations, it can be very challenging…

At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data. While over-parameterized models based on neural networks are now ubiquitous in machine learning applications, our…

Graph Neural Networks (GNNs) have been extensively used in various real-world applications. However, the predictive uncertainty of GNNs stemming from diverse sources such as inherent randomness in data and model training errors can lead to…

Graph Neural Networks (GNNs) have received a lot of interest in the recent times. From the early spectral architectures that could only operate on undirected graphs per a transductive learning paradigm to the current state of the art…