Related papers: Designing a Linearized Potential Function in Neura…

The effectiveness of non-parametric, kernel-based methods for function estimation comes at the price of high computational complexity, which hinders their applicability in adaptive, model-based control. Motivated by approximation techniques…

The goal of this thesis is to develop the optimisation and generalisation theoretic foundations of learning in artificial neural networks. On optimisation, a new theoretical framework is proposed for deriving architecture-dependent…

We propose tensorial neural networks (TNNs), a generalization of existing neural networks by extending tensor operations on low order operands to those on high order ones. The problem of parameter learning is challenging, as it corresponds…

A maximum entropy framework based on Tsallis entropy is proposed to depict long tail behavior of queue lengths in broadband networks. Queue length expression as measured in terms of number of packets involves Hurwitz-zeta function. When the…

We study the emergence of sparse representations in neural networks. We show that in unsupervised models with regularization, the emergence of sparsity is the result of the input data samples being distributed along highly non-linear or…

Deep neural networks (DNNs) have become increasingly important due to their excellent empirical performance on a wide range of problems. However, regularization is generally achieved by indirect means, largely due to the complex set of…

Theoretical neuroscientists often try to understand how the structure of a neural network relates to its function by focusing on structural features that would either follow from optimization or occur consistently across possible…

We investigate the cumulative Tsallis entropy, an information measure recently introduced as a cumulative version of the classical Tsallis differential entropy, which is itself a generalization of the Boltzmann-Gibbs statistics. This…

Pinsker's and Fannes' type bounds on the Tsallis relative entropy are derived. The monotonicity property of the quantum $f$-divergence is used for its estimating from below. For order $\alpha\in(0,1)$, a family of lower bounds of Pinsker…

We develop a geometric realization of a broad class of generalized black hole entropy functionals by establishing their direct correspondence with the Misner$-$Sharp quasilocal mass and the Wald Noether$-$charge entropy in scalar$-$tensor…

Neural networks have attracted a lot of attention due to its success in applications such as natural language processing and computer vision. For large scale data, due to the tremendous number of parameters in neural networks, overfitting…

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer neural network with random iid inputs. We study the generalization performances of standard classifiers in the…

Deep neural networks have emerged as powerful tools for learning operators defined over infinite-dimensional function spaces. However, existing theories frequently encounter difficulties related to dimensionality and limited…

Modern generative modelling systems are increasingly improved by expanding model capacity, training data, and computational resources. While empirical studies have documented such scaling behaviour across architectures including generative…

We have discussed the Tsallis entropy in finite $N$-unit nonextensive systems, by using the multivariate $q$-Gaussian probability distribution functions (PDFs) derived by the maximum entropy methods with the normal average and the…

The scope of this work is the constraint-based synthesis of termination arguments for the restricted class of programs called linear lasso programs. A termination argument consists of a ranking function as well as a set of supporting…

Various iterative reconstruction algorithms for inverse problems can be unfolded as neural networks. Empirically, this approach has often led to improved results, but theoretical guarantees are still scarce. While some progress on…

Our universe is homogeneous and isotropic, and its perturbations obey translation and rotation symmetry. In this work we develop Translation and Rotation Equivariant Normalizing Flow (TRENF), a generative Normalizing Flow (NF) model which…

The paper is devoted to discretization of integral norms of functions from a given finite dimensional subspace. This problem is very important in applications but there is no systematic study of it. We present here a new technique, which…

Cross-entropy loss and focal loss are the most common choices when training deep neural networks for classification problems. Generally speaking, however, a good loss function can take on much more flexible forms, and should be tailored for…