Related papers: Depth, Highness and DNR degrees

Recently, very deep convolutional neural networks (CNNs) have shown outstanding performance in object recognition and have also been the first choice for dense classification problems such as semantic segmentation. However, repeated…

We construct a nonrecursive set \(A\le_T\emptyset'\) and a uniformly computable family of sets \(C_0,C_1,\dots\), all bounded finite-one equivalent to \(A\), such that the corresponding \(1\)-degrees form a copy of the dense linear order…

A generic computation of a subset $A$ of $\mathbb{N}$ is a computation which correctly computes most of the bits of $A$, but which potentially does not halt on all inputs. The motivation for this concept is derived from complexity theory,…

Dense Retrieval (DR) reaches state-of-the-art results in first-stage retrieval, but little is known about the mechanisms that contribute to its success. Therefore, in this work, we conduct an interpretation study of recently proposed DR…

This work is a continuation of [13]. We study the linear disjointness between higher-order oscillating sequences and nonlinear dynamical systems. Specifically, we prove that any oscillating sequence of order $m=d+k-1$ and any simple…

Classifiers built with neural networks handle large-scale high dimensional data, such as facial images from computer vision, extremely well while traditional statistical methods often fail miserably. In this paper, we attempt to understand…

We investigate enumerability properties for classes of sets which permit recursive, lexicographically increasing approximations, or left-r.e. sets. In addition to pinpointing the complexity of left-r.e. Martin-L\"{o}f, computably, Schnorr,…

In this paper, we provide algorithms to rank, unrank, and randomly generate certain degree-restricted classes of Cayley trees. Specifically, we consider classes of trees that have a given degree sequence or a given multiset of degrees. If…

Previous work has shown that DNNs with large depth $L$ and $L_{2}$-regularization are biased towards learning low-dimensional representations of the inputs, which can be interpreted as minimizing a notion of rank $R^{(0)}(f)$ of the learned…

Drawing inspiration from a recent paper of Heuberger, Krenn, and Lipnik, we define the class of strongly k-recursive sequences. We show that every k-automatic sequence is strongly $k$-recursive, therefore k-recursive, and discuss that the…

We introduce random-kernel networks, a multilayer extension of random feature models where depth is created by deterministic kernel composition and randomness enters only in the outermost layer. We prove that deeper constructions can…

A sound and complete algorithm for nominal unification of higher-order expressions with a recursive let is described, and shown to run in non-deterministic polynomial time. We also explore specializations like nominal letrec-matching for…

Denote by $\mathbb{N}$ and $\mathbb{P}$ the set of all positive integers and prime numbers, respectively. Let $\mathbb{P}=\{p_1<p_2<\dots <p_n<\dots\}$, where $p_n$ is the $n$-th prime number. For $k\in\mathbb{N}$ we recursively define…

The tremendous recent progress in analyzing the training dynamics of overparameterized neural networks has primarily focused on wide networks and therefore does not sufficiently address the role of depth in deep learning. In this work, we…

Various powerful deep neural network architectures have made great contribution to the exciting successes of deep learning in the past two decades. Among them, deep Residual Networks (ResNets) are of particular importance because they…

On account of its many successes in inference tasks and denoising applications, Dictionary Learning (DL) and its related sparse optimization problems have garnered a lot of research interest. While most solutions have focused on single…

Our goal is to quickly find top $k$ lists of nodes with the largest degrees in large complex networks. If the adjacency list of the network is known (not often the case in complex networks), a deterministic algorithm to find a node with the…

This paper develops a general theory for first-order descent methods whose search directions are restricted to a prescribed dictionary in a reflexive Banach space. Instead of assuming that the linear span of the dictionary is dense, as in…

We study the problem of estimating the relative depth order of point pairs in a monocular image. Recent advances mainly focus on using deep convolutional neural networks (DCNNs) to learn and infer the ordinal information from multiple…

The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that the best models with large data may be deep neural…