Related papers: Introduction to Set Shaping Theory
Submodular set functions are undoubtedly among the most important building blocks of combinatorial optimization. Somewhat surprisingly, continuous counterparts of such functions have also appeared in an analytic line of research where they…
In several domains, data objects can be decomposed into sets of simpler objects. It is then natural to represent each object as the set of its components or parts. Many conventional machine learning algorithms are unable to process this…
Obtaining compositional mappings is important for the model to generalize well compositionally. To better understand when and how to encourage the model to learn such mappings, we study their uniqueness through different perspectives.…
The study of theory combination in Satisfiability Modulo Theories (SMT) involves various model theoretic properties (e.g., stable infiniteness, smoothness, etc.). We show that such properties can be partly captured by the natural density of…
Image retargeting, which resizes images to one with a prescribed aspect ratio by determining an optimal warping map, has gained substantial interest in imaging science. Despite significant advances, existing methods often fail to ensure…
Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. We…
Grokking, or delayed generalization, is an intriguing learning phenomenon where test set loss decreases sharply only after a model's training set loss has converged. This challenges conventional understanding of the training dynamics in…
Widespread use of the Internet and social networks invokes the generation of big data, which is proving to be useful in a number of applications. To deal with explosively growing amounts of data, data analytics has emerged as a critical…
Length Generalization is the essential capacity of autonomous agents to perform tasks in longer contexts than those encountered during training. To systematically study this feat, we test how well models can approximate the next token…
Sampling theory in spaces other than the space of band-limited functions has recently received considerable attention. This is in part because the band-limitedness assumption is not very realistic in many applications. In addition,…
This book provides an introduction to string field theory (SFT). String theory is usually formulated in the worldsheet formalism, which describes a single string (first-quantization). While this approach is intuitive and could be pushed far…
We introduce the Vector Fitting algorithm for the creation of reduced-order models from the sampled response of a linear time-invariant system. This data-driven approach to reduction is particularly useful when the system under modeling is…
We introduce the notion of watching systems in graphs, which is a generalization of that of identifying codes. We give some basic properties of watching systems, an upper bound on the minimum size of a watching system, and results on the…
The problem of storing a set of strings --- a string dictionary --- in compact form appears naturally in many cases. While classically it has represented a small part of the whole data to be processed (e.g., for Natural Language processing…
Set theory brought revolution to philosophy of mathematics and it can bring revolution to philosophy of physics too. All that stands in the way is the intuition that sets of physical objects cannot themselves be physical objects, which…
The purpose of this paper is twofold. First, the definition of new statistical convergence with Fibonacci sequence is given and some fundamental properties of statistical convergence are examined. Second, approximation theory worked as a…
The aim of these notes is to give recent developments in string theory. In particular, we discuss the string spectrums, compactifications, brane physics and dualities.
Despite the remarkable empirical performance of Transformers, their theoretical understanding remains elusive. Here, we consider a deep multi-head self-attention network, that is closely related to Transformers yet analytically tractable.…
We study the CPT theorem for a two-dimensional conformal field theory on an arbitrary Riemann surface. On the sphere the theorem follows from the assumption that the correlation functions have standard hermiticity properties and are…
In this paper, we propose a method for transferring feature representation to lightweight student models from larger teacher models. We mathematically define a new notion called \textit{perception coherence}. Based on this notion, we…