Related papers: Shape Preserving Tree Transducers
We investigate how embedding dimension affects the emergence of an internal "world model" in a transformer trained with reinforcement learning to perform bubble-sort-style adjacent swaps. Models achieve high accuracy even with very small…
The purpose of this article is twofold. On one hand, we reveal the equivalence of shift of finite type between a one-sided shift $X$ and its associated hom tree-shift $\mathcal{T}_{X}$, as well as the equivalence in the sofic shift. On the…
Cell biomechanics involve a great number of complex phenomena that are fundamental to the evolution of life itself and other associated processes, ranging from the very early stages of embryo-genesis to the maintenance of damaged structures…
When trained on language data, do transformers learn some arbitrary computation that utilizes the full capacity of the architecture or do they learn a simpler, tree-like computation, hypothesized to underlie compositional meaning systems…
We answer an open question in the theory of transducer degrees on the existence of a diamond structure in the transducer hierarchy. Transducer degrees are the equivalence classes formed by word transformations which can be realized by a…
Tensor dimensionality reduction is one of the fundamental tools for modern data science. To address the high computational overhead, fiber-wise sampled subtensors that preserve the original tensor rank are often used in designing efficient…
We characterise the respective semigroups of mappings that preserve, or that preserve or reverse orientation of a finite cycle, in terms of their actions on oriented triples and oriented quadruples. This leads to a proof that the latter…
Model order reduction provides low-complexity high-fidelity surrogate models that allow rapid and accurate solutions of parametric differential equations. The development of reduced order models for parametric \emph{nonlinear} Hamiltonian…
We study three different kinds of embeddings of tree patterns: weakly-injective, ancestor-preserving, and lca-preserving. While each of them is often referred to as injective embedding, they form a proper hierarchy and their computational…
Scattering transforms are non-trainable deep convolutional architectures that exploit the multi-scale resolution of a wavelet filter bank to obtain an appropriate representation of data. More importantly, they are proven invariant to…
We introduce a notion of weak definability of first order structures, show that various classification-theoretic properties are or are not preserved under it, and that the properties which are preserved can also be characterized in terms of…
Shape preservation behavior of a spline consists of criterial conditions for preserving convexity, inflection, collinearity, torsion and coplanarity shapes of data polgonal arc. We present our results which acts as an improvement in the…
When doing representation learning on data that lives on a known non-trivial manifold embedded in high dimensional space, it is natural to desire the encoder to be homeomorphic when restricted to the manifold, so that it is bijective and…
The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by…
Deep learning has been used as a powerful tool for various tasks in computer vision, such as image segmentation, object recognition and data generation. A key part of end-to-end training is designing the appropriate encoder to extract…
In deep learning theory, the covariance matrix of the representations serves as a proxy to examine the network's trainability. Motivated by the success of Transformers, we study the covariance matrix of a modified Softmax-based attention…
We consider a transmission problem on a polygonal partition for the two-dimensional conductivity equation. For suitable classes of partitions we establish the exact behaviour of the gradient of solutions in a neighbourhood of the vertexes…
Persistent homology analysis provides means to capture the connectivity structure of data sets in various dimensions. On the mathematical level, by defining a metric between the objects that persistence attaches to data sets, we can…
Computing a morph between two drawings of a graph is a classical problem in computational geometry and graph drawing. While this problem has been widely studied in the context of planar graphs, very little is known about the existence of…
Evolutionary forms, as well as exterior forms, are skew-symmetric differential forms. But in contrast to the exterior forms, the basis of evolutionary forms is deforming manifolds (with unclosed metric forms). Such forms possess a…