Related papers: An Invitation to the Euler Characteristic Transfor…
Transformer models have advanced the state of the art in many Natural Language Processing (NLP) tasks. In this paper, we present a new Transformer architecture, Extended Transformer Construction (ETC), that addresses two key challenges of…
To capture user preference, transformer models have been widely applied to model sequential user behavior data. The core of transformer architecture lies in the self-attention mechanism, which computes the pairwise attention scores in a…
We geometrically construct a homology theory that generalizes the Euler characteristic mod 2 to objects in the unoriented cobordism ring N_*(X) of a topological space X. This homology theory Eh_* has coefficients Z/2 in every nonnegative…
The Euler characteristic of a cell complex is often thought of as the alternating sum of the number of cells of each dimension. When the complex is infinite, the sum diverges. Nevertheless, it can sometimes be evaluated; in particular, this…
In this paper we obtain some explicit expressions for the Euler characteristic of a rank n coherent sheaf F on P^N and of its twists F(t) as polynomials in the Chern classes c_i(F), also giving algorithms for the computation. The employed…
Recently, large-scale transformer-based models have been proven to be effective over various tasks across many domains. Nevertheless, applying them in industrial production requires tedious and heavy works to reduce inference costs. To fill…
Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. We…
Everyone knows that the Euler characteristic of a combinatorial manifold is given by the alternating sum of its numbers of simplices. It is shown that there are other linear combinations of the numbers of simplices which are combinatorial…
We introduce EELBERT, an approach for compression of transformer-based models (e.g., BERT), with minimal impact on the accuracy of downstream tasks. This is achieved by replacing the input embedding layer of the model with dynamic, i.e.…
We consider different generalizations of the Euler formula and discuss the properties of the associated trigonometric functions. The problem is analyzed from different points of view and it is shown that it can be formulated in a natural…
In this short paper, we give two proofs that the Euler characteristic is multiplicative, for fiber sequences of finitely dominated spaces. This is equivalent to proving that the Becker-Gottlieb transfer is functorial on $\pi_0$.
Transformer models are permutation equivariant. To supply the order and type information of the input tokens, position and segment embeddings are usually added to the input. Recent works proposed variations of positional encodings with…
Graph transformers typically embed every node in a single Euclidean space, blurring heterogeneous topologies. We prepend a lightweight Riemannian mixture-of-experts layer that routes each node to various kinds of manifold, mixture of…
We propose a definition of an Euler characteristic for unbounded chain complexes by taking the (usual) Euler characteristics of successively longer parts of the complex, weighted inversely proportional to the length, and passing to the…
The Euler characteristic $\chi =|V|-|E|$ and the total length $\mathcal{L}$ are the most important topological and geometrical characteristics of a metric graph. Here, $|V|$ and $|E|$ denote the number of vertices and edges of a graph. The…
The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification is to ensure that the extracted measurements remain invariant to transformations that…
We introduce a rotation-invariant representation of planar shapes. In particular, this representation encodes shapes as vectors such that the Euclidean distance between them serves as a valid shape distance. For standardized, star-shaped…
Euler's elastica constitute an appealing variational image inpainting model. It minimises an energy that involves the total variation as well as the level line curvature. These components are transparent and make it attractive for shape…
In the quest for dynamic multimodal probing of a material's structure and functionality, it is critical to be able to quantify the chemical state on the atomic and nanoscale using element specific electronic and structurally sensitive tools…
The task of Named Entity Recognition (NER) is an important component of many natural language processing systems, such as relation extraction and knowledge graph construction. In this work, we present a simple and effective approach for…