Related papers: Space filling positionality and the Spiroformer
Using convolutional neural networks for 360images can induce sub-optimal performance due to distortions entailed by a planar projection. The distortion gets deteriorated when a rotation is applied to the 360image. Thus, many researches…
If a collection of identical particles is poured into a container, different shapes will fill to different densities. But what is the shape that fills a container as close as possible to a pre-specified, desired density? We demonstrate a…
A region of two-dimensional space has been filled randomly with large number of growing circular discs allowing only a `slight' overlapping among them just before their growth stop. More specifically, each disc grows from a nucleation…
Viewing Transformers as interacting particle systems, we describe the geometry of learned representations when the weights are not time dependent. We show that particles, representing tokens, tend to cluster toward particular limiting…
This report covers our reproduction effort of the paper 'Differentiable Spatial Planning using Transformers' by Chaplot et al. . In this paper, the problem of spatial path planning in a differentiable way is considered. They show that their…
Blending schemes based on circles provide smooth `fair' interpolations between series of points. Here we demonstrate a simple, robust set of algorithms for performing circle blends for a range of cases. An arbitrary level of G-continuity…
In this paper, we present DevFormer, a novel transformer-based architecture for addressing the complex and computationally demanding problem of hardware design optimization. Despite the demonstrated efficacy of transformers in domains…
Transformers are arguably the main workhorse in recent Natural Language Processing research. By definition a Transformer is invariant with respect to reordering of the input. However, language is inherently sequential and word order is…
Transformers can generate predictions in two approaches: 1. auto-regressively by conditioning each sequence element on the previous ones, or 2. directly produce an output sequences in parallel. While research has mostly explored upon this…
Hyperbolic geometry have shown significant potential in modeling complex structured data, particularly those with underlying tree-like and hierarchical structures. Despite the impressive performance of various hyperbolic neural networks…
Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. We…
Very recently, Window-based Transformers, which computed self-attention within non-overlapping local windows, demonstrated promising results on image classification, semantic segmentation, and object detection. However, less study has been…
Inversive geometry can be used to generate exactly self-similar space-filling sphere packings. We present a construction method in two dimensions and generalize it to search for packings in higher dimensions. We newly discover 29…
Hard sphere systems are often used to model simple fluids. The configuration spaces of hard spheres in a three-dimensional torus modulo various symmetry groups are comparatively simple, and could provide valuable information about the…
Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their…
We present filling as a new type of spatial subdivision problem that is related to covering and packing. Filling addresses the optimal placement of overlapping objects lying entirely inside an arbitrary shape so as to cover the most…
World modelling, i.e. building a representation of the rules that govern the world so as to predict its evolution, is an essential ability for any agent interacting with the physical world. Recent applications of the Transformer…
Static granular packings are model hard-sphere glass formers. The nature of glass transition has remained a hotly debated issue. We review recent experimental progresses in using granular materials to study glass transitions. We focus on…
In this work, instead of directly predicting the pixel-level segmentation masks, the problem of referring image segmentation is formulated as sequential polygon generation, and the predicted polygons can be later converted into segmentation…
Transformers are widely used deep learning architectures. Existing transformers are mostly designed for sequences (texts or time series), images or videos, and graphs. This paper proposes a novel transformer model for massive (up to a…