Related papers: xPerT: Extended Persistence Transformer
This paper introduces persistent homology, which is a powerful tool to characterize the shape of data using the mathematical concept of topology. We explain the fundamental idea of persistent homology from scratch using some examples. We…
This paper explores the novel deep learning Transformers architectures for high-frequency Bitcoin-USDT log-return forecasting and compares them to the traditional Long Short-Term Memory models. A hybrid Transformer model, called…
Topological Data Analysis (TDA) is a rising field of computational topology in which the topological structure of a data set can be observed by persistent homology. By considering a sequence of sublevel sets, one obtains a filtration that…
In topological data analysis, persistent homology characterizes robust topological features in data and it has a summary representation, called a persistence diagram. Statistical research for persistence diagrams have been actively…
Transformers have become keystone models in natural language processing over the past decade. They have achieved great popularity in deep learning applications, but the increasing sizes of the parameter spaces required by transformer models…
Modern deep neural networks have achieved great successes in medical image analysis. However, the features captured by convolutional neural networks (CNNs) or Transformers tend to be optimized for pixel intensities and neglect key…
Persistent homology is a popular tool in Topological Data Analysis. It provides numerical characteristics of data sets which reflect global geometric properties. In order to be useful in practice, for example for feature generation in…
Graph Transformer is gaining increasing attention in the field of machine learning and has demonstrated state-of-the-art performance on benchmarks for graph representation learning. However, as current implementations of Graph Transformer…
Time series forecasting remains a critical challenge across various domains, often complicated by high-dimensional data and long-term dependencies. This paper presents a novel transformer architecture for time series forecasting,…
Recently, large-scale transformer-based models have been proven to be effective over various tasks across many domains. Nevertheless, applying them in industrial production requires tedious and heavy works to reduce inference costs. To fill…
Pretraining plays a pivotal role in acquiring generalized knowledge from large-scale data, achieving remarkable successes as evidenced by large models in CV and NLP. However, progress in the graph domain remains limited due to fundamental…
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling. We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a…
The persistent homology of a stationary point process on ${\bf R}^N$ is studied in this paper. As a generalization of continuum percolation theory, we study higher dimensional topological features of the point process such as loops,…
Persistent homology is a fundamental tool in Topological Data Analysis. The associated algebraic structure is the persistence module, a sequence of vector spaces connected by linear maps. Persistence modules admit a complete and…
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges belong to the same types, making them…
Topological data analysis and its main method, persistent homology, provide a toolkit for computing topological information of high-dimensional and noisy data sets. Kernels for one-parameter persistent homology have been established to…
This paper describes a memory-efficient transformer model designed to drive a reduction in memory usage and execution time by substantial orders of magnitude without impairing the model's performance near that of the original model.…
Natural language processing (NLP) made an impressive jump with the introduction of Transformers. ChatGPT is one of the most famous examples, changing the perception of the possibilities of AI even outside the research community. However,…
Convolution neural networks (CNNs) have succeeded in compressive image sensing. However, due to the inductive bias of locality and weight sharing, the convolution operations demonstrate the intrinsic limitations in modeling the long-range…
Transformers have demonstrated great success in numerous domains including natural language processing and bioinformatics. This success stems from the use of the attention mechanism by these models in order to represent and propagate…