Related papers: BERTops: Studying BERT Representations under a Top…

Topological data analysis uses tools from topology -- the mathematical area that studies shapes -- to create representations of data. In particular, in persistent homology, one studies one-parameter families of spaces associated with data,…

In recent years, the introduction of the Transformer models sparked a revolution in natural language processing (NLP). BERT was one of the first text encoders using only the attention mechanism without any recurrent parts to achieve…

Qualitative methods such as the linear sampling method and the factorization method reconstruct acoustic scatterers through sampling indicators. In practice, these indicators are gray-scale fields on a prescribed sampling window and a…

Balancing performance and interpretability in multivariate time series classification is a significant challenge due to data complexity and high dimensionality. This paper introduces PHeatPruner, a method integrating persistent homology and…

We study the problem of learning representations with controllable connectivity properties. This is beneficial in situations when the imposed structure can be leveraged upstream. In particular, we control the connectivity of an…

For nearly three decades, spatial games have produced a wealth of insights to the study of behavior and its relation to population structure. However, as different rules and factors are added or altered, the dynamics of spatial models often…

We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks. Sentence-level…

BERT (Bidirectional Encoder Representations from Transformers) and related pre-trained Transformers have provided large gains across many language understanding tasks, achieving a new state-of-the-art (SOTA). BERT is pre-trained on two…

Persistent homology is a popular technique in topological data analysis that tracks the lifespans of homological features in a nested sequence of spaces. This data is typically presented in a multi-set called a persistence diagram or a…

In this paper, we study the persistent homology of the offset filtration of algebraic varieties. We prove the algebraicity of two quantities central to the computation of persistent homology. Moreover, we connect persistent homology and…

Persistent homology encodes the evolution of homological features of a multifiltered cell complex in the form of a multigraded module over a polynomial ring, called a multiparameter persistence module, and quantifies it through invariants…

This study introduces a significant architectural advancement in feature fusion for lyrical content classification by integrating auxiliary structural features directly into the self-attention mechanism of a pre-trained Transformer. I…

Recent methods leverage a hypernet to handle the performance-fairness trade-offs in federated learning. This hypernet maps the clients' preferences between model performance and fairness to preference-specifc models on the trade-off curve,…

Topological data analysis can extract effective information from higher-dimensional data. Its mathematical basis is persistent homology. The persistent homology can calculate topological features at different spatiotemporal scales of the…

Transfer learning (TL) for high-dimensional regression (HDR) is an important problem in machine learning, particularly when dealing with limited sample size in the target task. However, there currently lacks a method to quantify the…

Using software UDEC to simulate the instability failure process of slope under seismic load, studing the dynamic response of slope failure, obtaining the deformation characteristics and displacement cloud map of slope, then analyzing the…

Transformer-based methods have achieved state-of-the-art performance in time series forecasting (TSF) by capturing positional and semantic topological relationships among input tokens. However, it remains unclear whether existing…

Topological Data Analysis (TDA) is a rising field of computational topology in which the topological structure of a data set can be observed by persistent homology. By considering a sequence of sublevel sets, one obtains a filtration that…

Pre-training models are an important tool in Natural Language Processing (NLP), while the BERT model is a classic pre-training model whose structure has been widely adopted by followers. It was even chosen as the reference model for the…

Fine-tuned Bidirectional Encoder Representations from Transformers (BERT)-based sequence classification models have proven to be effective for detecting Alzheimer's Disease (AD) from transcripts of human speech. However, previous research…