Related papers: Finding the Missing Data: A BERT-inspired Approach…

Transformer-based language models have taken many fields in NLP by storm. BERT and its derivatives dominate most of the existing evaluation benchmarks, including those for Word Sense Disambiguation (WSD), thanks to their ability in…

Even though BERT achieves successful performance improvements in various supervised learning tasks, applying BERT for unsupervised tasks still holds a limitation that it requires repetitive inference for computing contextual language…

The success of bidirectional encoders using masked language models, such as BERT, on numerous natural language processing tasks has prompted researchers to attempt to incorporate these pre-trained models into neural machine translation…

Empowered by deep learning, semantic communication marks a paradigm shift from transmitting raw data to conveying task-relevant meaning, enabling more efficient and intelligent wireless systems. In this study, we explore a deep…

Deep learning (DL)-based channel state information (CSI) feedback has shown promising potential to improve spectrum efficiency in massive MIMO systems. However, practical DL approaches require a sizeable CSI dataset for each scenario, and…

In frequency division duplex (FDD) systems, acquiring channel state information (CSI) at the base station (BS) traditionally relies on limited feedback from mobile terminals (MTs). However, the accuracy of channel reconstruction from…

Artificial Intelligence and Machine Learning have witnessed rapid, significant improvements in Natural Language Processing (NLP) tasks. Utilizing Deep Learning, researchers have taken advantage of repository comments in Software Engineering…

Modern power grids are undergoing significant changes driven by information and communication technologies (ICTs), and evolving into smart grids with higher efficiency and lower operation cost. Using ICTs, however, comes with an inevitable…

Massive MIMO systems can achieve high spectrum and energy efficiency in downlink (DL) based on accurate estimate of channel state information (CSI). Existing works have developed learning-based DL CSI estimation that lowers uplink feedback…

Limited computational budgets often prevent transformers from being used in production and from having their high accuracy utilized. TinyBERT addresses the computational efficiency by self-distilling BERT into a smaller transformer…

Accurate and robust wireless localization is a key enabler for a wide range of mobile computing applications. Fingerprint-based localization using channel state information (CSI) has attracted significant attention due to its high accuracy…

This paper presents a novel hybrid deep learning framework designed to enhance the robustness of CSI-based Human Activity Recognition (HAR) within bandwidth-constrained Wi-Fi sensing environments. The core of our proposed methodology is a…

The bidirectional encoder representations from transformers (BERT) model has recently advanced the state-of-the-art in passage re-ranking. In this paper, we analyze the results produced by a fine-tuned BERT model to better understand the…

Massive multiple-input multiple-output (MIMO) technology is a key enabler of modern wireless communication systems, which demand accurate downlink channel state information (CSI) for optimal performance. Although deep learning (DL) has…

The era of ubiquitous, affordable wireless connectivity has opened doors to countless practical applications. In this context, ambient backscatter communication (AmBC) stands out, utilizing passive tags to establish connections with readers…

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing…

Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory.…

It is known that a deep neural network model pre-trained with large-scale data greatly improves the accuracy of various tasks, especially when there are resource constraints. However, the information needed to solve a given task can vary,…

Pre-trained language models like BERT have achieved great success in a wide variety of NLP tasks, while the superior performance comes with high demand in computational resources, which hinders the application in low-latency IR systems. We…

Recently developed large pre-trained language models, e.g., BERT, have achieved remarkable performance in many downstream natural language processing applications. These pre-trained language models often contain hundreds of millions of…