English
Related papers

Related papers: Designing Pre-training Datasets from Unlabeled Dat…

200 papers

Transformers have the capacity to act as supervised learning algorithms: by properly encoding a set of labeled training ("in-context") examples and an unlabeled test example into an input sequence of vectors of the same dimension, the…

Machine Learning · Computer Science 2024-12-16 Spencer Frei , Gal Vardi

Recently, there has been a vast interest in self-supervised learning (SSL) where the model is pre-trained on large scale unlabeled data and then fine-tuned on a small labeled dataset. The common wisdom is that SSL helps resource-limited…

Audio and Speech Processing · Electrical Eng. & Systems 2021-07-13 Chengyi Wang , Yu Wu , Shujie Liu , Jinyu Li , Yao Qian , Kenichi Kumatani , Furu Wei

Self-supervised learning (SSL) offers a promising approach for learning electroencephalography (EEG) representations from unlabeled data, reducing the need for expensive annotations for clinical applications like sleep staging and seizure…

While analytics of sleep electroencephalography (EEG) holds certain advantages over other methods in clinical applications, high variability across subjects poses a significant challenge when it comes to deploying machine learning models…

Machine Learning · Computer Science 2023-10-05 Manoj Vishwanath , Steven Cao , Nikil Dutt , Amir M. Rahmani , Miranda M. Lim , Hung Cao

Timely diagnosis is important for saving the life of epileptic patients. In past few years, a lot of treatments are available for epilepsy. These treatments require use of anti-seizure drugs but are not effective in controlling frequency of…

Machine Learning · Computer Science 2021-11-08 Shivam Gupta , Jyoti Meena , O. P Gupta

Epilepsy is a prevalent neurological disorder characterized by recurrent and unpredictable seizures, necessitating accurate prediction for effective management and patient care. Application of machine learning (ML) on electroencephalogram…

Signal Processing · Electrical Eng. & Systems 2023-08-11 Md. Simul Hasan Talukder , Rejwan Bin Sulaiman

Self-training and unsupervised pre-training have emerged as effective approaches to improve speech recognition systems using unlabeled data. However, it is not clear whether they learn similar patterns or if they can be effectively…

Transductive learning is a supervised machine learning task in which, unlike in traditional inductive learning, the unlabelled data that require labelling are a finite set and are available at training time. Similarly to inductive learning…

Machine Learning · Computer Science 2025-07-31 Lorenzo Volpi , Alejandro Moreo , Fabrizio Sebastiani

Deep neural networks (DNNs) have been shown to over-fit a dataset when being trained with noisy labels for a long enough time. To overcome this problem, we present a simple and effective method self-ensemble label filtering (SELF) to…

Computer Vision and Pattern Recognition · Computer Science 2019-10-07 Duc Tam Nguyen , Chaithanya Kumar Mummadi , Thi Phuong Nhung Ngo , Thi Hoai Phuong Nguyen , Laura Beggel , Thomas Brox

Electroencephalography (EEG) decoding is a challenging task due to the limited availability of labelled data. While transfer learning is a promising technique to address this challenge, it assumes that transferable data domains and task are…

Signal Processing · Electrical Eng. & Systems 2023-08-07 Bruno Aristimunha , Raphael Y. de Camargo , Walter H. Lopez Pinaya , Sylvain Chevallier , Alexandre Gramfort , Cedric Rommel

We exploit a self-supervised deep multi-task learning framework for electrocardiogram (ECG) -based emotion recognition. The proposed solution consists of two stages of learning a) learning ECG representations and b) learning to classify…

Signal Processing · Electrical Eng. & Systems 2020-08-11 Pritam Sarkar , Ali Etemad

Silent speech decoding, which performs unvocalized human speech recognition from electroencephalography/electromyography (EEG/EMG), increases accessibility for speech-impaired humans. However, data collection is difficult and performed…

Quantitative Methods · Quantitative Biology 2025-06-18 Masakazu Inoue , Motoshige Sato , Kenichi Tomeoka , Nathania Nah , Eri Hatakeyama , Kai Arulkumaran , Ilya Horiguchi , Shuntaro Sasai

In this study, we demonstrate the application of a hybrid Vision Transformer (ViT) model, pretrained on ImageNet, on an electroencephalogram (EEG) regression task. Despite being originally trained for image classification tasks, when…

Computer Vision and Pattern Recognition · Computer Science 2023-08-02 Ruiqi Yang , Eric Modesitt

Deep neural networks for time series must capture complex temporal patterns, to effectively represent dynamic data. Self- and semi-supervised learning methods show promising results in pre-training large models, which -- when finetuned for…

Machine Learning · Computer Science 2025-08-15 Yuhan Xie , William Cappelletti , Mahsa Shoaran , Pascal Frossard

The electroencephalogram (EEG) is one of the most precious technologies to understand the happenings inside our brain and further understand our body's happenings. Automatic prediction of oncoming seizures using the EEG signals helps the…

Signal Processing · Electrical Eng. & Systems 2022-11-08 Abhijeet Bhattacharya

Scalp electroencephalography (EEG) and intracranial EEG (iEEG) are vital for epilepsy diagnosis and treatment. Their unified analysis offers the potential to harness the complementary strengths of each modality but is challenging due to…

Neurons and Cognition · Quantitative Biology 2025-06-23 Runkai Zhang , Hua Yu , John Q. Gan , Haixian Wang

Electroencephalography (EEG) allows for source measurement of electrical brain activity. Particularly for inverse localization, the electrode positions on the scalp need to be known. Often, systems such as optical digitizing scanners are…

Computer Vision and Pattern Recognition · Computer Science 2019-08-13 Nils Gessert , Martin Gromniak , Marcel Bengs , Lars Matthäus , Alexander Schlaefer

Electroencephalography has been validated as an effective technique for detecting Parkinson's disease,particularly in its early stages.However,the high cost of EEG data annotation often results in limited dataset size and considerable…

Machine Learning · Computer Science 2025-08-22 Qian Zhang , Ruilin Zhang , Jun Xiao , Yifan Liu , Zhe Wang

Being able to predict the performance of circuits without running expensive simulations is a desired capability that can catalyze automated design. In this paper, we present a supervised pretraining approach to learn circuit representations…

Machine Learning · Computer Science 2022-04-04 Kourosh Hakhamaneshi , Marcel Nassar , Mariano Phielipp , Pieter Abbeel , Vladimir Stojanović

Classification models for electroencephalogram (EEG) data show a large decrease in performance when evaluated on unseen test sub jects. We reduce this performance decrease using new regularization techniques during model training. We…

Machine Learning · Computer Science 2023-10-16 Niklas Smedemark-Margulies , Ye Wang , Toshiaki Koike-Akino , Jing Liu , Kieran Parsons , Yunus Bicer , Deniz Erdogmus