English

Training a Subsampling Mechanism in Expectation

Machine Learning 2017-04-11 v3
Authors: Colin Raffel , Dieterich Lawson
View on arXiv ↗ PDF ↗

Abstract

We describe a mechanism for subsampling sequences and show how to compute its expected output so that it can be trained with standard backpropagation. We test this approach on a simple toy problem and discuss its shortcomings.

Cite

@article{arxiv.1702.06914,
  title  = {Training a Subsampling Mechanism in Expectation},
  author = {Colin Raffel and Dieterich Lawson},
  journal= {arXiv preprint arXiv:1702.06914},
  year   = {2017}
}

Comments

Camera-ready version. Includes additional figures in an appendix

Related papers

View all related →
Machine Learning · Statistics
NYTRO: When Subsampling Meets Early Stopping
Tomas Angles, Raffaello Camoriano, Alessandro Rudi, Lorenzo Rosasco
2018-02-02
Machine Learning · Computer Science
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes
Cedric De Boom, Thomas Demeester, Bart Dhoedt
2018-01-10
Machine Learning · Computer Science
Expectation Programming: Adapting Probabilistic Programming Systems to Estimate Expectations Efficiently
Tim Reichelt, Adam Goliński, Luke Ong, Tom Rainforth
2022-06-22
Machine Learning · Computer Science
Learning Probabilistic Programs Using Backpropagation
Avi Pfeffer
2017-05-17
Computation and Language · Computer Science
On Compressing Sequences for Self-Supervised Speech Models
Yen Meng, Hsuan-Jui Chen, Jiatong Shi, Shinji Watanabe +3
2022-10-26
Computation · Statistics
Subset Simulation Method for Rare Event Estimation: An Introduction
Konstantin Zuev
2015-05-14
Machine Learning · Computer Science
Neural Execution Engines: Learning to Execute Subroutines
Yujun Yan, Kevin Swersky, Danai Koutra, Parthasarathy Ranganathan +1
2020-10-26
Machine Learning · Computer Science
Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations
Tianxiang Zhao, Wenchao Yu, Suhang Wang, Lu Wang +5
2023-06-14
Systems and Control · Electrical Eng. & Systems
Task Decomposition for Iterative Learning Model Predictive Control
Charlott Vallon, Francesco Borrelli
2020-03-13
Computation and Language · Computer Science
Scheduled Sampling for Transformers
Tsvetomila Mihaylova, André F. T. Martins
2019-06-28
Machine Learning · Computer Science
Model-specific Data Subsampling with Influence Functions
Anant Raj, Cameron Musco, Lester Mackey, Nicolo Fusi
2020-10-21
Methodology · Statistics
A subsampling approach for large data sets when the Generalised Linear Model is potentially misspecified
Amalan Mahendran, Helen Thompson, James M. McGree
2025-10-08
Computation and Language · Computer Science
Scheduled Sampling Based on Decoding Steps for Neural Machine Translation
Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu +1
2021-09-01
Methodology · Statistics
Subsampling to Enhance Efficiency in Input Uncertainty Quantification
Henry Lam, Huajie Qian
2021-05-20
Quantitative Methods · Quantitative Biology
Adaptive machine learning for protein engineering
Brian L. Hie, Kevin K. Yang
2021-07-07
Computer Vision and Pattern Recognition · Computer Science
Prompt Distribution Learning
Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu +1
2022-05-09
Methodology · Statistics
Prediction model for rare events in longitudinal follow-up and resampling methods
Pierre Druilhet, Mathieu Berthe, Stéphanie Léger
2023-06-21
Numerical Analysis · Mathematics
Subsampling for tensor least squares: Optimization and statistical perspectives
Ling Tang, Hanyu Li
2022-12-01
Machine Learning · Computer Science
Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges
Xiaoqian Liu, Jianbin Jiao, Junge Zhang
2024-01-08
Neurons and Cognition · Quantitative Biology
Tackling the subsampling problem to infer collective properties from limited data
Anna Levina, Viola Priesemann, Johannes Zierenberg
2022-11-17
Machine Learning · Computer Science
Enhancing accuracy of deep learning algorithms by training with low-discrepancy sequences
Siddhartha Mishra, T. Konstantin Rusch
2020-05-27
Computation and Language · Computer Science
Explain and Predict, and then Predict Again
Zijian Zhang, Koustav Rudra, Avishek Anand
2021-02-05
Signal Processing · Electrical Eng. & Systems
Ensemble Wrapper Subsampling for Deep Modulation Classification
Sharan Ramjee, Shengtai Ju, Diyu Yang, Xiaoyu Liu +2
2020-05-12
Machine Learning · Computer Science
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer
2015-09-24
Artificial Intelligence · Computer Science
Selecting Representative Examples for Program Synthesis
Yewen Pu, Zachery Miranda, Armando Solar-Lezama, Leslie Pack Kaelbling
2018-06-08
R2 v1 2026-06-22T18:25:36.469Z