English

Think Again Networks and the Delta Loss

Computation and Language 2019-05-02 v2 Machine Learning
Authors: Alexandre Salle , Marcelo Prates
View on arXiv ↗ PDF ↗

Abstract

This short paper introduces an abstraction called Think Again Networks (ThinkNet) which can be applied to any state-dependent function (such as a recurrent neural network).

Keywords

think again networks delta loss state-dependent function recurrent neural network

Cite

@article{arxiv.1904.11816,
  title  = {Think Again Networks and the Delta Loss},
  author = {Alexandre Salle and Marcelo Prates},
  journal= {arXiv preprint arXiv:1904.11816},
  year   = {2019}
}

Comments

redacted experiments on language modeling due to evaluation error

Related papers

View all related →
Machine Learning · Computer Science
State representation learning with recurrent capsule networks
Louis Annabi, Michael Garcia Ortiz
2019-02-25
Neural and Evolutionary Computing · Computer Science
Delta Networks for Optimized Recurrent Network Computation
Daniel Neil, Jun Haeng Lee, Tobi Delbruck, Shih-Chii Liu
2016-12-19
Neural and Evolutionary Computing · Computer Science
A Novel Framework for Recurrent Neural Networks with Enhancing Information Processing and Transmission between Units
Xi Chen, Zhihong Deng, Gehui Shen, Ting Huang
2018-06-05
Artificial Intelligence · Computer Science
Applying Policy Iteration for Training Recurrent Neural Networks
I. Szita, A. Lorincz
2007-05-23
Machine Learning · Computer Science
Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
Robin M. Schmidt
2019-12-13
Neural and Evolutionary Computing · Computer Science
Memory and Information Processing in Recurrent Neural Networks
Alireza Goudarzi, Sarah Marzen, Peter Banda, Guy Feldman +2
2016-04-26
Systems and Control · Electrical Eng. & Systems
On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments
Fabio Bonassi, Marcello Farina, Jing Xie, Riccardo Scattolini
2022-05-11
Neural and Evolutionary Computing · Computer Science
Network of Recurrent Neural Networks
Chao-Ming Wang
2017-10-11
Neural and Evolutionary Computing · Computer Science
Recent Advances in Recurrent Neural Networks
Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak +1
2018-02-26
Machine Learning · Computer Science
Never Reset Again: A Mathematical Framework for Continual Inference in Recurrent Neural Networks
Bojian Yin, Federico Corradi
2024-12-23
Machine Learning · Computer Science
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Neehal Tumma, Noel Loo, Daniela Rus
2026-04-24
Machine Learning · Computer Science
Reversible Recurrent Neural Networks
Matthew MacKay, Paul Vicol, Jimmy Ba, Roger Grosse
2018-10-26
Machine Learning · Statistics
Echoes of the Past: A Unified Perspective on Fading memory and Echo States
Juan-Pablo Ortega, Florian Rossmannek
2026-05-07
Machine Learning · Computer Science
ReasoNet: Learning to Stop Reading in Machine Comprehension
Yelong Shen, Po-Sen Huang, Jianfeng Gao, Weizhu Chen
2017-06-21
Computer Vision and Pattern Recognition · Computer Science
Retrospective Loss: Looking Back to Improve Training of Deep Neural Networks
Surgan Jandial, Ayush Chopra, Mausoom Sarkar, Piyush Gupta +2
2020-06-25
Machine Learning · Computer Science
Exploring Flip Flop memories and beyond: training recurrent neural networks with key insights
Cecilia Jarne
2023-08-01
Artificial Intelligence · Computer Science
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models
Juergen Schmidhuber
2015-12-01
Computation and Language · Computer Science
Tracking the World State with Recurrent Entity Networks
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes +1
2017-05-11
Machine Learning · Computer Science
On Computability, Learnability and Extractability of Finite State Machines from Recurrent Neural Networks
Reda Marzouk
2020-09-15
Computation and Language · Computer Science
A Survey of Retentive Network
Haiqi Yang, Zhiyuan Li, Yi Chang, Yuan Wu
2025-06-10
Machine Learning · Statistics
MinimalRNN: Toward More Interpretable and Trainable Recurrent Neural Networks
Minmin Chen
2018-06-21
Neurons and Cognition · Quantitative Biology
Neural Network Degeneration and its Relationship to the Brain
Jacob Adamczyk
2020-08-04
Analysis of PDEs · Mathematics
Continuous limits of residual neural networks in case of large input data
M. Herty, A. Thuenen, T. Trimborn, G. Visconti
2022-05-11
Machine Learning · Computer Science
A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation
Gang Chen
2018-01-16
Computation and Language · Computer Science
Acyclic and Cyclic Reversing Computations in Petri Nets
Kamila Barylska, Anna Gogolińska
2023-06-22