English

A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks

Machine Learning 2018-02-27 v2
Authors: Behnam Neyshabur , Srinadh Bhojanapalli , Nathan Srebro
View on arXiv ↗ PDF ↗

Abstract

We present a generalization bound for feedforward neural networks in terms of the product of the spectral norm of the layers and the Frobenius norm of the weights. The generalization bound is derived using a PAC-Bayes analysis.

Keywords

generalization bounds

Cite

@article{arxiv.1707.09564,
  title  = {A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks},
  author = {Behnam Neyshabur and Srinadh Bhojanapalli and Nathan Srebro},
  journal= {arXiv preprint arXiv:1707.09564},
  year   = {2018}
}

Comments

Accepted to ICLR 2018

Related papers

View all related →
Machine Learning · Computer Science
PAC-Bayesian Margin Bounds for Convolutional Neural Networks
Konstantinos Pitas, Mike Davies, Pierre Vandergheynst
2018-04-24
Machine Learning · Computer Science
A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks
Renjie Liao, Raquel Urtasun, Richard Zemel
2020-12-15
Machine Learning · Statistics
Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds
Xinping Yi, Gaojie Jin, Xiaowei Huang, Shi Jin
2026-01-14
Machine Learning · Computer Science
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning
Wei Huang, Chunrui Liu, Yilan Chen, Tianyu Liu +1
2022-02-07
Quantum Physics · Physics
A PAC-Bayesian approach to generalization for quantum models
Pablo Rodriguez-Grasa, Matthias C. Caro, Jens Eisert, Elies Gil-Fuster +2
2026-03-25
Machine Learning · Computer Science
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion
Haotian Ju, Dongyue Li, Aneesh Sharma, Hongyang R. Zhang
2023-10-25
Machine Learning · Statistics
A General Framework for the Practical Disintegration of PAC-Bayesian Bounds
Paul Viallard, Pascal Germain, Amaury Habrard, Emilie Morvant
2023-09-19
Machine Learning · Computer Science
PAC-Bayesian Spectrally-Normalized Bounds for Adversarially Robust Generalization
Jiancong Xiao, Ruoyu Sun, Zhi- Quan Luo
2023-10-31
Machine Learning · Computer Science
A PAC-Bayesian Generalization Bound for Equivariant Networks
Arash Behboodi, Gabriele Cesa, Taco Cohen
2022-10-25
Machine Learning · Statistics
Generalization bounds for deep learning
Guillermo Valle-Pérez, Ard A. Louis
2020-12-10
Machine Learning · Computer Science
Topology-Aware PAC-Bayesian Generalization Analysis for Graph Neural Networks
Xinping Yi
2026-04-14
Machine Learning · Computer Science
On the Importance of Gradient Norm in PAC-Bayesian Bounds
Itai Gat, Yossi Adi, Alexander Schwing, Tamir Hazan
2022-11-03
Machine Learning · Computer Science
Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience
Vaishnavh Nagarajan, J. Zico Kolter
2019-06-03
Machine Learning · Statistics
PAC-Bayesian Adversarially Robust Generalization Bounds for Graph Neural Network
Tan Sun, Junhong Lin
2024-07-09
Machine Learning · Computer Science
Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks
Benjamin Leblanc, Mathieu Bazinet, Nathaniel D'Amours, Alexandre Drouin +1
2025-06-06
Machine Learning · Statistics
Generalization Bounds for Convolutional Neural Networks
Shan Lin, Jingwei Zhang
2019-10-04
Machine Learning · Computer Science
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability
Alec Farid, Anirudha Majumdar
2021-10-27
Statistics Theory · Mathematics
Risk bounds for aggregated shallow neural networks using Gaussian prior
Laura Tinsi, Arnak S. Dalalyan
2022-02-04
Machine Learning · Computer Science
Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networks
Felix Biggs, Benjamin Guedj
2021-12-16
Machine Learning · Statistics
Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P. Adams +1
2019-02-26
Machine Learning · Computer Science
Generalization Performance of Hypergraph Neural Networks
Yifan Wang, Gonzalo R. Arce, Guangmo Tong
2025-01-28
Information Theory · Computer Science
A DPI-PAC-Bayesian Framework for Generalization Bounds
Muhan Guan, Farhad Farokhi, Jingge Zhu
2025-08-26
Machine Learning · Statistics
PAC-Bayesian Transportation Bound
Kohei Miyaguchi
2019-11-22
Machine Learning · Computer Science
PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations
Louis Fortier-Dubois, Gaël Letarte, Benjamin Leblanc, François Laviolette +1
2023-04-17
Machine Learning · Statistics
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets
Benjamin Dupuis, Paul Viallard, George Deligiannidis, Umut Simsekli
2025-02-11
R2 v1 2026-06-22T21:01:27.287Z