Gemma 2: 提升实际规模下的开放语言模型性能
计算机视觉与模式识别
2024-08-02 v1 机器学习
机器人学
系统与控制
系统与控制
摘要
本文介绍Gemma 2，这是Gemma系列轻量级最先进开放模型的新成员，规模从20亿到270亿参数不等。在本新版本中，我们对Transformer架构应用了若干已知技术修改，如交错式局部-全局注意力(Beltagy等, 2020a)和组查询注意力(Ainslie等, 2023)。我们还使用知识蒸馏(Hinton等, 2015)而非下一个标记预测训练2B和9B模型。 resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.
引用
@article{arxiv.2408.00117,
title = {Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods},
author = {Xusheng Luo and Tianhao Wei and Simin Liu and Ziwei Wang and Luis Mattei-Mendez and Taylor Loper and Joshua Neighbor and Casidhe Hutchison and Changliu Liu},
journal= {arXiv preprint arXiv:2408.00117},
year = {2024}
}
备注
25 pages, 10 figures, 5 tables