自监督模型少样本学习中的文本与跨语言监督注入
音频与语音处理
2021-10-12 v1 计算与语言
摘要
自监督模型预训练近来引起了极大关注，但相对较少的工作探索了在微调这些模型时使用额外资源。我们展示了通用音素集声学模型如何利用跨语言监督来改进预训练自监督表示向新语言的迁移。我们还展示了如何利用目标语言文本，基于无网格最大互信息（LF-MMI）目标实现并改进微调。在三种低资源语言中，这些技术大幅提升了少样本学习性能。
引用
@article{arxiv.2110.04863,
title = {Injecting Text and Cross-lingual Supervision in Few-shot Learning from Self-Supervised Models},
author = {Matthew Wiesner and Desh Raj and Sanjeev Khudanpur},
journal= {arXiv preprint arXiv:2110.04863},
year = {2021}
}
备注
\c{opyright} 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works