BERTian Poetics: Constrained Composition with Masked LMs
Computation and Language
2021-10-29 v1
Abstract
Masked language models have recently been interpreted as energy-based sequence models that can be generated from using a Metropolis--Hastings sampler. This short paper demonstrates how this can be instrumentalized for constrained composition and explores the poetics implied by such a usage. Our focus on constraints makes it especially apt to understand the generated text through the poetics of the OuLiPo movement.
Cite
@article{arxiv.2110.15181,
title = {BERTian Poetics: Constrained Composition with Masked LMs},
author = {Christopher Akiki and Martin Potthast},
journal= {arXiv preprint arXiv:2110.15181},
year = {2021}
}
Comments
Accepted as a poster at the 2021 NeurIPS Workshop on Machine Learning for Creativity and Design