Anjalie Field author Sascha Rothe author Simon Baumgartner author Cong Yu author Abe Ittycheriah author 2020-jul text Association for Computational Linguistics Online conference publication We evaluate the performance of transformer encoders with various decoders for information organization through a new task: generation of section headings for Wikipedia articles. Our analysis shows that decoders containing attention mechanisms over the encoder output achieve high-scoring results by generating extractive text. In contrast, a decoder without attention better facilitates semantic encoding and can be used to generate section embeddings. We additionally introduce a new loss function, which further encourages the decoder to generate high-quality embeddings. field-etal-2020-generative https://www.aclweb.org/anthology/2020.ngt-1.9 2020-jul 79 87