File Download
There are no files associated with this item.
Supplementary
-
Citations:
- Appears in Collections:
Conference Paper: DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Title | DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models |
---|---|
Authors | |
Issue Date | 1-May-2023 |
Abstract | Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at https://github.com/Shark-NLP/DiffuSeq |
Persistent Identifier | http://hdl.handle.net/10722/333819 |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Gong, Shansan | - |
dc.contributor.author | Li, Mukai | - |
dc.contributor.author | Feng, Jiangtao | - |
dc.contributor.author | Wu, Zhiyong | - |
dc.contributor.author | Kong, Lingpeng | - |
dc.date.accessioned | 2023-10-06T08:39:20Z | - |
dc.date.available | 2023-10-06T08:39:20Z | - |
dc.date.issued | 2023-05-01 | - |
dc.identifier.uri | http://hdl.handle.net/10722/333819 | - |
dc.description.abstract | <p>Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at <a href="https://github.com/Shark-NLP/DiffuSeq">https://github.com/Shark-NLP/DiffuSeq</a><br></p> | - |
dc.language | eng | - |
dc.relation.ispartof | International Conference on Learning Representations (ICLR 2023) (01/05/2023-05/05/2023, Kigali, Rwanda) | - |
dc.title | DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models | - |
dc.type | Conference_Paper | - |
dc.identifier.doi | 10.48550/arXiv.2210.08933 | - |