publications

2024

1. Thesis

   Towards Controllable and Data-Efficient Natural Language Generation

   Du, Wanyu

   University of Virginia Apr 2024

   Abs PDF

   Large Language Models (LLMs) have revolutionized artificial intelligence (AI) applications by significantly simplifying users’ efforts to provide detailed and task-specific instructions to AI systems. The adoption of transformer architecture has notably enhanced LLMs’ ability to learn intricate linguistic patterns and world knowledge from extensive text datasets. Moreover, the large-scale pre-training on multiple natural language processing (NLP) tasks enables LLMs to address a wide range of NLP challenges with improved performance and efficiency. While LLMs excel in natural language understanding, they still face undeniable challenges in natural language generation (NLG). These challenges include generating undesired outputs (e.g., factually incorrect, harmful, biased contents), difficulty in adapting to low-resource domain-specific tasks, and misalignment with user intentions. To address the above challenges, this dissertation introduces methods to refine the generative capabilities of LLMs through controllable and data-efficient natural language generation techniques. The goal is to improve the alignment of generated content with user intentions using controllable and data-efficient LLMs. We design three major components to achieve the goal: (1) user intention identification to align models with human preferences; (2) controllable LLMs to produce outputs that meet specific user requirements; (3) data-efficient NLG to enhance LLM adaptability to low-resource domain-specific tasks. In summary, this dissertation provides a framework to understand user intentions and develop controllable generation methods to align LLMs with these intentions. At the end of the dissertation, an interactive text generation application is presented to demonstrate the benefits of leveraging user intentions and controllable LLMs for human-AI collaborative text generation.

2023

1. Preprint

   Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation

   Du, Wanyu, and Ji, Yangfeng

   ArXiv preprint Nov 2023

   Abs PDF

   The development of trustworthy conversational information-seeking systems relies on dialogue models that can generate faithful and accurate responses based on relevant knowledge texts. However, two main challenges hinder this task. Firstly, language models may generate hallucinations due to data biases present in their pretraining corpus. Secondly, knowledge texts often contain redundant and irrelevant information that distracts the model’s attention from the relevant text span. Previous works use additional data annotations on the knowledge texts to learn a knowledge identification module in order to bypass irrelevant information, but collecting such high-quality span annotations can be costly. In this work, we leverage reinforcement learning algorithms to overcome the above challenges by introducing a novel reward function. Our reward function combines an accuracy metric and a faithfulness metric to provide a balanced quality judgment of generated responses, which can be used as a cost-effective approximation to a human preference reward model when only a few preference annotations are available. Empirical experiments on two conversational information-seeking datasets demonstrate that our method can compete with other strong supervised learning baselines.

2. Workshop

   Explaining Predictive Uncertainty by Looking Back at Model Explanations

   Chen, Hanjie,  Du, Wanyu, and Ji, Yangfeng

   AAAI Workshop on Uncertainty Reasoning and Quantification in Decision Making Feb 2023

   Abs PDF

   Predictive uncertainty estimation of pre-trained language models is an important measure of how likely people can trust their predictions. However, little is known about what makes a model prediction uncertain. Explaining predictive uncertainty is an important complement to explaining prediction labels in helping users understand model decision making and gaining their trust on model predictions, while has been largely ignored in prior works. In this work, we propose to explain the predictive uncertainty of pre-trained language models by extracting uncertain words from existing model explanations. We find the uncertain words are those identified as making negative contributions to prediction labels, while actually explaining the predictive uncertainty. Experiments show that uncertainty explanations are indispensable to explaining models and helping humans understand model prediction behavior.

2022

1. EMNLP

   Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation

   Du, Wanyu, Chen, Hanjie, and Ji, Yangfeng

   Findings of the Association for Computational Linguistics: EMNLP 2022 Dec 2022

   Abs PDF Code

   In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost on annotating MR-to-Text pairs. Previous works on self-training leverage fine-tuned conversational models to automatically generate pseudo-labeled MR-to-Text pairs for further fine-tuning. However, some self-augmented data may be noisy or uninformative for the model to learn from. In this work, we propose a two-phase self-augmentation procedure to generate high-quality pseudo-labeled MR-to-Text pairs: the first phase selects the most informative MRs based on model’s prediction uncertainty; with the selected MRs, the second phase generates accurate responses by aggregating multiple perturbed latent representations from each MR. Empirical experiments on two benchmark datasets, FewShotWOZ and FewShotSGD, show that our method generally outperforms existing self-training methods on both automatic and human evaluations.

2. EMNLP

   FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows

   Zhao, Jianqiao, Li, Yanyang,  Du, Wanyu, Ji, Yangfeng, Yu, Dong, Lyu, Michael R., and Wang, Liwei

   Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022) Dec 2022

   Abs PDF

   Despite recent progress in open-domain dialogue evaluation, how to develop automatic metrics remains an open problem. We explore the potential of dialogue evaluation featuring dialog act information, which was hardly explicitly modeled in previous methods. However, defined at the utterance level in general, dialog act is of coarse granularity, as an utterance can contain multiple segments possessing different functions. Hence, we propose segment act, an extension of dialog act from utterance level to segment level, and crowdsource a large-scale dataset for it. To utilize segment act flows, sequences of segment acts, for evaluation, we develop the first consensus-based dialogue evaluation framework, FlowEval. This framework provides a reference-free approach for dialog evaluation by finding pseudo-references. Extensive experiments against strong baselines on three benchmark datasets demonstrate the effectiveness and other desirable characteristics of our FlowEval, pointing out a potential path for better dialogue evaluation.

3. EMNLP

   Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks

   Kim, Zae Myung,  Du, Wanyu, Raheja, Vipul, Kumar, Dhruv, and Kang, Dongyeop

   Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022) Dec 2022

   Abs PDF Code

   Iterative text revision improves text quality by fixing grammatical errors, rephrasing for better readability or contextual appropriateness, or reorganizing sentence structures throughout a document. Most recent research has focused on understanding and classifying different types of edits in the iterative revision process from human-written text instead of building accurate and robust systems for iterative text revision. In this work, we aim to build an end-to-end text revision system that can iteratively generate helpful edits by explicitly detecting editable spans (where-to-edit) with their corresponding edit intents and then instructing a revision model to revise the detected edit spans. Leveraging datasets from other related text editing NLP tasks, combined with the specification of editable spans, leads our system to more accurately model the process of iterative text refinement, as evidenced by empirical results and human evaluations. Our system significantly outperforms previous baselines on our text revision tasks and other standard text revision tasks, including grammatical error correction, text simplification, sentence fusion, and style transfer. Through extensive qualitative and quantitative analysis, we make vital connections between edit intentions and writing quality, and better computational modeling of iterative text revisions.

4. ACL

   Understanding Iterative Revision from Human-Written Text

   Du, Wanyu, Raheja, Vipul, Kumar, Dhruv, Kim, Zae Myung, Lopez, Melissa, and Kang, Dongyeop

   Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics May 2022

   Abs PDF Demo Code

   Writing is, by nature, a strategic, adaptive, and, more importantly, an iterative process. A crucial part of writing is editing and revising the text. Previous works on text revision have focused on defining edit intention taxonomies within a single domain or developing computational models with a single level of edit granularity, such as sentence-level edits, which differ from human’s revision cycles. This work describes IteraTeR: the first large-scale, multi-domain, edit-intention annotated corpus of iteratively revised text. In particular, IteraTeR is collected based on a new framework to comprehensively model the iterative text revisions that generalizes to a variety of domains, edit intentions, revision depths, and granularities. When we incorporate our annotated edit intentions, both generative and action-based text revision models significantly improve automatic evaluations. Through our work, we better understand the text revision process, making vital connections between edit intentions and writing quality, enabling the creation of diverse corpora to support computational modeling of iterative text revisions.

5. Workshop

   Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision

   Du, Wanyu, Kim, Zae Myung, Raheja, Vipul, Kumar, Dhruv, and Kang, Dongyeop

   Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022) May 2022

   Abs PDF Demo Code

   Revision is an essential part of the human writing process. It tends to be strategic, adaptive, and, more importantly, iterative in nature. Despite the success of large language models on text revision tasks, they are limited to non-iterative, one-shot revisions. Examining and evaluating the capability of large language models for making continuous revisions and collaborating with human writers is a critical step towards building effective writing assistants. In this work, we present a human-in-the-loop iterative text revision system, Read, Revise, Repeat (R3), which aims at achieving high quality text revisions with minimal human efforts by reading model-generated revisions and user feedbacks, revising documents, and repeating human-machine interactions. In R3, a text revision model provides text editing suggestions for human writers, who can accept or reject the suggested edits. The accepted edits are then incorporated into the model for the next iteration of document revision. Writers can therefore revise documents iteratively by interacting with the system and simply accepting/rejecting its suggested edits until the text revision model stops making further revisions or reaches a predefined maximum number of revisions. Empirical experiments show that R3 can generate revisions with comparable acceptance rate to human writers at early revision depths, and the human-machine interaction can get higher quality revisions with fewer iterations and edits. The collected human-model interaction dataset and system code are available at \urlhttps://github.com/vipulraheja/IteraTeR. Our system demonstration is available at \urlhttps://youtu.be/lK08tIpEoaE.

6. Workshop

   Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors

   Du, Wanyu, Zhao, Jianqiao, Wang, Liwei, and Ji, Yangfeng

   The Sixth Workshop on Structured Prediction for NLP (SPNLP 2022) May 2022

   Abs PDF Code

   Generating high quality texts with high diversity is important for many NLG applications, but current methods mostly focus on building deterministic models to generate higher quality texts and do not provide many options for promoting diversity. In this work, we present a novel latent structured variable model to generate high quality texts by enriching contextual representation learning of encoder-decoder models. Specifically, we introduce a stochastic function to map deterministic encoder hidden states into random context variables. The proposed stochastic function is sampled from a Gaussian process prior to (1) provide infinite number of joint Gaussian distributions of random context variables (diversity-promoting) and (2) explicitly model dependency between context variables (accurate-encoding). To address the learning challenge of Gaussian processes, we propose an efficient variational inference approach to approximate the posterior distribution of random context variables. We evaluate our method in two typical text generation tasks: paraphrase generation and text style transfer. Experimental results on benchmark datasets demonstrate that our method improves the generation quality and diversity compared with other baselines.

2021

1. EMNLP

   SideControl: Controlled Open-domain Dialogue Generation via Additive Side Networks

   Du, Wanyu, and Ji, Yangfeng

   Findings of the Association for Computational Linguistics: EMNLP 2021 Nov 2021

   Abs PDF Code

   Transformer-based pre-trained language models boost the performance of open-domain dialogue systems. Prior works leverage Transformer-based pre-trained language models to generate texts with desired attributes in two general approaches: (1) gradient-based methods: updating all latent representations of pre-trained models with gradients from attribute models; (2) weighted-decoding methods: re-ranking beam candidates from pre-trained models with attribute functions. However, gradient-based methods lead to high computation cost and can easily get overfitted on small training sets, while weighted-decoding methods are inherently constrained by the low-variance high-bias pre-trained model. In this work, we propose a novel approach to control the generation of Transformer-based pre-trained language models: the SideControl framework, which leverages a novel control attributes loss to incorporate useful control signals, and is shown to perform well with very limited training samples. We evaluate our proposed method on two benchmark open-domain dialogue datasets, and results show that the SideControl framework has better controllability, higher generation quality and better sample-efficiency than existing gradient-based and weighted-decoding baselines.

2. Workshop

   The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

   Gehrmann, et al.

   Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Aug 2021

   Abs PDF Website

   We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of automated metrics, datasets, and human evaluation standards. Due to this moving target, new models often still evaluate on divergent anglo-centric corpora with well-established, but flawed, metrics. This disconnect makes it challenging to identify the limitations of current models and opportunities for progress. Addressing this limitation, GEM provides an environment in which models can easily be applied to a wide set of tasks and in which evaluation strategies can be tested. Regular updates to the benchmark will help NLG research become more multilingual and evolve the challenge alongside models. This paper serves as the description of the data for the 2021 shared task at the associated GEM Workshop.

3. INLG

   Contextualizing Variation in Text Style Transfer Datasets

   Schoch, Stephanie,  Du, Wanyu, and Ji, Yangfeng

   Proceedings of the 14th International Conference on Natural Language Generation Aug 2021

   Abs PDF

   Text style transfer involves rewriting the content of a source sentence in a target style. Despite there being a number of style tasks with available data, there has been limited systematic discussion of how text style datasets relate to each other. This understanding, however, is likely to have implications for selecting multiple data sources for model training. While it is prudent to consider inherent stylistic properties when determining these relationships, we also must consider how a style is realized in a particular dataset. In this paper, we conduct several empirical analyses of existing text style datasets. Based on our results, we propose a categorization of stylistic and dataset properties to consider when utilizing or comparing text style datasets.

2019

1. EMNLP

   An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation

   Du, Wanyu, and Ji, Yangfeng

   Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) Nov 2019

   Abs PDF Code

   Generating paraphrases from given sentences involves decoding words step by step from a large vocabulary. To learn a decoder, supervised learning which maximizes the likelihood of tokens always suffers from the exposure bias. Although both reinforcement learning (RL) and imitation learning (IL) have been widely used to alleviate the bias, the lack of direct comparison leads to only a partial image on their benefits. In this work, we present an empirical study on how RL and IL can help boost the performance of generating paraphrases, with the pointer-generator as a base model. Experiments on the benchmark datasets show that (1) imitation learning is constantly better than reinforcement learning; and (2) the pointer-generator models with imitation learning outperform the state-of-the-art methods with a large margin.