We present a large, tunable neural conversational response generation model, DIALOGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and training pipeline are publicly released to facilitate research into neural response generation and the development of more intelligent opendomain dialogue systems.

最近的大规模预训练的进步，例如GPT-3允许从给定提示生成看似高质量的文本。然而，这种一代系统经常遭受幻觉的事实问题，并且本身并不是旨在包含有用的外部信息。接地的代表似乎提供了补救措施，但他们的培训通常依赖于提供信息相关文件的很少可用的并行数据。我们提出了一个框架，通过在语言模型信号上共同训练接地的发生器和文档检索来缓解这种数据约束。该模型学会奖励具有生成中最高效用的文档的检索，并用专家混合（MOE）合并来术语术，以产生后续文本。我们证明，发电机和猎犬都可以利用这种联合培训，协同作用，以生产散文和对话一代中的更多信息和相关文本。

我们介绍了Godel（接地开放对话语言模型），这是对话框的大型预训练的语言模型。与诸如Dialogpt之类的早期模型相比，Godel利用了一个新的扎根预训练阶段，旨在更好地支持将Godel适应广泛的下游对话框任务，这些任务需要当前对话外部的信息（例如，数据库或文档）到产生良好的回应。针对一系列基准测试的实验，这些基准涵盖了面向任务的对话框，对话质量质量检查和接地的开放式对话框，表明Godel在几次以上的微调设置中优于最先进的预训练的对话模型，就人类和自动评估。我们评估方法的一个新颖特征是引入了一个效用概念，该概念除了其交流特征（内在评估）外，还评估了响应的有用性（外部评估）。我们表明，外部评估提供了改进的通道间一致性和与自动指标的相关性。代码和数据处理脚本公开可用。

预先接受的语言模型（PLM）在神经对话建模中标志着巨大的飞跃。虽然PLMS在大型文本语料库上进行预先培训，但通常在具有特定领域知识和对话风格的稀缺对话数据上进行微调。然而，在大型预先训练模型中充分利用现有知识的同时定制语言模型仍然是一个挑战。在本文中，我们提出了一种预先接受训练的对话建模的新方法，将对话生成问题作为一个快速学习任务。而不是在有限的对话数据上进行微调，我们的方法，DialogPrompt学习针对对话背景优化的连续提示嵌入，从而从大型预训练模型中促进了知识。为了鼓励模型更好地利用提示嵌入，提示编码器被设计为在输入对话框上下文中的条件。流行对话数据集的实验表明，我们的方法显着优于微调基线和通用及时学习方法。此外，人类评估强烈支持对DialialPrompt的优越性在响应生成质量方面。

深度神经语言模型的最新进展与大规模数据集的能力相结合，加速了自然语言生成系统的发展，这些系统在多种任务和应用程序上下文中产生流利和连贯的文本（在各种成功程度上）。但是，为所需的用户控制这些模型的输出仍然是一个开放的挑战。这不仅对于自定义生成语言的内容和样式至关重要，而且对于他们在现实世界中的安全可靠部署至关重要。我们提出了一项关于受约束神经语言生成的新兴主题的广泛调查，在该主题中，我们通过区分条件和约束（后者是在输出文本上而不是输入的可检验条件），正式定义和分类自然语言生成问题，目前是可检验的）约束文本生成任务，并查看受限文本生成的现有方法和评估指标。我们的目的是强调这个新兴领域的最新进展和趋势，以告知最有希望的方向和局限性，以推动受约束神经语言生成研究的最新作品。

Sequence-to-sequence neural network models for generation of conversational responses tend to generate safe, commonplace responses (e.g., I don't know) regardless of the input. We suggest that the traditional objective function, i.e., the likelihood of output (response) given input (message) is unsuited to response generation tasks. Instead we propose using Maximum Mutual Information (MMI) as the objective function in neural models. Experimental results demonstrate that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations. Input: What are you doing? −0.86 I don't know. −1.09 Get out of here. −1.03 I don't know! −1.09 I'm going home. −1.06 Nothing. −1.09 Oh my god! −1.09 Get out of the way. −1.10 I'm talking to you. Input: what is your name? −0.91 I don't know. ... −0.92 I don't know! −1.55 My name is Robert. −0.92 I don't know, sir. −1.58 My name is John. −0.97 Oh, my god! −1.59 My name's John. Input: How old are you? −0.79 I don't know. ... −1.06 I'm fine.−1.64 Twenty-five. −1.17 I'm all right.−1.66 Five. −1.17 I'm not sure.−1.71 Eight.

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

We present SODA: the first publicly available, million-scale high-quality social dialogue dataset. Using SODA, we train COSMO: a generalizable conversation agent outperforming previous best-performing agents on both in- and out-of-domain datasets. In contrast to most existing crowdsourced, small-scale dialogue corpora, we distill 1.5M socially-grounded dialogues from a pre-trained language model (InstructGPT; Ouyang et al., 2022). Dialogues are distilled by contextualizing social commonsense knowledge from a knowledge graph (Atomic10x; West et al., 2022). Human evaluation shows that dialogues in SODA are more consistent, specific, and (surprisingly) natural than prior human-authored datasets - e.g., DailyDialog (Li et al., 2017), BlendedSkillTalk (Smith et al., 2020). In addition, extensive evaluations show that COSMO is significantly more natural and consistent on unseen datasets than best-performing dialogue models - e.g., GODEL (Peng et al., 2022), BlenderBot (Roller et al., 2021), DialoGPT (Zhang et al., 2020). Furthermore, it is sometimes even preferred to the original human-written gold responses. We make our data, models, and code public.

在本文中，我们介绍了基于大型预训练的语言模型（PLM）pangu-alpha（Zeng等，2021）的中国预训练的开放域对话生成模型。与其他对大量对话数据进行培训的预训练的对话模型不同，我们旨在通过继承PLM的有价值的语言能力和知识来构建强大的对话模型，并以相对较少的数据和计算成本构建强大的对话模型。为此，我们训练大型PLM Pangu-Alpha的Pangu-bot，该机器人已被证明在各种中国自然语言任务上表现出色。我们研究了pangu-bot产生的响应的不同方面，包括响应质量，知识和安全性。我们表明，Pangu-Bot优于最先进的中国对话系统（CDIALGPT（Wang等，2020），Eva（Zhou等，2021），EVA2.0（Gu等，2022）） W.R.T.以上三个方面。我们还证明，可以轻松地部署pangu-bot，以在没有进一步训练的情况下产生情感反应。在整个经验分析中，我们还指出，Pangu-bot响应质量，知识正确性和安全性仍然远非完美，进一步的探索对于建立可靠且智能的对话系统是必不可少的。我们的型号和代码将在https://github.com/huawei-noah/pretretaining-language-model/tree/master/master/pangu-bot上提供。

The goal of building dialogue agents that can converse with humans naturally has been a long-standing dream of researchers since the early days of artificial intelligence. The well-known Turing Test proposed to judge the ultimate validity of an artificial intelligence agent on the indistinguishability of its dialogues from humans'. It should come as no surprise that human-level dialogue systems are very challenging to build. But, while early effort on rule-based systems found limited success, the emergence of deep learning enabled great advance on this topic. In this thesis, we focus on methods that address the numerous issues that have been imposing the gap between artificial conversational agents and human-level interlocutors. These methods were proposed and experimented with in ways that were inspired by general state-of-the-art AI methodologies. But they also targeted the characteristics that dialogue systems possess.

Long-range context modeling is crucial to both dialogue understanding and generation. The most popular method for dialogue context representation is to concatenate the last-$k$ previous utterances. However, this method may not be ideal for conversations containing long-range dependencies. In this work, we propose DialoGX, a novel encoder-decoder based framework for conversational response generation with a generalized and explainable context representation that can look beyond the last-$k$ utterances. Hence the method is adaptive to conversations with long-range dependencies. The main idea of our approach is to identify and utilize the most relevant historical utterances instead of the last-$k$ utterances in chronological order. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST) tasks. DialoGX achieves comparable performance with the state-of-the-art models on DailyDialog dataset. We also observe performance gain in existing DST models with our proposed context representation strategy on MultiWOZ dataset. We justify our context representation through the lens of psycholinguistics and show that the relevance score of previous utterances agrees well with human cognition which makes DialoGX explainable as well.

对话模型面临一个困难的权衡。对模型进行了大量文本培训，但是他们的响应需要仅限于对话框的所需范围和样式。由于用于实现前者包含与后者不兼容的语言的数据集，因此在较小的策划数据集上对预训练的对话框模型进行了微调。但是，微调过程剥夺了他们产生多种反应的能力，最终将他们降低到乏味的对话伙伴。在本文中，我们调查提示是否可以减轻上述权衡。具体来说，我们试验了在查询上调节提示，而不是训练所有查询的单个提示。通过遵循直觉，即冻结预训练的语言模型将保持其表现力，我们发现与微调相比，提示可以达到更高的BLEU得分，并显着改善了响应的多样性和新颖性。

我们提出了Blenderbot 3，这是一个175B参数对话模型，能够通过访问Internet和长期内存进行开放域对话，并接受了大量用户定义的任务的培训。我们同时发布了模型权重和代码，还将模型部署在公共网页上，以与有机用户进行交互。该技术报告描述了该模型的构建方式（建筑，模型和培训计划）以及其部署的细节，包括安全机制。人类评估表明，它优于现有的开放域对话代理，包括其前身（Roller等，2021； Komeili等，2022）。最后，我们使用部署收集的数据详细介绍了持续学习的计划，该数据也将公开发布。因此，该研究计划的目标是使社区能够研究通过互动学习的不断改进的负责任的代理商。

Controllable Text Generation (CTG) is emerging area in the field of natural language generation (NLG). It is regarded as crucial for the development of advanced text generation technologies that are more natural and better meet the specific constraints in practical applications. In recent years, methods using large-scale pre-trained language models (PLMs), in particular the widely used transformer-based PLMs, have become a new paradigm of NLG, allowing generation of more diverse and fluent text. However, due to the lower level of interpretability of deep neural networks, the controllability of these methods need to be guaranteed. To this end, controllable text generation using transformer-based PLMs has become a rapidly growing yet challenging new research hotspot. A diverse range of approaches have emerged in the recent 3-4 years, targeting different CTG tasks which may require different types of controlled constraints. In this paper, we present a systematic critical review on the common tasks, main approaches and evaluation methods in this area. Finally, we discuss the challenges that the field is facing, and put forward various promising future directions. To the best of our knowledge, this is the first survey paper to summarize CTG techniques from the perspective of PLMs. We hope it can help researchers in related fields to quickly track the academic frontier, providing them with a landscape of the area and a roadmap for future research.

语言是人类交流的主要工具，其中幽默是最有吸引力的部分之一。使用计算机，又称自然语言生成（NLG）的人类产生自然语言，已广泛用于对话系统，聊天机器人，机器翻译以及计算机AID创建，例如Idea Generations，剧本。但是，自然语言的幽默方面相对不足，尤其是在预训练的语言模型时代。在这项工作中，我们旨在初步测试NLG是否可以像人类一样产生幽默。我们构建了一个新的数据集，该数据集由众多数字化的中国可笑的串扰脚本（称为c $^3 $简称），该脚本适用于1800年代以来名为“ Xiangsheng”的流行中国表演艺术。 （为了方便非中国扬声器，我们在本文中称为“ Xiangsheng”的“ Crosstalk”。）我们基准了各种一代方法，包括训练seq2seq，微调中级PLMS和大型PLMS（大型PLMS）（有无微调）。此外，我们还进行了人类评估，表明1）大规模预处理在很大程度上提高了串扰的产生质量； 2）即使是从最佳PLM产生的脚本也远非我们的期望，只有65％的人类创建的串扰质量。我们得出结论，使用大型PLM可以在很大程度上改善幽默的产生，但仍处于起步阶段。 \ url {https://github.com/anonno2/crosstalk-generation}公开可用数据和基准代码。

没有一致响应的对话系统并不令人着迷。在这项研究中，我们建立了一个对话系统，可以根据给定的角色设置（角色）响应以带来一致性。考虑到语言模型迅速增加的趋势，我们提出了一种使用迅速调整的方法，该方法在预训练的大规模语言模型上使用了低学习成本。英语和日语中自动和手动评估的结果表明，可以使用比微调更少的计算资源来构建具有更自然和个性化响应的对话系统。

聊天机器人用于许多应用程序中，例如自动化代理，智能家庭助理，在线游戏中的互动角色等。因此，确保他们不会以不希望的方式行事，对用户提供令人反感或有毒的反应。这并不是一项琐碎的任务，因为最先进的聊天机器人模型是在从互联网公开收集的大型公共数据集上培训的。本文提出了对聊天机器人中毒性的首次大规模测量。我们表明，公开可用的聊天机器人很容易在喂养有毒的查询时提供有毒的反应。更令人担忧的是，一些无毒的查询也会触发有毒反应。然后，我们着手设计和实验攻击，即毒性，该攻击依赖于微调的GPT-2来产生无毒的查询，使聊天机器人以有毒的方式做出反应。我们广泛的实验评估表明，我们的攻击对公共聊天机器人模型有效，并且优于先前工作提出的手动制作的恶意查询。我们还评估了针对毒性的三种防御机制，表明它们要么以影响聊天机器人的效用而降低攻击性能，要么仅有效地减轻了一部分攻击。这强调了对计算机安全和在线安全社区进行更多研究的需求，以确保聊天机器人模型不会伤害其用户。总体而言，我们有信心有毒可以用作审计工具，我们的工作将为设计更有效的聊天机器人安全防御措施铺平道路。

真实的人类对话数据是复杂，异质和嘈杂的，从该数据中构建开放域对话系统仍然是一项艰巨的任务。实际上，此类对话数据仍然包含大量信息和知识，但是，它们没有得到充分探索。在本文中，我们展示了现有的开放域对话生成方法，这些方法记住上下文响应配对的数据，并使用自动回归或编码模型模型不利于培训数据。与当前的方法不同，使用外部知识，我们探索了一个检索生成培训框架，该培训框架可以通过将它们视为“证据”来利用异质和嘈杂的培训数据。特别是，我们使用Bertscore进行检索，这给出了证据和一代的更好品质。公开可用数据集的实验表明，我们的方法可以帮助模型产生更好的响应，即使此类培训数据通常会留下深刻的印象为低质量数据。这种性能增益与通过扩大训练组更好的改进的绩效增益相当，甚至更好。我们还发现，模型性能与检索到的证据的相关性有正相关。此外，我们的方法在零拍实验上表现良好，这表明我们的方法对现实世界数据可能更强大。

本文对过去二十年来对自然语言生成（NLG）的研究提供了全面的审查，特别是与数据到文本生成和文本到文本生成深度学习方法有关，以及NLG的新应用技术。该调查旨在（a）给出关于NLG核心任务的最新综合，以及该领域采用的建筑;（b）详细介绍各种NLG任务和数据集，并提请注意NLG评估中的挑战，专注于不同的评估方法及其关系;（c）强调一些未来的强调和相对近期的研究问题，因为NLG和其他人工智能领域的协同作用而增加，例如计算机视觉，文本和计算创造力。

Natural Language Generation (NLG) represents a large collection of tasks in the field of NLP. While many of these tasks have been tackled well by the cross-entropy (CE) loss, the task of dialog generation poses a few unique challenges for this loss function. First, CE loss assumes that for any given input, the only possible output is the one available as the ground truth in the training dataset. In general, this is not true for any task, as there can be multiple semantically equivalent sentences, each with a different surface form. This problem gets exaggerated further for the dialog generation task, as there can be multiple valid responses (for a given context) that not only have different surface forms but are also not semantically equivalent. Second, CE loss does not take the context into consideration while processing the response and, hence, it treats all ground truths with equal importance irrespective of the context. But, we may want our final agent to avoid certain classes of responses (e.g. bland, non-informative or biased responses) and give relatively higher weightage for more context-specific responses. To circumvent these shortcomings of the CE loss, in this paper, we propose a novel loss function, CORAL, that directly optimizes recently proposed estimates of human preference for generated responses. Using CORAL, we can train dialog generation models without assuming non-existence of response other than the ground-truth. Also, the CORAL loss is computed based on both the context and the response. Extensive comparisons on two benchmark datasets show that the proposed methods outperform strong state-of-the-art baseline models of different sizes.