现有摘要系统主要生成纯粹依赖源文档内容的摘要。但是，即使对于人类，我们通常需要一些引用或示例，帮助我们充分了解源文档并以特定格式写入摘要。但是如何找到高质量的样式，并将它们纳入总结系统仍然挑战和探索。在本文中，我们提出了一种由致密的猎犬和摘要提升的新型检索增强的抽象概要框架。首先，检索几个密切相关的示例作为补充输入，以帮助生成模型更全面地了解文本。此外，检索的示例也可以在引导模型以捕获特定语料库的写入风格中起作用。我们在多个域和两个骨干型号的各种摘要数据集上验证我们的方法：BERT和BART。结果表明，与强大的预训练模型相比，我们的框架在胭脂-1分数中获得了1.38〜4.66的显着改善，并在账单上实现了新的最先进。人类评估表明我们的检索增强模型可以更好地捕获特定于域的书写风格。

用于提取和抽象性摘要系统的传统培训范例始终仅使用令牌级别或句子级培训目标。但是，始终从摘要级别评估输出摘要，从而导致培训和评估的不一致。在本文中，我们提出了一个基于对比度学习的重新排列框架，用于一阶段的摘要，称为COLO。通过建模对比目标，我们表明摘要模型能够根据摘要级别的分数直接生成摘要，而无需其他模块和参数。广泛的实验表明，CORO在CNN/DailyMail基准测试中提高了单阶段系统的提取和抽象结果，将其提高到44.58和46.33 Rouge-1得分，同时保留了参数效率和推断效率。与最先进的多阶段系统相比，我们节省了100多个GPU训练时间，并在推理期间获得3〜8加速比，同时保持可比的结果。

文本摘要的重写方法结合了提取性和抽象的方法，使用抽象模型提高了提取性摘要的简洁性和可读性。退出重写系统将每个提取性句子作为唯一的输入，它相对集中，但可能会失去必要的背景知识和话语上下文。在本文中，我们调查了上下文化的重写，该重写消耗了整个文档并考虑了摘要上下文。我们将上下文重写正式化为具有组标签对齐的SEQ2SEQ，将组标签引入了模拟对齐方式的解决方案，并通过基于内容的地址来识别提取句子。结果表明，我们的方法显着优于非上下文重写系统，而无需加强学习，从而在多个提取器上实现了胭脂分数的强烈改进。

诸如学术文章和商业报告之类的长期文件一直是详细说明重要问题和需要额外关注的复杂主题的标准格式。自动汇总系统可以有效地将长文档置于简短而简洁的文本中，以封装最重要的信息，从而在帮助读者的理解中很重要。最近，随着神经体系结构的出现，已经做出了重大的研究工作，以推动自动文本摘要系统，以及有关将这些系统扩展到长期文档领域的挑战的大量研究。在这项调查中，我们提供了有关长期文档摘要的研究的全面概述，以及其研究环境的三个主要组成部分的系统评估：基准数据集，汇总模型和评估指标。对于每个组成部分，我们在长期汇总的背景下组织文献，并进行经验分析，以扩大有关当前研究进度的观点。实证分析包括一项研究基准数据集的内在特征，摘要模型的多维分析以及摘要评估指标的综述。根据总体发现，我们通过提出可能在这个快速增长的领域中提出未来探索的方向来得出结论。

Text summarization is a user-preference based task, i.e., for one document, users often have different priorities for summary. As a key aspect of customization in summarization, granularity is used to measure the semantic coverage between the summary and source document. However, developing systems that can generate summaries with customizable semantic coverage is still an under-explored topic. In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum. We take events as the basic semantic units of the source documents and propose to rank these events by their salience. We also develop a model to summarize input documents with given events as anchors and hints. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner. Meanwhile, we annotate a new benchmark GranuDUC that contains multiple summaries at different granularities for each document cluster. Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines. Further, by exploiting the event information, GranuSum also exhibits state-of-the-art performance under the conventional unsupervised abstractive setting. Dataset for this paper can be found at: https://github.com/maszhongming/GranuDUC

Nowadays, time-stamped web documents related to a general news query floods spread throughout the Internet, and timeline summarization targets concisely summarizing the evolution trajectory of events along the timeline. Unlike traditional document summarization, timeline summarization needs to model the time series information of the input events and summarize important events in chronological order. To tackle this challenge, in this paper, we propose a Unified Timeline Summarizer (UTS) that can generate abstractive and extractive timeline summaries in time order. Concretely, in the encoder part, we propose a graph-based event encoder that relates multiple events according to their content dependency and learns a global representation of each event. In the decoder part, to ensure the chronological order of the abstractive summary, we propose to extract the feature of event-level attention in its generation process with sequential information remained and use it to simulate the evolutionary attention of the ground truth summary. The event-level attention can also be used to assist in extracting summary, where the extracted summary also comes in time sequence. We augment the previous Chinese large-scale timeline summarization dataset and collect a new English timeline dataset. Extensive experiments conducted on these datasets and on the out-of-domain Timeline 17 dataset show that UTS achieves state-of-the-art performance in terms of both automatic and human evaluations.

对比学习模型在无监督的视觉表示学习中取得了巨大成功，这使得相同图像的不同视图的特征表示之间的相似性最大化，同时最小化不同图像的视图的特征表示之间的相似性。在文本摘要中，输出摘要是输入文档的较短形式，它们具有类似的含义。在本文中，我们提出了对监督抽象文本摘要的对比学习模型，在那里我们查看文档，它的金摘要及其模型生成的摘要，与相同的平均表示的不同视图，并在培训期间最大化它们之间的相似性。我们在三个不同的摘要数据集上改进了一个强序列到序列文本生成模型（即，BART）。人类评估还表明，与其对应物相比，我们的模型达到了更好的忠实性评级，没有对比的目标。

上下文：堆栈溢出对于寻求编程问题答案的软件开发人员非常有帮助。先前的研究表明，越来越多的问题质量低，因此从潜在的答案者那里获得了更少的关注。 Gao等。提出了一个基于LSTM的模型（即BilstM-CC），以自动从代码片段中生成问题标题，以提高问题质量。但是，只有在问题主体中使用代码段无法为标题生成提供足够的信息，而LSTMS无法捕获令牌之间的远程依赖性。目的：本文提出了基于深度学习的新型模型CCBERT，旨在通过充分利用整个问题主体的双模式信息来增强问题标题生成的性能。方法：CCBERT遵循编码器范式范式，并使用Codebert将问题主体编码为隐藏的表示形式，堆叠的变压器解码器以生成预测的代币，以及附加的复制注意层来完善输出分布。编码器和解码器都执行多头自我注意操作，以更好地捕获远程依赖性。本文构建了一个数据集，该数据集包含大约200,000个高质量问题，该数据从Stack Overflow正式发布的数据中滤除，以验证CCBERT模型的有效性。结果：CCBERT优于数据集上的所有基线模型。对仅代码和低资源数据集进行的实验表明，CCBERT的优势性能较小。人类评估还显示了CCBERT关于可读性和相关标准的出色表现。

Content-Controllable Summarization generates summaries focused on the given controlling signals. Due to the lack of large-scale training corpora for the task, we propose a plug-and-play module RelAttn to adapt any general summarizers to the content-controllable summarization task. RelAttn first identifies the relevant content in the source documents, and then makes the model attend to the right context by directly steering the attention weight. We further apply an unsupervised online adaptive parameter searching algorithm to determine the degree of control in the zero-shot setting, while such parameters are learned in the few-shot setting. By applying the module to three backbone summarization models, experiments show that our method effectively improves all the summarizers, and outperforms the prefix-based method and a widely used plug-and-play model in both zero- and few-shot settings. Tellingly, more benefit is observed in the scenarios when more control is needed.

Bidirectional Encoder Representations from Transformers (BERT; Devlin et al. 2019) represents the latest incarnation of pretrained language models which have recently advanced a wide range of natural language processing tasks. In this paper, we showcase how BERT can be usefully applied in text summarization and propose a general framework for both extractive and abstractive models. We introduce a novel document-level encoder based on BERT which is able to express the semantics of a document and obtain representations for its sentences. Our extractive model is built on top of this encoder by stacking several intersentence Transformer layers. For abstractive summarization, we propose a new fine-tuning schedule which adopts different optimizers for the encoder and the decoder as a means of alleviating the mismatch between the two (the former is pretrained while the latter is not). We also demonstrate that a two-staged fine-tuning approach can further boost the quality of the generated summaries. Experiments on three datasets show that our model achieves stateof-the-art results across the board in both extractive and abstractive settings. 1

Prompts with different control signals (e.g., length, keywords, etc.) can be used to control text summarization. When control signals are available, they can control the properties of generated summaries and potentially improve summarization quality (since more information are given). Unfortunately, control signals are not already available during inference time. In this paper, we propose Lotus (shorthand for Latent Prompt Tuning for Summarization), which is a single model that can be applied in both controlled and uncontrolled (without control signals) modes. During training, Lotus learns latent prompt representations from prompts with gold control signals using a contrastive learning objective. Experiments show Lotus in uncontrolled mode consistently improves upon strong (uncontrollable) summarization models across four different summarization datasets. We also demonstrate generated summaries can be controlled using prompts with user specified control tokens.

由于免费的在线百科全书具有大量内容，因此Wikipedia和Wikidata是许多自然语言处理（NLP）任务的关键，例如信息检索，知识基础构建，机器翻译，文本分类和文本摘要。在本文中，我们介绍了Wikides，这是一个新颖的数据集，用于为文本摘要问题提供Wikipedia文章的简短描述。该数据集由6987个主题上的80K英语样本组成。我们设置了一种两阶段的摘要方法 - 描述生成（I阶段）和候选排名（II阶段）作为一种依赖于转移和对比学习的强大方法。对于描述生成，与其他小规模的预训练模型相比，T5和BART表现出了优越性。通过将对比度学习与Beam Search的不同输入一起应用，基于度量的排名模型优于直接描述生成模型，在主题独立拆分和独立于主题的独立拆分中，最高可达22个胭脂。此外，第II期中的结果描述得到了人类评估的支持，其中45.33％以上，而I阶段的23.66％则支持针对黄金描述。在情感分析方面，生成的描述无法有效地从段落中捕获所有情感极性，同时从黄金描述中更好地完成此任务。自动产生的新描述减少了人类为创建它们的努力，并丰富了基于Wikidata的知识图。我们的论文对Wikipedia和Wikidata产生了实际影响，因为有成千上万的描述。最后，我们预计Wikides将成为从短段落中捕获显着信息的相关作品的有用数据集。策划的数据集可公开可用：https：//github.com/declare-lab/wikides。

Multi-document summarization (MDS) has traditionally been studied assuming a set of ground-truth topic-related input documents is provided. In practice, the input document set is unlikely to be available a priori and would need to be retrieved based on an information need, a setting we call open-domain MDS. We experiment with current state-of-the-art retrieval and summarization models on several popular MDS datasets extended to the open-domain setting. We find that existing summarizers suffer large reductions in performance when applied as-is to this more realistic task, though training summarizers with retrieved inputs can reduce their sensitivity retrieval errors. To further probe these findings, we conduct perturbation experiments on summarizer inputs to study the impact of different types of document retrieval errors. Based on our results, we provide practical guidelines to help facilitate a shift to open-domain MDS. We release our code and experimental results alongside all data or model artifacts created during our investigation.

Narrative summarization aims to produce a distilled version of a narrative to describe its most salient events and characters. Summarizing a narrative is challenging as it requires an understanding of event causality and character behaviors. To encourage research in this direction, we propose NarraSum, a large-scale narrative summarization dataset. It contains 122K narrative documents, which are collected from plot descriptions of movies and TV episodes with diverse genres, and their corresponding abstractive summaries. Experiments show that there is a large performance gap between humans and the state-of-the-art summarization models on NarraSum. We hope that this dataset will promote future research in summarization, as well as broader studies of natural language understanding and generation. The dataset is available at https://github.com/zhaochaocs/narrasum.

学术研究是解决以前从未解决过的问题的探索活动。通过这种性质，每个学术研究工作都需要进行文献审查，以区分其Novelties尚未通过事先作品解决。在自然语言处理中，该文献综述通常在“相关工作”部分下进行。鉴于研究文件的其余部分和引用的论文列表，自动相关工作生成的任务旨在自动生成“相关工作”部分。虽然这项任务是在10年前提出的，但直到最近，它被认为是作为科学多文件摘要问题的变种。然而，即使在今天，尚未标准化了自动相关工作和引用文本生成的问题。在这项调查中，我们进行了一个元研究，从问题制定，数据集收集，方法方法，绩效评估和未来前景的角度来比较相关工作的现有文献，以便为读者洞察到国家的进步 - 最内容的研究，以及如何进行未来的研究。我们还调查了我们建议未来工作要考虑整合的相关研究领域。

Recent lay language generation systems have used Transformer models trained on a parallel corpus to increase health information accessibility. However, the applicability of these models is constrained by the limited size and topical breadth of available corpora. We introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. The abstract and the corresponding lay language summary are written by domain experts, assuring the quality of our dataset. Furthermore, qualitative evaluation of expert-authored plain language summaries has revealed background explanation as a key strategy to increase accessibility. Such explanation is challenging for neural models to generate because it goes beyond simplification by adding content absent from the source. We derive two specialized paired corpora from CELLS to address key challenges in lay language generation: generating background explanations and simplifying the original abstract. We adopt retrieval-augmented models as an intuitive fit for the task of background explanation generation, and show improvements in summary quality and simplicity while maintaining factual correctness. Taken together, this work presents the first comprehensive study of background explanation for lay language generation, paving the path for disseminating scientific knowledge to a broader audience. CELLS is publicly available at: https://github.com/LinguisticAnomalies/pls_retrieval.

健康素养被出现为制定适当的健康决策和确保治疗结果的关键因素。然而，医学术语和该领域的专业语言的复杂结构使健康信息尤为难以解释。因此，迫切需要对自动化方法来提高生物医学文献的可访问性，以提高一般人群。这个问题可以作为医疗保健专业人员语言与公众的语言之间的翻译问题。在本文中，我们介绍了自动化生物医学科学评论的制定语言摘要的新任务，建设了一个数据集，以支持自动化方法的开发和评估，以提高生物医学文献的可访问性。我们对解决这项任务的各种挑战进行了分析，包括不仅对关键要点的总结，而且还概述了对背景知识和专业语言的简化的解释。我们试验最先进的摘要模型以及多种数据增强技术，并使用自动指标和人工评估评估其性能。结果表明，与专家专家专门开发的参考摘要相比，使用当代神经架构产生的自动产生的摘要可以实现有希望的质量和可读性（最佳Rouge-L为50.24和Flesch-Kincaid可读性得分为13.30）。我们还讨论了目前尝试的局限性，为未来工作提供了洞察和方向。

Current abstractive summarization systems present important weaknesses which prevent their deployment in real-world applications, such as the omission of relevant information and the generation of factual inconsistencies (also known as hallucinations). At the same time, automatic evaluation metrics such as CTC scores have been recently proposed that exhibit a higher correlation with human judgments than traditional lexical-overlap metrics such as ROUGE. In this work, we intend to close the loop by leveraging the recent advances in summarization metrics to create quality-aware abstractive summarizers. Namely, we propose an energy-based model that learns to re-rank summaries according to one or a combination of these metrics. We experiment using several metrics to train our energy-based re-ranker and show that it consistently improves the scores achieved by the predicted summaries. Nonetheless, human evaluation results show that the re-ranking approach should be used with care for highly abstractive summaries, as the available metrics are not yet sufficiently reliable for this purpose.

主题控制的摘要是一个具有广泛潜在应用的新兴研究领域。但是，现有方法受到重大局限性。首先，目前尚无针对此任务的确定评估指标。此外，现有的方法基于经常性架构，与最新的基于变压器的架构相比，这可能会大大限制其性能，同时它们还需要对模型的架构进行修改以控制主题。在这项工作中，我们提出了一种新的面向主题的评估措施，以根据生成的摘要与所需主题之间的主题亲和力自动评估生成的摘要。我们还进行了一项用户研究，以验证该措施的可靠性。最后，我们提出了简单而有力的方法，用于将主题控制的摘要要么将主题嵌入到模型的体系结构中，要么采用控制令牌来指导摘要生成。实验结果表明，与更复杂的基于嵌入的方法相比，对照令牌可以实现更好的性能，同时更快。

以查询为中心的摘要（QFS）旨在产生应答感兴趣的特定问题的摘要，从而实现更大的用户控制和个性化。虽然最近发布的数据集如QMSUM或Aquamuse，促进QFS中的研究工作，但该领域缺乏对适用建模方法的广泛空间的全面研究。在本文中，考虑到两种普遍的方法，我们对QFS进行了系统探索，探讨了QFS：两阶段的采掘解决方案和端到端模型。在这些类别中，我们调查现有方法，并呈现了在QMSUM数据集上实现最先进的性能的两个模型扩展，其边缘高达3.38 Rouge-1，3.72 Rouge-2和3.28 Rouge-L。通过定量实验，我们突出了不同模型配置之间的权衡，并探讨了摘要任务之间的转移能力。代码和检查点公开可用：https：//github.com/salesforce/query-focused-sum。