A primary objective of news articles is to establish the factual record for an event, frequently achieved by conveying both the details of the specified event (i.e., the 5 Ws; Who, What, Where, When and Why regarding the event) and how people reacted to it (i.e., reported statements). However, existing work on news summarization almost exclusively focuses on the event details. In this work, we propose the novel task of summarizing the reactions of different speakers, as expressed by their reported statements, to a given event. To this end, we create a new multi-document summarization benchmark, SUMREN, comprising 745 summaries of reported statements from various public figures obtained from 633 news articles discussing 132 events. We propose an automatic silver training data generation approach for our task, which helps smaller models like BART achieve GPT-3 level performance on this task. Finally, we introduce a pipeline-based framework for summarizing reported speech, which we empirically show to generate summaries that are more abstractive and factual than baseline query-focused summarization approaches.

Conventional closed-world information extraction (IE) approaches rely on human ontologies to define the scope for extraction. As a result, such approaches fall short when applied to new domains. This calls for systems that can automatically infer new types from given corpora, a task which we refer to as type discovery. To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type. Then we use the similarity between inferred names to induce clusters. Observing that this abstraction-based representation is often complementary to the entity/trigger token representation, we set up these two representations as two views and design our model as a co-training framework. Our experiments on multiple relation extraction and event extraction datasets consistently show the advantage of our type abstraction approach. Code available at https://github.com/raspberryice/type-discovery-abs.

Several works have proven that finetuning is an applicable approach for debiasing contextualized word embeddings. Similarly, discrete prompts with semantic meanings have shown to be effective in debiasing tasks. With unfixed mathematical representation at the token level, continuous prompts usually surpass discrete ones at providing a pre-trained language model (PLM) with additional task-specific information. Despite this, relatively few efforts have been made to debias PLMs by prompt tuning with continuous prompts compared to its discrete counterpart. Furthermore, for most debiasing methods that alter a PLM's original parameters, a major problem is the need to not only decrease the bias in the PLM but also to ensure that the PLM does not lose its representation ability. Finetuning methods typically have a hard time maintaining this balance, as they tend to violently remove meanings of attribute words. In this paper, we propose ADEPT, a method to debias PLMs using prompt tuning while maintaining the delicate balance between removing biases and ensuring representation ability. To achieve this, we propose a new training criterion inspired by manifold learning and equip it with an explicit debiasing term to optimize prompt tuning. In addition, we conduct several experiments with regard to the reliability, quality, and quantity of a previously proposed attribute training corpus in order to obtain a clearer prototype of a certain attribute, which indicates the attribute's position and relative distances to other words on the manifold. We evaluate ADEPT on several widely acknowledged debiasing benchmarks and downstream tasks, and find that it achieves competitive results while maintaining (and in some cases even improving) the PLM's representation ability. We further visualize words' correlation before and after debiasing a PLM, and give some possible explanations for the visible effects.

Humans can classify an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen concepts to describe the new category. For example, we might describe mavens as "a kind of large birds with black feathers", so that others can use their knowledge of concepts "large birds" and "black feathers" to recognize a maven. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language explanations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses the explanations into logical structure the and then reasons along this structure on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, mainly because it performs better on tasks requiring more logical reasoning. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.

Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles. Existing methods focus on capturing the overall visual scene of each frame, ignoring fine-grained argument-level information. Inspired by the definition of events as changes of states, we propose a novel framework to detect video events by tracking the changes in the visual states of all involved arguments, which are expected to provide the most informative evidence for the extraction of video events. In order to capture the visual state changes of arguments, we decompose them into changes in pixels within objects, displacements of objects, and interactions among multiple arguments. We further propose Object State Embedding, Object Motion-aware Embedding and Argument Interaction Embedding to encode and track these changes respectively. Experiments on various video event extraction tasks demonstrate significant improvements compared to state-of-the-art models. In particular, on verb classification, we achieve 3.49% absolute gains (19.53% relative gains) in F1@5 on Video Situation Recognition.

Pre-trained language models (PLMs) achieve remarkable performance on many downstream tasks, but may fail in giving reliable estimates of their predictive uncertainty. Given the lack of a comprehensive understanding of PLMs calibration, we take a close look into this new research problem, aiming to answer two questions: (1) Do PLMs learn to become calibrated in the training process? (2) How effective are existing calibration methods? For the first question, we conduct fine-grained control experiments to study the dynamic change in PLMs' calibration performance in training. We consider six factors as control variables, including dataset difficulty, available training samples, training steps, the number of tunable parameters, model scale, and pretraining. In experiments, we observe a consistent change in calibration performance across six factors. We find that PLMs don't learn to become calibrated in training, evidenced by the continual increase in confidence, no matter the predictions are correct or not. We highlight that our finding presents some contradiction with two established conclusions: (a) Larger PLMs are more calibrated; (b) Pretraining improves model calibration. Next, we study the effectiveness of existing calibration methods in mitigating the overconfidence issue, in both in-distribution and various out-of-distribution settings. Besides unlearnable calibration methods, we adapt two recently proposed learnable methods that directly collect data to train models to have reasonable confidence estimations. Also, we propose extended learnable methods based on existing ones to further improve or maintain PLMs calibration without sacrificing the original task performance. Experimental results show that learnable methods significantly reduce PLMs' confidence in wrong predictions, and our methods exhibit superior performance compared with previous methods.

从新闻文章中提取事件的信息论点是信息提取的一个具有挑战性的问题，这需要对每个文档的全球上下文理解。尽管有关文档级提取的最新工作已经超越了单句子，并提高了端到端模型的跨句子推理能力，但它们仍然受到某些输入序列长度约束的限制，通常忽略事件之间的全局上下文。为了解决此问题，我们通过构建文档存储器存储来记录上下文事件信息，并利用它隐含，明确地帮助解码以后事件的参数，从而引入了一个新的基于全局神经生成的框架，以用于文档级事件参数提取提取文档级别的事件参数提取。经验结果表明，我们的框架的表现要优于先验方法，并且使用约束的解码设计对对抗注释的示例更为强大。 （我们的代码和资源可在https://github.com/xinyadu/memory_docie上获得研究目的。）

初始化时（OPAI）的一次性网络修剪是降低网络修剪成本的有效方法。最近，人们越来越相信数据在OPAI中是不必要的。但是，我们通过两种代表性的OPAI方法，即剪切和掌握的消融实验获得了相反的结论。具体而言，我们发现信息数据对于增强修剪性能至关重要。在本文中，我们提出了两种新颖的方法，即判别性的单发网络修剪（DOP）和超级缝制，以通过高级视觉判别图像贴片来修剪网络。我们的贡献如下。（1）广泛的实验表明OPAI是数据依赖性的。（2）超级缝线的性能明显优于基准图像网上的原始OPAI方法，尤其是在高度压缩的模型中。

由于伪造的信息广泛，事实检查引起了人们的关注。大多数事实核对方法仅仅是由于其他语言中的数据稀缺问题而侧重于英语的主张。缺乏低资源语言的事实检查数据集要求采用有效的跨语义转移技术来进行事实检查。此外，以不同语言的可信赖信息可以互补，有助于验证事实。为此，我们介绍了第一个以跨语性检索为增强的事实检查框架，该框架通过跨语言检索器汇总了从多种语言中获取的证据。鉴于缺乏具有索赔式查询的跨语性信息检索数据集，我们使用拟议的跨语性倒数式紧固任务（X-ICT）来训练检索器，这是一种自我监督的算法，该算法通过翻译一个标题来创建训练实例通道。 XICT的目标是学习跨语性检索，其中模型学会确定与给定翻译标题相对应的段落。在X-FACT数据集上，我们的方法在零击跨语言设置中比先前的系统实现了2.23％的绝对F1改进。源代码和数据可在https://github.com/khuangaf/concrete上公开获取。

在本文中，我们提出了Tetris，这是一个面向目标脚本完成的新任务。与以前的工作不同，它考虑了一个更现实，更通用的设置，其中输入不仅包括目标，还包括其他用户上下文，包括偏好和历史记录。为了使用基于知识的方法解决问题，我们介绍了任务概念图，这是一种自动从教学网站构建的知识库。不同于常识知识基础（例如ConceptNet），任务概念图架构架构介绍了专门用于完成任务的各种基于名词短语的节点。为了将这些图形集成到脚本学习中，我们设计了两种从知识库中获取概念的方法，以作为下游脚本完成的提示。在我们的基于Wikihow的数据集中，我们发现从任务概念图中合并概念会始终提高性能，并证明任务概念图的好处。此外，具有金色标准概念的模型迅速胜过基线，进一步证实了在目标脚本完成中对特定于任务知识的需求。数据集，存储库，模型和演示将公开使用，以促进对这项新任务的进一步研究。