尽管对抽象中的英语句子进行了广泛的研究,但是通过自动度量标准与金图相比,它与金图类进行了比较,但是统一图表表示的全文解析缺乏定义明确的表示和评估。利用以前的工作中的超级信托级别注释,我们介绍了一种用于导出统一图形表示的简单算法,避免了从合并不合并和缺乏连贯性信息丢失的陷阱。接下来,我们描述了对Swatch度量标准的改进,使其易于进行比较文档级图形,并使用它重新评估最佳已发布的文档级AMR解析器。我们还提出了一种与COREREFER解决系统的顶部组合的管道方法,为未来的研究提供了强大的基线。
translated by 谷歌翻译
At the foundation of scientific evaluation is the labor-intensive process of peer review. This critical task requires participants to consume vast amounts of highly technical text. Prior work has annotated different aspects of review argumentation, but discourse relations between reviews and rebuttals have yet to be examined. We present DISAPERE, a labeled dataset of 20k sentences contained in 506 review-rebuttal pairs in English, annotated by experts. DISAPERE synthesizes label sets from prior work and extends them to include fine-grained annotation of the rebuttal sentences, characterizing their context in the review and the authors' stance towards review arguments. Further, we annotate every review and rebuttal sentence. We show that discourse cues from rebuttals can shed light on the quality and interpretation of reviews. Further, an understanding of the argumentative strategies employed by the reviewers and authors provides useful signal for area chairs and other decision makers.
translated by 谷歌翻译
超过三十年,研究人员已经开发和分析了潜伏树诱导的方法作为无监督句法解析的方法。尽管如此,与其监督的对应物相比,现代系统仍然不足以使其具有任何实际用途作为文本的结构注释。在这项工作中,我们提出了一种技术,该技术以跨度约束(即短语包围)的形式使用远端监督,以提高在无监督选项解析中的性能。使用相对少量的跨度约束,我们可以大大提高Diora的输出,这是一个已经竞争的无监督解析系统。与完整的解析树注释相比,可以通过最小的努力来获取跨度约束,例如使用从维基百科派生的词典,以查找确切的文本匹配。我们的实验显示了基于实体的跨度约束,提高了英语WSJ Penn TreeBank的选区分析超过5 F1。此外,我们的方法延伸到跨度约束易于实现的任何域,以及作为一个案例研究,我们通过从工艺数据集解析生物医学文本来证明其有效性。
translated by 谷歌翻译
例如,查询是一个众所周知的信息检索任务,其中由用户选择文档作为搜索查询,目标是从大集合中检索相关文档。但是,文档通常涵盖主题的多个方面。要解决此方案,我们将通过示例介绍面位查询的任务,其中用户还可以指定除输入查询文档之外的更精细的粗体方面。我们专注于在科学文献搜索中的应用。我们设想能够沿着专门选择的修辞结构元素作为对此问题的一种解决方案来检索类似于查询科学纸的科学论文。在这项工作中,我们称之为方面的修辞结构元素,表明了科学论文的目标,方法或结果。我们介绍并描述了一个专家注释的测试集合,以评估培训的型号以执行此任务。我们的测试收集包括一个不同的50套英文查询文件,从计算语言学和机器学习场所绘制。我们仔细遵循TREC用于深度-K池(k = 100或250)使用的注释指南,结果数据收集包括具有高注释协议的分级相关性分数。在我们的数据集中评估的最先进模型显示出进一步的工作中的显着差距。可以在此处访问我们的数据集:https://github.com/iesl/csfcube
translated by 谷歌翻译
本文档提供了SNACS的详细语言描述(Adposition和Case Supersenses的语义网络; Schneider等,2018),这是52个语义标签(“ Supersenses”)的库存,这些标签(“ Supersenses”)表征了在某种程度上使用ADIP定位和案例标记的使用。粒度水平,如Streusle语料库中所示(https://github.com/nert-nlp/streusle/;版本4.5 track track track offelines guidelines guidelines版本2.6)。尽管SNACS的库存渴望成为普遍的,但该文档是特定于英语的。其他语言的文档将单独发布。版本2是Schneider等人对英语提出的超音库存的修订。 (2015,2016)(此后为“ V1”),这又基于以前的计划。本清单是在对英语的V1语料库注释进行广泛审查后开发的,以及以前未分析的属格案例所有人(Blodgett和Schneider,2018年),并考虑了希伯来语,印地语,韩国和德国的定义和案例现象的考虑。 Hwang等。 (2017)介绍了V2方案的理论基础。 Schneider等。 (2018)总结了该方案,其应用于英语语料库数据以及自动歧义任务。刘等。 (2021)提供了一个英语词法语义识别标签仪,其中包括SNACS标签的输出。该文档也可以与Xposition网站上的语料库数据一起浏览(Gessler等,2022):http://www.xposition.org/
translated by 谷歌翻译
Modeling lies at the core of both the financial and the insurance industry for a wide variety of tasks. The rise and development of machine learning and deep learning models have created many opportunities to improve our modeling toolbox. Breakthroughs in these fields often come with the requirement of large amounts of data. Such large datasets are often not publicly available in finance and insurance, mainly due to privacy and ethics concerns. This lack of data is currently one of the main hurdles in developing better models. One possible option to alleviating this issue is generative modeling. Generative models are capable of simulating fake but realistic-looking data, also referred to as synthetic data, that can be shared more freely. Generative Adversarial Networks (GANs) is such a model that increases our capacity to fit very high-dimensional distributions of data. While research on GANs is an active topic in fields like computer vision, they have found limited adoption within the human sciences, like economics and insurance. Reason for this is that in these fields, most questions are inherently about identification of causal effects, while to this day neural networks, which are at the center of the GAN framework, focus mostly on high-dimensional correlations. In this paper we study the causal preservation capabilities of GANs and whether the produced synthetic data can reliably be used to answer causal questions. This is done by performing causal analyses on the synthetic data, produced by a GAN, with increasingly more lenient assumptions. We consider the cross-sectional case, the time series case and the case with a complete structural model. It is shown that in the simple cross-sectional scenario where correlation equals causation the GAN preserves causality, but that challenges arise for more advanced analyses.
translated by 谷歌翻译
KL-regularized reinforcement learning from expert demonstrations has proved successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks. However, we show that KL-regularized reinforcement learning with behavioral reference policies derived from expert demonstrations can suffer from pathological training dynamics that can lead to slow, unstable, and suboptimal online learning. We show empirically that the pathology occurs for commonly chosen behavioral policy classes and demonstrate its impact on sample efficiency and online policy performance. Finally, we show that the pathology can be remedied by non-parametric behavioral reference policies and that this allows KL-regularized reinforcement learning to significantly outperform state-of-the-art approaches on a variety of challenging locomotion and dexterous hand manipulation tasks.
translated by 谷歌翻译
Scientists and philosophers have debated whether humans can trust advanced artificial intelligence (AI) agents to respect humanity's best interests. Yet what about the reverse? Will advanced AI agents trust humans? Gauging an AI agent's trust in humans is challenging because--absent costs for dishonesty--such agents might respond falsely about their trust in humans. Here we present a method for incentivizing machine decisions without altering an AI agent's underlying algorithms or goal orientation. In two separate experiments, we then employ this method in hundreds of trust games between an AI agent (a Large Language Model (LLM) from OpenAI) and a human experimenter (author TJ). In our first experiment, we find that the AI agent decides to trust humans at higher rates when facing actual incentives than when making hypothetical decisions. Our second experiment replicates and extends these findings by automating game play and by homogenizing question wording. We again observe higher rates of trust when the AI agent faces real incentives. Across both experiments, the AI agent's trust decisions appear unrelated to the magnitude of stakes. Furthermore, to address the possibility that the AI agent's trust decisions reflect a preference for uncertainty, the experiments include two conditions that present the AI agent with a non-social decision task that provides the opportunity to choose a certain or uncertain option; in those conditions, the AI agent consistently chooses the certain option. Our experiments suggest that one of the most advanced AI language models to date alters its social behavior in response to incentives and displays behavior consistent with trust toward a human interlocutor when incentivized.
translated by 谷歌翻译
The cooperation of a human pilot with an autonomous agent during flight control realizes parallel autonomy. A parallel-autonomous system acts as a guardian that significantly enhances the robustness and safety of flight operations in challenging circumstances. Here, we propose an air-guardian concept that facilitates cooperation between an artificial pilot agent and a parallel end-to-end neural control system. Our vision-based air-guardian system combines a causal continuous-depth neural network model with a cooperation layer to enable parallel autonomy between a pilot agent and a control system based on perceived differences in their attention profile. The attention profiles are obtained by computing the networks' saliency maps (feature importance) through the VisualBackProp algorithm. The guardian agent is trained via reinforcement learning in a fixed-wing aircraft simulated environment. When the attention profile of the pilot and guardian agents align, the pilot makes control decisions. If the attention map of the pilot and the guardian do not align, the air-guardian makes interventions and takes over the control of the aircraft. We show that our attention-based air-guardian system can balance the trade-off between its level of involvement in the flight and the pilot's expertise and attention. We demonstrate the effectivness of our methods in simulated flight scenarios with a fixed-wing aircraft and on a real drone platform.
translated by 谷歌翻译
As demand for large corpora increases with the size of current state-of-the-art language models, using web data as the main part of the pre-training corpus for these models has become a ubiquitous practice. This, in turn, has introduced an important challenge for NLP practitioners, as they are now confronted with the task of developing highly optimized models and pipelines for pre-processing large quantities of textual data, which implies, effectively classifying and filtering multilingual, heterogeneous and noisy data, at web scale. One of the main components of this pre-processing step for the pre-training corpora of large language models, is the removal of adult and harmful content. In this paper we explore different methods for detecting adult and harmful of content in multilingual heterogeneous web data. We first show how traditional methods in harmful content detection, that seemingly perform quite well in small and specialized datasets quickly break down when confronted with heterogeneous noisy web data. We then resort to using a perplexity based approach but with a twist: Instead of using a so-called "clean" corpus to train a small language model and then use perplexity so select the documents with low perplexity, i.e., the documents that resemble this so-called "clean" corpus the most. We train solely with adult and harmful textual data, and then select the documents having a perplexity value above a given threshold. This approach will virtually cluster our documents into two distinct groups, which will greatly facilitate the choice of the threshold for the perplexity and will also allow us to obtain higher precision than with the traditional classification methods for detecting adult and harmful content.
translated by 谷歌翻译