Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community, but have not received as much attention as lower-level tasks like speech and speaker recognition. In particular, there are not nearly as many SLU task benchmarks, and many of the existing ones use data that is not freely available to all researchers. Recent work has begun to introduce such benchmark datasets for several tasks. In this work, we introduce several new annotated SLU benchmark tasks based on freely available speech data, which complement existing benchmarks and address gaps in the SLU evaluation landscape. We contribute four tasks: question answering and summarization involve inference over longer speech sequences; named entity localization addresses the speech-specific task of locating the targeted content in the signal; dialog act classification identifies the function of a given speech utterance. We follow the blueprint of the Spoken Language Understanding Evaluation (SLUE) benchmark suite. In order to facilitate the development of SLU models that leverage the success of pre-trained speech representations, we will be publishing for each task (i) annotations for a relatively small fine-tuning set, (ii) annotated development and test sets, and (iii) baseline models for easy reproducibility and comparisons. In this work, we present the details of data collection and annotation and the performance of the baseline models. We also perform sensitivity analysis of pipeline models' performance (speech recognizer + text model) to the speech recognition accuracy, using more than 20 state-of-the-art speech recognition models.

Many self-supervised speech models, varying in their pre-training objective, input modality, and pre-training data, have been proposed in the last few years. Despite impressive empirical successes on downstream tasks, we still have a limited understanding of the properties encoded by the models and the differences across models. In this work, we examine the intermediate representations for a variety of recent models. Specifically, we measure acoustic, phonetic, and word-level properties encoded in individual layers, using a lightweight analysis tool based on canonical correlation analysis (CCA). We find that these properties evolve across layers differently depending on the model, and the variations relate to the choice of pre-training objective. We further investigate the utility of our analyses for downstream tasks by comparing the property trends with performance on speech recognition and spoken language understanding tasks. We discover that CCA trends provide reliable guidance to choose layers of interest for downstream tasks and that single-layer performance often matches or improves upon using all layers, suggesting implications for more efficient use of pre-trained models.

口语语言理解（SLU）任务涉及从语音音频信号映射到语义标签。鉴于此类任务的复杂性，可能预期良好的性能需要大量标记的数据集，这很难为每个新任务和域收集。但是，最近的自我监督讲话表现的进步使得考虑使用有限标记的数据学习SLU模型是可行的。在这项工作中，我们专注于低资源讨论（ner）并解决问题：超越自我监督的预培训，我们如何使用未为任务注释的外部语音和/或文本数据？我们借鉴了各种方法，包括自我训练，知识蒸馏和转移学习，并考虑其对端到端模型和管道（语音识别后跟文本型号）的适用性。我们发现，这些方法中的几种方法可以在资源受限的环境中提高绩效，超出了训练有素的表示的福利。与事先工作相比，我们发现改进的F1分数高达16％。虽然最好的基线模型是一种管道方法，但使用外部数据时最终通过端到端模型实现的最佳性能。我们提供了详细的比较和分析，例如，端到端模型能够专注于更加立列人的单词。

通过共享数据集和基准，已经促进了语音处理的进展。历史上，这些都集中在自动语音识别（ASR），扬声器标识或其他较低级别的任务上。兴趣在更高层次的口语中越来越多，理解任务，包括使用端到端模型，但是此类任务的注释数据集较少。与此同时，最近的工作显示了预先培训通用表示的可能性，然后使用相对较少标记的数据进行微调的多个任务。我们建议为口语语言理解（屠宰）创建一套基准任务，由有限尺寸标记的培训集和相应的评估集组成。该资源将允许研究界跟踪进度，评估高级任务的预先接受预期的表示，并研究开放的问题，例如管道与端到端方法的实用性。我们介绍了雪橇基准套件的第一阶段，包括指定实体识别，情感分析和相应数据集上的ASR。我们专注于自然产生的（未读取或综合）语音和自由可用的数据集。我们为VoxceReb和Voxpopuli数据集的子集提供新的转录和注释，基线模型的评估指标和结果，以及重现基线的开源工具包，并评估新模型。

Recently proposed self-supervised learning approaches have been successful for pre-training speech representation models. The utility of these learned representations has been observed empirically, but not much has been studied about the type or extent of information encoded in the pre-trained representations themselves. Developing such insights can help understand the capabilities and limits of these models and enable the research community to more efficiently develop their usage for downstream applications. In this work, we begin to fill this gap by examining one recent and successful pre-trained model (wav2vec 2.0), via its intermediate representation vectors, using a suite of analysis tools. We use the metrics of canonical correlation, mutual information, and performance on simple downstream tasks with non-parametric probes, in order to (i) query for acoustic and linguistic information content, (ii) characterize the evolution of information across model layers, and (iii) understand how fine-tuning the model for automatic speech recognition (ASR) affects these observations. Our findings motivate modifying the fine-tuning protocol for ASR, which produces improved word error rates in a low-resource setting.

Participants in political discourse employ rhetorical strategies -- such as hedging, attributions, or denials -- to display varying degrees of belief commitments to claims proposed by themselves or others. Traditionally, political scientists have studied these epistemic phenomena through labor-intensive manual content analysis. We propose to help automate such work through epistemic stance prediction, drawn from research in computational semantics, to distinguish at the clausal level what is asserted, denied, or only ambivalently suggested by the author or other mentioned entities (belief holders). We first develop a simple RoBERTa-based model for multi-source stance predictions that outperforms more complex state-of-the-art modeling. Then we demonstrate its novel application to political science by conducting a large-scale analysis of the Mass Market Manifestos corpus of U.S. political opinion books, where we characterize trends in cited belief holders -- respected allies and opposed bogeymen -- across U.S. political ideologies.

Mixup is a popular data augmentation technique based on creating new samples by linear interpolation between two given data samples, to improve both the generalization and robustness of the trained model. Knowledge distillation (KD), on the other hand, is widely used for model compression and transfer learning, which involves using a larger network's implicit knowledge to guide the learning of a smaller network. At first glance, these two techniques seem very different, however, we found that ``smoothness" is the connecting link between the two and is also a crucial attribute in understanding KD's interplay with mixup. Although many mixup variants and distillation methods have been proposed, much remains to be understood regarding the role of a mixup in knowledge distillation. In this paper, we present a detailed empirical study on various important dimensions of compatibility between mixup and knowledge distillation. We also scrutinize the behavior of the networks trained with a mixup in the light of knowledge distillation through extensive analysis, visualizations, and comprehensive experiments on image classification. Finally, based on our findings, we suggest improved strategies to guide the student network to enhance its effectiveness. Additionally, the findings of this study provide insightful suggestions to researchers and practitioners that commonly use techniques from KD. Our code is available at https://github.com/hchoi71/MIX-KD.

资源说明框架（RDF）和属性图（PG）是表示，存储和查询图数据的两个最常用的数据模型。我们提出了表达推理图存储（ERGS） - 构建在Janusgraph（属性图存储）顶部的图存储，该图还允许存储和查询RDF数据集。首先，我们描述了如何将RDF数据转换为属性图表示，然后描述将SPARQL查询转换为一系列Gremlin遍历的查询翻译模块。因此，开发的转换器和翻译器可以允许任何Apache TinkerPop符合图形数据库存储和查询RDF数据集。我们证明了使用JanusGraph作为基本属性图存储的建议方法的有效性，并将其性能与标准RDF系统进行比较。

正如GPT-3和T5所证明的那样，随着参数空间变得越来越大，变压器具有能力。但是，对于需要大量知识的任务，非参数存储器允许模型在计算成本和GPU内存需求的次线性增加中急剧增长。诸如RAG和Realm之类的最新模型已将检索引入条件生成。这些模型结合了从一系列语料库中的神经初始检索。我们基于这一研究，提出了RE2G，该研究将神经初始检索和重新融合到基于巴特的序列到序列的生成中。我们的阅读方法还允许从无与伦比分数的来源合并结果，从而实现BM25和神经初始检索的合奏。为了训练我们的系统端到端，我们引入了一种新颖的知识蒸馏变体，以在目标序列输出上仅使用地面真理来训练初始检索，重读者和生成。我们在四个不同的任务中发现了很大的收益：零击插槽填充，问答，事实检查和对话，相对增长了9％至34％，比以前的苏格兰短裙排行榜上的最先前的排行榜相比。我们将代码作为开源提供，网址为https://github.com/ibm/kgi-slot-filling/tree/re2g。

与数字计算相比，模拟计算具有吸引力，因为它可以达到更高的计算密度和更高的能源效率。但是，与数字电路不同，由于晶体管偏置偏差，温度变化和有限的动态范围的差异，传统的模拟计算电路不能轻易地在不同的过程节点上映射。在这项工作中，我们概括了先前报道的基于边缘传播的模拟计算框架，用于设计新颖的\ textit {基于形状的模拟计算}（S-AC）电路，这些电路可以轻松地在不同的过程节点上交叉映射。与数字设计类似的S-AC设计也可以缩放以获得精确，速度和功率。作为概念验证，我们展示了实现机器学习（ML）体系结构中通常使用的数学功能的S-AC电路的几个示例。使用电路模拟，我们证明了电路输入/输出特性从平面CMOS 180NM工艺映射到FinFET 7NM工艺时保持健壮。同样，使用基准数据集，我们证明了基于S-AC的神经网络的分类精度在两个过程中映射到温度变化时仍然坚固。