基于价值的深度增强学习（RL）算法遭受主要由函数近似和时间差（TD）学习引起的估计偏差。此问题会引起故障状态 - 动作值估计，因此损害了学习算法的性能和鲁棒性。尽管提出了几种技术来解决，但学习算法仍然遭受这种偏差。在这里，我们介绍一种技术，该技术使用经验重放机制消除了截止策略连续控制算法中的估计偏差。我们在加权双延迟深度确定性政策梯度算法中自适应地学习加权超参数β。我们的方法名为Adaptive-WD3（AWD3）。我们展示了Openai健身房的连续控制环境，我们的算法匹配或优于最先进的脱离政策政策梯度学习算法。

经验重放机制允许代理多次使用经验。在以前的作品中，过渡的抽样概率根据其重要性进行调整。重新分配采样概率在每次迭代后的重传缓冲器的每个过渡是非常低效的。因此，经验重播优先算法重新计算时，相应的过渡进行采样，以获得计算效率转变的意义。然而，过渡的重要性水平动态变化的政策和代理人的价值函数被更新。此外，经验回放存储转换由可显著从代理的最新货币政策偏离剂的以前的政策产生。从代理引线的最新货币政策更关闭策略更新，这是有害的代理高偏差。在本文中，我们开发了一种新的算法，通过KL散度批次优先化体验重播（KLPER），其优先批次转换的，而不是直接优先每个过渡。此外，为了减少更新的截止policyness，我们的算法选择一个批次中的某一批次的数量和力量的通过很有可能是代理的最新货币政策所产生的一批学习代理。我们结合与深确定性政策渐变和Twin算法延迟深确定性政策渐变，并评估它在不同的连续控制任务。 KLPER提供培训期间的抽样效率，最终表现和政策的稳定性方面有前途的深确定性的连续控制算法的改进。

We investigate ensemble methods for prediction in an online setting. Unlike all the literature in ensembling, for the first time, we introduce a new approach using a meta learner that effectively combines the base model predictions via using a superset of the features that is the union of the base models' feature vectors instead of the predictions themselves. Here, our model does not use the predictions of the base models as inputs to a machine learning algorithm, but choose the best possible combination at each time step based on the state of the problem. We explore three different constraint spaces for the ensembling of the base learners that linearly combines the base predictions, which are convex combinations where the components of the ensembling vector are all nonnegative and sum up to 1; affine combinations where the weight vector components are required to sum up to 1; and the unconstrained combinations where the components are free to take any real value. The constraints are both theoretically analyzed under known statistics and integrated into the learning procedure of the meta learner as a part of the optimization in an automated manner. To show the practical efficiency of the proposed method, we employ a gradient-boosted decision tree and a multi-layer perceptron separately as the meta learners. Our framework is generic so that one can use other machine learning architectures as the ensembler as long as they allow for a custom differentiable loss for minimization. We demonstrate the learning behavior of our algorithm on synthetic data and the significant performance improvements over the conventional methods over various real life datasets, extensively used in the well-known data competitions. Furthermore, we openly share the source code of the proposed method to facilitate further research and comparison.

Adversarial training has been empirically shown to be more prone to overfitting than standard training. The exact underlying reasons still need to be fully understood. In this paper, we identify one cause of overfitting related to current practices of generating adversarial samples from misclassified samples. To address this, we propose an alternative approach that leverages the misclassified samples to mitigate the overfitting problem. We show that our approach achieves better generalization while having comparable robustness to state-of-the-art adversarial training methods on a wide range of computer vision, natural language processing, and tabular tasks.

Adversarial training is widely acknowledged as the most effective defense against adversarial attacks. However, it is also well established that achieving both robustness and generalization in adversarially trained models involves a trade-off. The goal of this work is to provide an in depth comparison of different approaches for adversarial training in language models. Specifically, we study the effect of pre-training data augmentation as well as training time input perturbations vs. embedding space perturbations on the robustness and generalization of BERT-like language models. Our findings suggest that better robustness can be achieved by pre-training data augmentation or by training with input space perturbation. However, training with embedding space perturbation significantly improves generalization. A linguistic correlation analysis of neurons of the learned models reveal that the improved generalization is due to `more specialized' neurons. To the best of our knowledge, this is the first work to carry out a deep qualitative analysis of different methods of generating adversarial examples in adversarial training of language models.

Automatic keyword extraction (AKE) has gained more importance with the increasing amount of digital textual data that modern computing systems process. It has various applications in information retrieval (IR) and natural language processing (NLP), including text summarisation, topic analysis and document indexing. This paper proposes a simple but effective post-processing-based universal approach to improve the performance of any AKE methods, via an enhanced level of semantic-awareness supported by PoS-tagging. To demonstrate the performance of the proposed approach, we considered word types retrieved from a PoS-tagging step and two representative sources of semantic information -- specialised terms defined in one or more context-dependent thesauri, and named entities in Wikipedia. The above three steps can be simply added to the end of any AKE methods as part of a post-processor, which simply re-evaluate all candidate keywords following some context-specific and semantic-aware criteria. For five state-of-the-art (SOTA) AKE methods, our experimental results with 17 selected datasets showed that the proposed approach improved their performances both consistently (up to 100\% in terms of improved cases) and significantly (between 10.2\% and 53.8\%, with an average of 25.8\%, in terms of F1-score and across all five methods), especially when all the three enhancement steps are used. Our results have profound implications considering the ease to apply our proposed approach to any AKE methods and to further extend it.

The horseshoe prior is known to possess many desirable properties for Bayesian estimation of sparse parameter vectors, yet its density function lacks an analytic form. As such, it is challenging to find a closed-form solution for the posterior mode. Conventional horseshoe estimators use the posterior mean to estimate the parameters, but these estimates are not sparse. We propose a novel expectation-maximisation (EM) procedure for computing the MAP estimates of the parameters in the case of the standard linear model. A particular strength of our approach is that the M-step depends only on the form of the prior and it is independent of the form of the likelihood. We introduce several simple modifications of this EM procedure that allow for straightforward extension to generalised linear models. In experiments performed on simulated and real data, our approach performs comparable, or superior to, state-of-the-art sparse estimation methods in terms of statistical performance and computational cost.

We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increasingly used interchangeably during fluent speech. Such CS complicates traditional speech recognition and translation, as we must recognize which language was spoken first and then apply a language-dependent recognizer and subsequent translation component to generate the desired target language output. Such a pipeline introduces latency and errors. In this paper, we eliminate the need for that, by treating speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language. LAST delivers comparable recognition and speech translation accuracy in monolingual usage, while reducing latency and error rate considerably when CS is observed.

在网上进行的虚假信息广泛传播，包括错误信息和虚假信息已成为我们高度数字化和全球化社会的主要问题。已经进行了大量研究来更好地理解在线虚假信息的不同方面，例如不同参与者的行为和传播模式，以及使用技术和社会技术手段更好地检测和预防此类信息。在线检测和揭穿虚假信息的一种主要方法是使用人类事实检查器，这些事实检查器可以通过自动化工具来帮助。尽管进行了大量研究，但我们注意到缺乏描述虚假信息和事实检查的复杂生态系统的概念模型存在很大的差距。在本文中，我们报告了此类生态系统的第一批图形模型，这些模型重点介绍了在多种情况下在线的虚假信息，包括传统媒体和用户生成的内容。拟议的模型涵盖了广泛的实体类型和关系，可以成为研究人员和从业者在线研究虚假信息以及事实检查的效果的新工具。

AI的最新进展，尤其是深度学习，导致创建新的现实合成媒体（视频，图像和音频）以及对现有媒体的操纵的创建显着增加，这导致了新术语的创建。 'deepfake'。基于英语和中文中的研究文献和资源，本文对Deepfake进行了全面的概述，涵盖了这一新兴概念的多个重要方面，包括1）不同的定义，2）常用的性能指标和标准以及3）与DeepFake相关的数据集，挑战，比赛和基准。此外，该论文还报告了2020年和2021年发表的12条与DeepFake相关的调查论文的元评估，不仅关注上述方面，而且集中在对关键挑战和建议的分析上。我们认为，就涵盖的各个方面而言，本文是对深层的最全面评论，也是第一个涵盖英语和中国文学和资源的文章。