间隔分析(或间隔结合传播,IBP)是一种流行的技术,用于验证和培训可提供稳健的深度神经网络,在可靠的机器学习领域是一个根本的挑战。然而,尽管努力实质性努力,解决了这一关键挑战的进展已经停滞不前,呼吁课程是间隔算术是否是前进的可行路径。本文在分析神经网络的间隔算法的局限上,我们提出了两个基本结果。我们的主要不可能性定理表明,对于任何神经网络分类只是三个点,在这些点上有一个有效的规格,间隔分析无法证明。此外,在一个隐藏层神经网络的禁用情况下,我们显示出更强烈的不可能性结果:给定任何RADIUS $ \ alpha <1 $,有一组$ O(\ alpha ^ { - 1})$积分强大的RADIUS $ \ Alpha $,以2美元分开,可以证明没有一个隐式层网络通过间隔分析稳健地进行分类。
translated by 谷歌翻译
公平表示学习编码用户数据,以确保公平和实用性,无论下游应用。然而,学习单独公平的表示,即保证类似的个体相似地治疗,在计算机视觉等高维设置中仍然具有挑战性。在这项工作中,我们介绍了Lassi,是用于认证高维数据的个人公平的第一个代表学习方法。我们的主要洞察力是利用最近在生成建模方面的进步,以捕获生成潜空间中的类似个人。这允许通过使用对抗性训练将相似的个体覆盖相似的单独公平的表示,以最小化其表示之间的距离。最后,我们采用随机平滑,以证明类似的人在一起将类似的人映射在一起,反过来确保了下游应用的局部稳健性导致端到端的公平认证。我们对具有挑战性的真实世界形象数据的实验评估表明,我们的方法将认证的个人公平升高至60%,而不会显着影响任务效用。
translated by 谷歌翻译
我们提出了一种基于随机平滑的图像和点云进行分割的新认证方法。该方法利用一种新颖的可扩展算法进行预测和认证,该算法正确说明了多次测试,这是确保统计保证所必需的。我们方法的关键是依靠已建立的多次测试校正机制,以及弃权分类单像素或点的能力,同时仍然坚固地分割整个输入。我们对综合数据和挑战数据集的实验评估,例如Pascal环境,城市景观和Shapenet,表明我们的算法可以首次实现现实世界中的竞争精度和认证保证。我们在https://github.com/eth-sri/sementation-smoothing上提供实施。
translated by 谷歌翻译
Neural compression offers a domain-agnostic approach to creating codecs for lossy or lossless compression via deep generative models. For sequence compression, however, most deep sequence models have costs that scale with the sequence length rather than the sequence complexity. In this work, we instead treat data sequences as observations from an underlying continuous-time process and learn how to efficiently discretize while retaining information about the full sequence. As a consequence of decoupling sequential information from its temporal discretization, our approach allows for greater compression rates and smaller computational complexity. Moreover, the continuous-time approach naturally allows us to decode at different time intervals. We empirically verify our approach on multiple domains involving compression of video and motion capture sequences, showing that our approaches can automatically achieve reductions in bit rates by learning how to discretize.
translated by 谷歌翻译
This paper extends quantile factor analysis to a probabilistic variant that incorporates regularization and computationally efficient variational approximations. By means of synthetic and real data experiments it is established that the proposed estimator can achieve, in many cases, better accuracy than a recently proposed loss-based estimator. We contribute to the literature on measuring uncertainty by extracting new indexes of low, medium and high economic policy uncertainty, using the probabilistic quantile factor methodology. Medium and high indexes have clear contractionary effects, while the low index is benign for the economy, showing that not all manifestations of uncertainty are the same.
translated by 谷歌翻译
We introduce organism networks, which function like a single neural network but are composed of several neural particle networks; while each particle network fulfils the role of a single weight application within the organism network, it is also trained to self-replicate its own weights. As organism networks feature vastly more parameters than simpler architectures, we perform our initial experiments on an arithmetic task as well as on simplified MNIST-dataset classification as a collective. We observe that individual particle networks tend to specialise in either of the tasks and that the ones fully specialised in the secondary task may be dropped from the network without hindering the computational accuracy of the primary task. This leads to the discovery of a novel pruning-strategy for sparse neural networks
translated by 谷歌翻译
Overfitting is a problem in Convolutional Neural Networks (CNN) that causes poor generalization of models on unseen data. To remediate this problem, many new and diverse data augmentation methods (DA) have been proposed to supplement or generate more training data, and thereby increase its quality. In this work, we propose a new data augmentation algorithm: VoronoiPatches (VP). We primarily utilize non-linear recombination of information within an image, fragmenting and occluding small information patches. Unlike other DA methods, VP uses small convex polygon-shaped patches in a random layout to transport information around within an image. Sudden transitions created between patches and the original image can, optionally, be smoothed. In our experiments, VP outperformed current DA methods regarding model variance and overfitting tendencies. We demonstrate data augmentation utilizing non-linear re-combination of information within images, and non-orthogonal shapes and structures improves CNN model robustness on unseen data.
translated by 谷歌翻译
An optimal delivery of arguments is key to persuasion in any debate, both for humans and for AI systems. This requires the use of clear and fluent claims relevant to the given debate. Prior work has studied the automatic assessment of argument quality extensively. Yet, no approach actually improves the quality so far. Our work is the first step towards filling this gap. We propose the task of claim optimization: to rewrite argumentative claims to optimize their delivery. As an initial approach, we first generate a candidate set of optimized claims using a sequence-to-sequence model, such as BART, while taking into account contextual information. Our key idea is then to rerank generated candidates with respect to different quality metrics to find the best optimization. In automatic and human evaluation, we outperform different reranking baselines on an English corpus, improving 60% of all claims (worsening 16% only). Follow-up analyses reveal that, beyond copy editing, our approach often specifies claims with details, whereas it adds less evidence than humans do. Moreover, its capabilities generalize well to other domains, such as instructional texts.
translated by 谷歌翻译
We consider distributed learning in the presence of slow and unresponsive worker nodes, referred to as stragglers. In order to mitigate the effect of stragglers, gradient coding redundantly assigns partial computations to the worker such that the overall result can be recovered from only the non-straggling workers. Gradient codes are designed to tolerate a fixed number of stragglers. Since the number of stragglers in practice is random and unknown a priori, tolerating a fixed number of stragglers can yield a sub-optimal computation load and can result in higher latency. We propose a gradient coding scheme that can tolerate a flexible number of stragglers by carefully concatenating gradient codes for different straggler tolerance. By proper task scheduling and small additional signaling, our scheme adapts the computation load of the workers to the actual number of stragglers. We analyze the latency of our proposed scheme and show that it has a significantly lower latency than gradient codes.
translated by 谷歌翻译
With the rise of AI and automation, moral decisions are being put into the hands of algorithms that were formerly the preserve of humans. In autonomous driving, a variety of such decisions with ethical implications are made by algorithms for behavior and trajectory planning. Therefore, we present an ethical trajectory planning algorithm with a framework that aims at a fair distribution of risk among road users. Our implementation incorporates a combination of five essential ethical principles: minimization of the overall risk, priority for the worst-off, equal treatment of people, responsibility, and maximum acceptable risk. To the best of the authors' knowledge, this is the first ethical algorithm for trajectory planning of autonomous vehicles in line with the 20 recommendations from the EU Commission expert group and with general applicability to various traffic situations. We showcase the ethical behavior of our algorithm in selected scenarios and provide an empirical analysis of the ethical principles in 2000 scenarios. The code used in this research is available as open-source software.
translated by 谷歌翻译