感知哈希映射图像具有相同语义内容与相同的$ n $ bit Hash值相同的图像，同时将语义不同的图像映射到不同的哈希。这些算法在网络安全方面具有重要的应用，例如版权侵权检测，内容指纹和监视。苹果的神经哈什（Neuralhash）就是这样一种系统，旨在检测用户设备上非法内容的存在，而不会损害消费者的隐私。我们提出了令人惊讶的发现，即神经锤差是线性的，这激发了新型黑盒攻击的发展，该攻击可以（i）逃避对“非法”图像的检测，（ii）产生近乎收集，以及（iii）有关哈希德的泄漏信息。图像，全部无访问模型参数。这些脆弱性对神经哈什的安全目标构成了严重威胁；为了解决这些问题，我们建议使用经典加密标准提出一个简单的修复程序。

Apple最近透露了它的深度感知散列系统的神经枢纽，以检测文件在文件上传到其iCloud服务之前的用户设备上的儿童性滥用材料（CSAM）。关于保护用户隐私和系统可靠性的公众批评很快就会出现。本文基于神经枢纽的深度感知哈希展示了第一综合实证分析。具体而言，我们表明当前深度感知散列可能不具有稳健性。对手可以通过应用图像的略微变化来操纵散列值，或者通过基于梯度的方法或简单地执行标准图像转换，强制或预防哈希冲突来操纵。这种攻击允许恶意演员轻松利用检测系统：从隐藏滥用材料到框架无辜的用户，一切都是可能的。此外，使用散列值，仍然可以对存储在用户设备上的数据进行推断。在我们的观点中，根据我们的结果，其目前形式的深度感知散列通常不适用于强大的客户端扫描，不应从隐私角度使用。

Content scanning systems employ perceptual hashing algorithms to scan user content for illegal material, such as child pornography or terrorist recruitment flyers. Perceptual hashing algorithms help determine whether two images are visually similar while preserving the privacy of the input images. Several efforts from industry and academia propose to conduct content scanning on client devices such as smartphones due to the impending roll out of end-to-end encryption that will make server-side content scanning difficult. However, these proposals have met with strong criticism because of the potential for the technology to be misused and re-purposed. Our work informs this conversation by experimentally characterizing the potential for one type of misuse -- attackers manipulating the content scanning system to perform physical surveillance on target locations. Our contributions are threefold: (1) we offer a definition of physical surveillance in the context of client-side image scanning systems; (2) we experimentally characterize this risk and create a surveillance algorithm that achieves physical surveillance rates of >40% by poisoning 5% of the perceptual hash database; (3) we experimentally study the trade-off between the robustness of client-side image scanning systems and surveillance, showing that more robust detection of illegal material leads to increased potential for physical surveillance.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.

已知深度学习系统容易受到对抗例子的影响。特别是，基于查询的黑框攻击不需要深入学习模型的知识，而可以通过提交查询和检查收益来计算网络上的对抗示例。最近的工作在很大程度上提高了这些攻击的效率，证明了它们在当今的ML-AS-A-Service平台上的实用性。我们提出了Blacklight，这是针对基于查询的黑盒对抗攻击的新防御。推动我们设计的基本见解是，为了计算对抗性示例，这些攻击在网络上进行了迭代优化，从而在输入空间中产生了非常相似的图像查询。 Blacklight使用在概率内容指纹上运行的有效相似性引擎来检测高度相似的查询来检测基于查询的黑盒攻击。我们根据各种模型和图像分类任务对八次最先进的攻击进行评估。 Blacklight通常只有几次查询后，都可以识别所有这些。通过拒绝所有检测到的查询，即使攻击者在帐户禁令或查询拒绝之后持续提交查询，Blacklight也可以防止任何攻击完成。 Blacklight在几个强大的对策中也很强大，包括最佳的黑盒攻击，该攻击近似于效率的白色框攻击。最后，我们说明了黑光如何推广到其他域，例如文本分类。

Deep hashing has been extensively utilized in massive image retrieval because of its efficiency and effectiveness. However, deep hashing models are vulnerable to adversarial examples, making it essential to develop adversarial defense methods for image retrieval. Existing solutions achieved limited defense performance because of using weak adversarial samples for training and lacking discriminative optimization objectives to learn robust features. In this paper, we present a min-max based Center-guided Adversarial Training, namely CgAT, to improve the robustness of deep hashing networks through worst adversarial examples. Specifically, we first formulate the center code as a semantically-discriminative representative of the input image content, which preserves the semantic similarity with positive samples and dissimilarity with negative examples. We prove that a mathematical formula can calculate the center code immediately. After obtaining the center codes in each optimization iteration of the deep hashing network, they are adopted to guide the adversarial training process. On the one hand, CgAT generates the worst adversarial examples as augmented data by maximizing the Hamming distance between the hash codes of the adversarial examples and the center codes. On the other hand, CgAT learns to mitigate the effects of adversarial samples by minimizing the Hamming distance to the center codes. Extensive experiments on the benchmark datasets demonstrate the effectiveness of our adversarial training algorithm in defending against adversarial attacks for deep hashing-based retrieval. Compared with the current state-of-the-art defense method, we significantly improve the defense performance by an average of 18.61%, 12.35%, and 11.56% on FLICKR-25K, NUS-WIDE, and MS-COCO, respectively.

The authors thank Nicholas Carlini (UC Berkeley) and Dimitris Tsipras (MIT) for feedback to improve the survey quality. We also acknowledge X. Huang (Uni. Liverpool), K. R. Reddy (IISC), E. Valle (UNICAMP), Y. Yoo (CLAIR) and others for providing pointers to make the survey more comprehensive.

This paper investigates recently proposed approaches for defending against adversarial examples and evaluating adversarial robustness. We motivate adversarial risk as an objective for achieving models robust to worst-case inputs. We then frame commonly used attacks and evaluation metrics as defining a tractable surrogate objective to the true adversarial risk. This suggests that models may optimize this surrogate rather than the true adversarial risk. We formalize this notion as obscurity to an adversary, and develop tools and heuristics for identifying obscured models and designing transparent models. We demonstrate that this is a significant problem in practice by repurposing gradient-free optimization techniques into adversarial attacks, which we use to decrease the accuracy of several recently proposed defenses to near zero. Our hope is that our formulations and results will help researchers to develop more powerful defenses.

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%.In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

许多最先进的ML模型在各种任务中具有优于图像分类的人类。具有如此出色的性能，ML模型今天被广泛使用。然而，存在对抗性攻击和数据中毒攻击的真正符合ML模型的稳健性。例如，Engstrom等人。证明了最先进的图像分类器可以容易地被任意图像上的小旋转欺骗。由于ML系统越来越纳入安全性和安全敏感的应用，对抗攻击和数据中毒攻击构成了相当大的威胁。本章侧重于ML安全的两个广泛和重要的领域：对抗攻击和数据中毒攻击。

This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier. Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images. The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methods.

Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.

由于具有强大的功能学习能力和高效率，深层哈希在大规模图像检索中取得了巨大的成功。同时，广泛的作品表明，深层神经网络（DNN）容易受到对抗例子的影响，并且探索针对深哈希的对抗性攻击吸引了许多研究工作。然而，尚未对Backdoor攻击（对DNNS的另一个著名威胁）进行深入研究。尽管图像分类领域已经提出了各种后门攻击，但现有方法未能实现真正的不可思议的后门攻击，该攻击享受着隐形触发器并同时享受清洁标签设置，而且它们也无法满足图像检索后门的内在需求。在本文中，我们提出了Badhash，这是第一个基于生成的无透感的后门攻击，对深哈希的攻击，它可以有效地用干净的标签产生隐形和投入特定的中毒图像。具体而言，我们首先提出了一种新的条件生成对抗网络（CGAN）管道，以有效生成中毒样品。对于任何给定的良性图像，它试图产生具有独特无形扳机的自然中毒对应物。为了提高攻击效果，我们引入了基于标签的对比学习网络LabCln来利用不同标签的语义特征，随后将其用于混淆和误导目标模型以学习嵌入式触发器。我们终于探索了在哈希空间中对图像检索的后门攻击的机制。在多个基准数据集上进行的广泛实验证明，Badhash可以生成不察觉的中毒样本，具有强大的攻击能力和对最新的深层哈希方案的可转移性。主要主题领域：[参与]多媒体搜索和建议

We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples. While defenses that cause obfuscated gradients appear to defeat iterative optimizationbased attacks, we find defenses relying on this effect can be circumvented. We describe characteristic behaviors of defenses exhibiting the effect, and for each of the three types of obfuscated gradients we discover, we develop attack techniques to overcome it. In a case study, examining noncertified white-box-secure defenses at ICLR 2018, we find obfuscated gradients are a common occurrence, with 7 of 9 defenses relying on obfuscated gradients. Our new attacks successfully circumvent 6 completely, and 1 partially, in the original threat model each paper considers.

深度神经网络已被证明容易受到对抗图像的影响。常规攻击努力争取严格限制扰动的不可分割的对抗图像。最近，研究人员已采取行动探索可区分但非奇异的对抗图像，并证明色彩转化攻击是有效的。在这项工作中，我们提出了对抗颜色过滤器（ADVCF），这是一种新颖的颜色转换攻击，在简单颜色滤波器的参数空间中通过梯度信息进行了优化。特别是，明确指定了我们的颜色滤波器空间，以便从攻击和防御角度来对对抗性色转换进行系统的鲁棒性分析。相反，由于缺乏这种明确的空间，现有的颜色转换攻击并不能为系统分析提供机会。我们通过用户研究进一步进行了对成功率和图像可接受性的不同颜色转化攻击之间的广泛比较。其他结果为在另外三个视觉任务中针对ADVCF的模型鲁棒性提供了有趣的新见解。我们还强调了ADVCF的人类解剖性，该advcf在实际使用方案中有希望，并显示出比对图像可接受性和效率的最新人解释的色彩转化攻击的优越性。

当系统的全面了解时然而，这种技术在灰盒设置中行动不成功，攻击者面部模板未知。在这项工作中，我们提出了一种具有新开发的目标函数的相似性的灰度逆势攻击（SGADV）技术。 SGAdv利用不同的评分来产生优化的对抗性实例，即基于相似性的对抗性攻击。这种技术适用于白盒和灰度箱攻击，针对使用不同分数确定真实或调用用户的身份验证系统。为了验证SGAdv的有效性，我们对LFW，Celeba和Celeba-HQ的面部数据集进行了广泛的实验，反对白盒和灰度箱设置的面部和洞察面的深脸识别模型。结果表明，所提出的方法显着优于灰色盒设置中的现有的对抗性攻击技术。因此，我们总结了开发对抗性示例的相似性基础方法可以令人满意地迎合去认证的灰度箱攻击场景。

基于深卷积神经网络（CNN）的面部识别表现出归因于提取的高判别特征的卓越精度性能。然而，经常忽略了深度学习模型（深度特征）提取的功能的安全性和隐私。本文提出了从深度功能中重建面部图像，而无需访问CNN网络配置作为约束优化问题。这种优化可最大程度地减少从原始面部图像中提取的特征与重建的面部图像之间的距离。我们没有直接解决图像空间中的优化问题，而是通过寻找GAN发电机的潜在向量来重新重新制定问题，然后使用它来生成面部图像。 GAN发电机在这个新颖的框架中起着双重作用，即优化目标和面部发电机的面部分布约束。除了新颖的优化任务之外，我们还提出了一条攻击管道，以基于生成的面部图像模拟目标用户。我们的结果表明，生成的面部图像可以达到最先进的攻击率在LFW上的最先进的攻击率在I型攻击下为0.1 \％。我们的工作阐明了生物识别部署，以符合隐私和安全政策。

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool deep neural networks in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying deep neural networks in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep neural networks, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full output of the network on arbitrarily many chosen inputs. We define three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partialinformation setting, and the label-only setting. We develop new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. We demonstrate that our methods are effective against an ImageNet classifier under our proposed threat models. We also demonstrate a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.