Backdoor attacks represent one of the major threats to machine learning models. Various efforts have been made to mitigate backdoors. However, existing defenses have become increasingly complex and often require high computational resources or may also jeopardize models' utility. In this work, we show that fine-tuning, one of the most common and easy-to-adopt machine learning training operations, can effectively remove backdoors from machine learning models while maintaining high model utility. Extensive experiments over three machine learning paradigms show that fine-tuning and our newly proposed super-fine-tuning achieve strong defense performance. Furthermore, we coin a new term, namely backdoor sequela, to measure the changes in model vulnerabilities to other attacks before and after the backdoor has been removed. Empirical evaluation shows that, compared to other defense methods, super-fine-tuning leaves limited backdoor sequela. We hope our results can help machine learning model owners better protect their models from backdoor threats. Also, it calls for the design of more advanced attacks in order to comprehensively assess machine learning models' backdoor vulnerabilities.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

机器学习模型容易记住敏感数据，使它们容易受到会员推理攻击的攻击，其中对手的目的是推断是否使用输入样本来训练模型。在过去的几年中，研究人员产生了许多会员推理攻击和防御。但是，这些攻击和防御采用各种策略，并在不同的模型和数据集中进行。但是，缺乏全面的基准意味着我们不了解现有攻击和防御的优势和劣势。我们通过对不同的会员推理攻击和防御措施进行大规模测量来填补这一空白。我们通过研究九项攻击和六项防御措施来系统化成员的推断，并在整体评估中衡量不同攻击和防御的性能。然后，我们量化威胁模型对这些攻击结果的影响。我们发现，威胁模型的某些假设，例如相同架构和阴影和目标模型之间的相同分布是不必要的。我们也是第一个对从Internet收集的现实世界数据而不是实验室数据集进行攻击的人。我们进一步研究是什么决定了会员推理攻击的表现，并揭示了通常认为过度拟合水平不足以成功攻击。取而代之的是，成员和非成员样本之间的熵/横向熵的詹森 - 香农距离与攻击性能的相关性更好。这为我们提供了一种新的方法，可以在不进行攻击的情况下准确预测会员推理风险。最后，我们发现数据增强在更大程度上降低了现有攻击的性能，我们提出了使用增强作用的自适应攻击来训练阴影和攻击模型，以改善攻击性能。

后门攻击已成为深度神经网络（DNN）的主要安全威胁。虽然现有的防御方法在检测或擦除后以后展示了有希望的结果，但仍然尚不清楚是否可以设计强大的培训方法，以防止后门触发器首先注入训练的模型。在本文中，我们介绍了\ emph {反后门学习}的概念，旨在培训\ emph {Clean}模型给出了后门中毒数据。我们将整体学习过程框架作为学习\ emph {clean}和\ emph {backdoor}部分的双重任务。从这种观点来看，我们确定了两个后门攻击的固有特征，因为他们的弱点2）后门任务与特定类（后门目标类）相关联。根据这两个弱点，我们提出了一般学习计划，反后门学习（ABL），在培训期间自动防止后门攻击。 ABL引入了标准培训的两级\ EMPH {梯度上升}机制，帮助分离早期训练阶段的后台示例，2）在后续训练阶段中断后门示例和目标类之间的相关性。通过对多个基准数据集的广泛实验，针对10个最先进的攻击，我们经验证明，后卫中毒数据上的ABL培训模型实现了与纯净清洁数据训练的相同性能。代码可用于\ url {https:/github.com/boylyg/abl}。

Backdoor attacks have emerged as one of the major security threats to deep learning models as they can easily control the model's test-time predictions by pre-injecting a backdoor trigger into the model at training time. While backdoor attacks have been extensively studied on images, few works have investigated the threat of backdoor attacks on time series data. To fill this gap, in this paper we present a novel generative approach for time series backdoor attacks against deep learning based time series classifiers. Backdoor attacks have two main goals: high stealthiness and high attack success rate. We find that, compared to images, it can be more challenging to achieve the two goals on time series. This is because time series have fewer input dimensions and lower degrees of freedom, making it hard to achieve a high attack success rate without compromising stealthiness. Our generative approach addresses this challenge by generating trigger patterns that are as realistic as real-time series patterns while achieving a high attack success rate without causing a significant drop in clean accuracy. We also show that our proposed attack is resistant to potential backdoor defenses. Furthermore, we propose a novel universal generator that can poison any type of time series with a single generator that allows universal attacks without the need to fine-tune the generative model for new time series datasets.

预训练模型（PTM）已被广泛用于各种下游任务。 PTM的参数分布在Internet上，可能会遭受后门攻击。在这项工作中，我们演示了PTMS的普遍脆弱性，在该工作中，可以通过任意下游任务中的后门攻击轻松控制PTMS。具体而言，攻击者可以添加一个简单的预训练任务，该任务将触发实例的输出表示限制为预定义的向量，即神经元级后门攻击（NEUBA）。如果在微调过程中未消除后门功能，则触发器可以通过预定义的矢量预测固定标签。在自然语言处理（NLP）和计算机视觉（CV）的实验中，我们表明Neuba绝对可以控制触发实例的预测，而无需了解下游任务。最后，我们将几种防御方法应用于Neuba，并发现模型修剪是通过排除后门神经元来抵抗Neuba的有希望的方向。我们的发现听起来是红色警报，用于广泛使用PTM。我们的源代码和模型可在\ url {https://github.com/thunlp/neuba}上获得。

后门学习是研究深神经网络（DNNS）脆弱性的一个新兴而重要的话题。在快速武器竞赛的地位上，正在连续或同时提出许多开创性的后门攻击和防御方法。但是，我们发现对新方法的评估通常是不可思议的，以验证其主张和实际绩效，这主要是由于快速发展，不同的环境以及实施和可重复性的困难。没有彻底的评估和比较，很难跟踪当前的进度并设计文献的未来发展路线图。为了减轻这一困境，我们建立了一个名为Backdoorbench的后门学习的全面基准。它由一个可扩展的基于模块化的代码库（当前包括8个最先进（SOTA）攻击和9种SOTA防御算法的实现），以及完整的后门学习的标准化协议。我们还基于5个模型和4个数据集，对9个防御措施的每对8次攻击进行全面评估，总共8,000对评估。我们从不同的角度进一步介绍了对这8,000次评估的不同角度，研究了对国防算法，中毒比率，模型和数据集对后门学习的影响。 \ url {https://backdoorbench.github.io}公开获得了Backdoorbench的所有代码和评估。

Transforming off-the-shelf deep neural network (DNN) models into dynamic multi-exit architectures can achieve inference and transmission efficiency by fragmenting and distributing a large DNN model in edge computing scenarios (e.g., edge devices and cloud servers). In this paper, we propose a novel backdoor attack specifically on the dynamic multi-exit DNN models. Particularly, we inject a backdoor by poisoning one DNN model's shallow hidden layers targeting not this vanilla DNN model but only its dynamically deployed multi-exit architectures. Our backdoored vanilla model behaves normally on performance and cannot be activated even with the correct trigger. However, the backdoor will be activated when the victims acquire this model and transform it into a dynamic multi-exit architecture at their deployment. We conduct extensive experiments to prove the effectiveness of our attack on three structures (ResNet-56, VGG-16, and MobileNet) with four datasets (CIFAR-10, SVHN, GTSRB, and Tiny-ImageNet) and our backdoor is stealthy to evade multiple state-of-the-art backdoor detection or removal methods.

后门攻击已被证明是对深度学习模型的严重安全威胁，并且检测给定模型是否已成为后门成为至关重要的任务。现有的防御措施主要建立在观察到后门触发器通常尺寸很小或仅影响几个神经元激活的观察结果。但是，在许多情况下，尤其是对于高级后门攻击，违反了上述观察结果，阻碍了现有防御的性能和适用性。在本文中，我们提出了基于新观察的后门防御范围。也就是说，有效的后门攻击通常需要对中毒训练样本的高预测置信度，以确保训练有素的模型具有很高的可能性。基于此观察结果，Dtinspector首先学习一个可以改变最高信心数据的预测的补丁，然后通过检查在低信心数据上应用学习补丁后检查预测变化的比率来决定后门的存在。对五次后门攻击，四个数据集和三种高级攻击类型的广泛评估证明了拟议防御的有效性。

视觉变压器（VITS）具有与卷积神经网络相比，具有较小的感应偏置的根本不同的结构。随着绩效的提高，VIT的安全性和鲁棒性也非常重要。与许多最近利用VIT反对对抗性例子的鲁棒性的作品相反，本文调查了代表性的病因攻击，即后门。我们首先检查了VIT对各种后门攻击的脆弱性，发现VIT也很容易受到现有攻击的影响。但是，我们观察到，VIT的清洁数据准确性和后门攻击成功率在位置编码之前对补丁转换做出了明显的反应。然后，根据这一发现，我们为VIT提出了一种通过补丁处理来捍卫基于补丁的触发后门攻击的有效方法。在包括CIFAR10，GTSRB和Tinyimagenet在内的几个基准数据集上评估了这些表演，这些数据表明，该拟议的新颖防御在减轻VIT的后门攻击方面非常成功。据我们所知，本文提出了第一个防御性策略，该策略利用了反对后门攻击的VIT的独特特征。

本文提出了针对回顾性神经网络（Badnets）的新型两级防御（NNOCULICULE），该案例在响应该字段中遇到的回溯测试输入，修复了预部署和在线的BADNET。在预部署阶段，NNICULICULE与清洁验证输入的随机扰动进行检测，以部分减少后门的对抗影响。部署后，NNOCULICULE通过在原始和预先部署修补网络之间录制分歧来检测和隔离测试输入。然后培训Constcan以学习清洁验证和隔离输入之间的转换;即，它学会添加触发器来清洁验证图像。回顾验证图像以及其正确的标签用于进一步重新培训预修补程序，产生我们的最终防御。关于全面的后门攻击套件的实证评估表明，NNOCLICULE优于所有最先进的防御，以制定限制性假设，并且仅在特定的后门攻击上工作，或者在适应性攻击中失败。相比之下，NNICULICULE使得最小的假设并提供有效的防御，即使在现有防御因攻击者而导致其限制假设而导致的现有防御无效的情况下。

This paper asks the intriguing question: is it possible to exploit neural architecture search (NAS) as a new attack vector to launch previously improbable attacks? Specifically, we present EVAS, a new attack that leverages NAS to find neural architectures with inherent backdoors and exploits such vulnerability using input-aware triggers. Compared with existing attacks, EVAS demonstrates many interesting properties: (i) it does not require polluting training data or perturbing model parameters; (ii) it is agnostic to downstream fine-tuning or even re-training from scratch; (iii) it naturally evades defenses that rely on inspecting model parameters or training data. With extensive evaluation on benchmark datasets, we show that EVAS features high evasiveness, transferability, and robustness, thereby expanding the adversary's design spectrum. We further characterize the mechanisms underlying EVAS, which are possibly explainable by architecture-level ``shortcuts'' that recognize trigger patterns. This work raises concerns about the current practice of NAS and points to potential directions to develop effective countermeasures.

与令人印象深刻的进步触动了我们社会的各个方面，基于深度神经网络（DNN）的AI技术正在带来越来越多的安全问题。虽然在考试时间运行的攻击垄断了研究人员的初始关注，但是通过干扰培训过程来利用破坏DNN模型的可能性，代表了破坏训练过程的可能性，这是破坏AI技术的可靠性的进一步严重威胁。在后门攻击中，攻击者损坏了培训数据，以便在测试时间诱导错误的行为。然而，测试时间误差仅在存在与正确制作的输入样本对应的触发事件的情况下被激活。通过这种方式，损坏的网络继续正常输入的预期工作，并且只有当攻击者决定激活网络内隐藏的后门时，才会发生恶意行为。在过去几年中，后门攻击一直是强烈的研究活动的主题，重点是新的攻击阶段的发展，以及可能对策的提议。此概述文件的目标是审查发表的作品，直到现在，分类到目前为止提出的不同类型的攻击和防御。指导分析的分类基于攻击者对培训过程的控制量，以及防御者验证用于培训的数据的完整性，并监控DNN在培训和测试中的操作时间。因此，拟议的分析特别适合于参考他们在运营的应用方案的攻击和防御的强度和弱点。

最近的研究表明，深层神经网络容易受到不同类型的攻击，例如对抗性攻击，数据中毒攻击和后门攻击。其中，后门攻击是最狡猾的攻击，几乎可以在深度学习管道的每个阶段发生。因此，后门攻击吸引了学术界和行业的许多兴趣。但是，大多数现有的后门攻击方法对于某些轻松的预处理（例如常见数据转换）都是可见的或脆弱的。为了解决这些限制，我们提出了一种强大而无形的后门攻击，称为“毒药”。具体而言，我们首先利用图像结构作为目标中毒区域，并用毒药（信息）填充它们以生成触发图案。由于图像结构可以在数据转换期间保持其语义含义，因此这种触发模式对数据转换本质上是强大的。然后，我们利用深度注射网络将这种触发模式嵌入封面图像中，以达到隐身性。与现有流行的后门攻击方法相比，毒药的墨水在隐形和健壮性方面都优于表现。通过广泛的实验，我们证明了毒药不仅是不同数据集和网络体系结构的一般性，而且对于不同的攻击场景也很灵活。此外，它对许多最先进的防御技术也具有非常强烈的抵抗力。

In this paper, we present a simple yet surprisingly effective technique to induce "selective amnesia" on a backdoored model. Our approach, called SEAM, has been inspired by the problem of catastrophic forgetting (CF), a long standing issue in continual learning. Our idea is to retrain a given DNN model on randomly labeled clean data, to induce a CF on the model, leading to a sudden forget on both primary and backdoor tasks; then we recover the primary task by retraining the randomized model on correctly labeled clean data. We analyzed SEAM by modeling the unlearning process as continual learning and further approximating a DNN using Neural Tangent Kernel for measuring CF. Our analysis shows that our random-labeling approach actually maximizes the CF on an unknown backdoor in the absence of triggered inputs, and also preserves some feature extraction in the network to enable a fast revival of the primary task. We further evaluated SEAM on both image processing and Natural Language Processing tasks, under both data contamination and training manipulation attacks, over thousands of models either trained on popular image datasets or provided by the TrojAI competition. Our experiments show that SEAM vastly outperforms the state-of-the-art unlearning techniques, achieving a high Fidelity (measuring the gap between the accuracy of the primary task and that of the backdoor) within a few minutes (about 30 times faster than training a model from scratch using the MNIST dataset), with only a small amount of clean data (0.1% of training data for TrojAI models).

自我监督学习是一种新兴的机器学习（ML）范式。与监督的学习相比，哪些利用高质量标记的数据集以实现良好的性能相比，自我监督的学习依赖于未标记的数据集来预先培训功能强大的编码器，然后可以将其视为各种下游任务的功能提取器。大量的数据和计算资源消耗使编码器本身成为模型所有者的宝贵知识产权。最近的研究表明，ML模型的版权受到模型窃取攻击的威胁，该攻击旨在训练替代模型以模仿给定模型的行为。我们从经验上表明，预训练的编码器极易受到模型窃取攻击的影响。但是，版权保护算法（例如水印）的大多数努力集中在分类器上。同时，预先培训的编码器版权保护的内在挑战在很大程度上仍然没有研究。我们通过提出SSLGuard，这是第一种用于预训练的编码器的水印算法。鉴于干净的预训练编码器，SSLGuard向其中注入了水印，并输出了水印版本。还采用了阴影训练技术来保留潜在模型窃取攻击下的水印。我们广泛的评估表明，SSLGuard在水印注入和验证方面有效，并且可以防止模型窃取和其他水印去除攻击，例如输入噪声，输出扰动，覆盖，覆盖，模型修剪和微调。

Contrastive learning pre-trains an image encoder using a large amount of unlabeled data such that the image encoder can be used as a general-purpose feature extractor for various downstream tasks. In this work, we propose PoisonedEncoder, a data poisoning attack to contrastive learning. In particular, an attacker injects carefully crafted poisoning inputs into the unlabeled pre-training data, such that the downstream classifiers built based on the poisoned encoder for multiple target downstream tasks simultaneously classify attacker-chosen, arbitrary clean inputs as attacker-chosen, arbitrary classes. We formulate our data poisoning attack as a bilevel optimization problem, whose solution is the set of poisoning inputs; and we propose a contrastive-learning-tailored method to approximately solve it. Our evaluation on multiple datasets shows that PoisonedEncoder achieves high attack success rates while maintaining the testing accuracy of the downstream classifiers built upon the poisoned encoder for non-attacker-chosen inputs. We also evaluate five defenses against PoisonedEncoder, including one pre-processing, three in-processing, and one post-processing defenses. Our results show that these defenses can decrease the attack success rate of PoisonedEncoder, but they also sacrifice the utility of the encoder or require a large clean pre-training dataset.

在对抗机器学习中，防止对深度学习系统的攻击的新防御能力在释放更强大的攻击后不久就会破坏。在这种情况下，法医工具可以通过追溯成功的根本原因来为现有防御措施提供宝贵的补充，并为缓解措施提供前进的途径，以防止将来采取类似的攻击。在本文中，我们描述了我们为开发用于深度神经网络毒物攻击的法医追溯工具的努力。我们提出了一种新型的迭代聚类和修剪解决方案，该解决方案修剪了“无辜”训练样本，直到所有剩余的是一组造成攻击的中毒数据。我们的方法群群训练样本基于它们对模型参数的影响，然后使用有效的数据解读方法来修剪无辜簇。我们从经验上证明了系统对三种类型的肮脏标签（后门）毒物攻击和三种类型的清洁标签毒药攻击的功效，这些毒物跨越了计算机视觉和恶意软件分类。我们的系统在所有攻击中都达到了98.4％的精度和96.8％的召回。我们还表明，我们的系统与专门攻击它的四种抗纤维法措施相对强大。

半监督学习（SSL）利用标记和未标记的数据来训练机器学习（ML）模型。最先进的SSL方法可以通过利用更少的标记数据来实现与监督学习相当的性能。但是，大多数现有作品都集中在提高SSL的性能。在这项工作中，我们通过研究SSL的培训数据隐私来采取不同的角度。具体而言，我们建议针对由SSL训练的ML模型进行的第一个基于数据增强的成员推理攻击。给定数据样本和黑框访问模型，成员推理攻击的目标是确定数据样本是否属于模型的训练数据集。我们的评估表明，拟议的攻击可以始终超过现有的成员推理攻击，并针对由SSL训练的模型实现最佳性能。此外，我们发现，SSL中会员泄漏的原因与受到监督学习中普遍认为的原因不同，即过度拟合（培训和测试准确性之间的差距）。我们观察到，SSL模型已被概括为测试数据（几乎为0个过度拟合），但“记住”训练数据通过提供更自信的预测，无论其正确性如何。我们还探索了早期停止，作为防止成员推理攻击SSL的对策。结果表明，早期停止可以减轻会员推理攻击，但由于模型的实用性降解成本。

As a critical threat to deep neural networks (DNNs), backdoor attacks can be categorized into two types, i.e., source-agnostic backdoor attacks (SABAs) and source-specific backdoor attacks (SSBAs). Compared to traditional SABAs, SSBAs are more advanced in that they have superior stealthier in bypassing mainstream countermeasures that are effective against SABAs. Nonetheless, existing SSBAs suffer from two major limitations. First, they can hardly achieve a good trade-off between ASR (attack success rate) and FPR (false positive rate). Besides, they can be effectively detected by the state-of-the-art (SOTA) countermeasures (e.g., SCAn). To address the limitations above, we propose a new class of viable source-specific backdoor attacks, coined as CASSOCK. Our key insight is that trigger designs when creating poisoned data and cover data in SSBAs play a crucial role in demonstrating a viable source-specific attack, which has not been considered by existing SSBAs. With this insight, we focus on trigger transparency and content when crafting triggers for poisoned dataset where a sample has an attacker-targeted label and cover dataset where a sample has a ground-truth label. Specifically, we implement $CASSOCK_{Trans}$ and $CASSOCK_{Cont}$. While both they are orthogonal, they are complementary to each other, generating a more powerful attack, called $CASSOCK_{Comp}$, with further improved attack performance and stealthiness. We perform a comprehensive evaluation of the three $CASSOCK$-based attacks on four popular datasets and three SOTA defenses. Compared with a representative SSBA as a baseline ($SSBA_{Base}$), $CASSOCK$-based attacks have significantly advanced the attack performance, i.e., higher ASR and lower FPR with comparable CDA (clean data accuracy). Besides, $CASSOCK$-based attacks have effectively bypassed the SOTA defenses, and $SSBA_{Base}$ cannot.