Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Backdoor attacks represent one of the major threats to machine learning models. Various efforts have been made to mitigate backdoors. However, existing defenses have become increasingly complex and often require high computational resources or may also jeopardize models' utility. In this work, we show that fine-tuning, one of the most common and easy-to-adopt machine learning training operations, can effectively remove backdoors from machine learning models while maintaining high model utility. Extensive experiments over three machine learning paradigms show that fine-tuning and our newly proposed super-fine-tuning achieve strong defense performance. Furthermore, we coin a new term, namely backdoor sequela, to measure the changes in model vulnerabilities to other attacks before and after the backdoor has been removed. Empirical evaluation shows that, compared to other defense methods, super-fine-tuning leaves limited backdoor sequela. We hope our results can help machine learning model owners better protect their models from backdoor threats. Also, it calls for the design of more advanced attacks in order to comprehensively assess machine learning models' backdoor vulnerabilities.

后门攻击已成为深度神经网络（DNN）的主要安全威胁。虽然现有的防御方法在检测或擦除后以后展示了有希望的结果，但仍然尚不清楚是否可以设计强大的培训方法，以防止后门触发器首先注入训练的模型。在本文中，我们介绍了\ emph {反后门学习}的概念，旨在培训\ emph {Clean}模型给出了后门中毒数据。我们将整体学习过程框架作为学习\ emph {clean}和\ emph {backdoor}部分的双重任务。从这种观点来看，我们确定了两个后门攻击的固有特征，因为他们的弱点2）后门任务与特定类（后门目标类）相关联。根据这两个弱点，我们提出了一般学习计划，反后门学习（ABL），在培训期间自动防止后门攻击。 ABL引入了标准培训的两级\ EMPH {梯度上升}机制，帮助分离早期训练阶段的后台示例，2）在后续训练阶段中断后门示例和目标类之间的相关性。通过对多个基准数据集的广泛实验，针对10个最先进的攻击，我们经验证明，后卫中毒数据上的ABL培训模型实现了与纯净清洁数据训练的相同性能。代码可用于\ url {https:/github.com/boylyg/abl}。

与令人印象深刻的进步触动了我们社会的各个方面，基于深度神经网络（DNN）的AI技术正在带来越来越多的安全问题。虽然在考试时间运行的攻击垄断了研究人员的初始关注，但是通过干扰培训过程来利用破坏DNN模型的可能性，代表了破坏训练过程的可能性，这是破坏AI技术的可靠性的进一步严重威胁。在后门攻击中，攻击者损坏了培训数据，以便在测试时间诱导错误的行为。然而，测试时间误差仅在存在与正确制作的输入样本对应的触发事件的情况下被激活。通过这种方式，损坏的网络继续正常输入的预期工作，并且只有当攻击者决定激活网络内隐藏的后门时，才会发生恶意行为。在过去几年中，后门攻击一直是强烈的研究活动的主题，重点是新的攻击阶段的发展，以及可能对策的提议。此概述文件的目标是审查发表的作品，直到现在，分类到目前为止提出的不同类型的攻击和防御。指导分析的分类基于攻击者对培训过程的控制量，以及防御者验证用于培训的数据的完整性，并监控DNN在培训和测试中的操作时间。因此，拟议的分析特别适合于参考他们在运营的应用方案的攻击和防御的强度和弱点。

后门攻击已被证明是对深度学习模型的严重安全威胁，并且检测给定模型是否已成为后门成为至关重要的任务。现有的防御措施主要建立在观察到后门触发器通常尺寸很小或仅影响几个神经元激活的观察结果。但是，在许多情况下，尤其是对于高级后门攻击，违反了上述观察结果，阻碍了现有防御的性能和适用性。在本文中，我们提出了基于新观察的后门防御范围。也就是说，有效的后门攻击通常需要对中毒训练样本的高预测置信度，以确保训练有素的模型具有很高的可能性。基于此观察结果，Dtinspector首先学习一个可以改变最高信心数据的预测的补丁，然后通过检查在低信心数据上应用学习补丁后检查预测变化的比率来决定后门的存在。对五次后门攻击，四个数据集和三种高级攻击类型的广泛评估证明了拟议防御的有效性。

在对抗机器学习中，防止对深度学习系统的攻击的新防御能力在释放更强大的攻击后不久就会破坏。在这种情况下，法医工具可以通过追溯成功的根本原因来为现有防御措施提供宝贵的补充，并为缓解措施提供前进的途径，以防止将来采取类似的攻击。在本文中，我们描述了我们为开发用于深度神经网络毒物攻击的法医追溯工具的努力。我们提出了一种新型的迭代聚类和修剪解决方案，该解决方案修剪了“无辜”训练样本，直到所有剩余的是一组造成攻击的中毒数据。我们的方法群群训练样本基于它们对模型参数的影响，然后使用有效的数据解读方法来修剪无辜簇。我们从经验上证明了系统对三种类型的肮脏标签（后门）毒物攻击和三种类型的清洁标签毒药攻击的功效，这些毒物跨越了计算机视觉和恶意软件分类。我们的系统在所有攻击中都达到了98.4％的精度和96.8％的召回。我们还表明，我们的系统与专门攻击它的四种抗纤维法措施相对强大。

Vertical federated learning (VFL) is an emerging paradigm that enables collaborators to build machine learning models together in a distributed fashion. In general, these parties have a group of users in common but own different features. Existing VFL frameworks use cryptographic techniques to provide data privacy and security guarantees, leading to a line of works studying computing efficiency and fast implementation. However, the security of VFL's model remains underexplored.

Backdoor attacks have emerged as one of the major security threats to deep learning models as they can easily control the model's test-time predictions by pre-injecting a backdoor trigger into the model at training time. While backdoor attacks have been extensively studied on images, few works have investigated the threat of backdoor attacks on time series data. To fill this gap, in this paper we present a novel generative approach for time series backdoor attacks against deep learning based time series classifiers. Backdoor attacks have two main goals: high stealthiness and high attack success rate. We find that, compared to images, it can be more challenging to achieve the two goals on time series. This is because time series have fewer input dimensions and lower degrees of freedom, making it hard to achieve a high attack success rate without compromising stealthiness. Our generative approach addresses this challenge by generating trigger patterns that are as realistic as real-time series patterns while achieving a high attack success rate without causing a significant drop in clean accuracy. We also show that our proposed attack is resistant to potential backdoor defenses. Furthermore, we propose a novel universal generator that can poison any type of time series with a single generator that allows universal attacks without the need to fine-tune the generative model for new time series datasets.

A recent trojan attack on deep neural network (DNN) models is one insidious variant of data poisoning attacks. Trojan attacks exploit an effective backdoor created in a DNN model by leveraging the difficulty in interpretability of the learned model to misclassify any inputs signed with the attacker's chosen trojan trigger. Since the trojan trigger is a secret guarded and exploited by the attacker, detecting such trojan inputs is a challenge, especially at run-time when models are in active operation. This work builds STRong Intentional Perturbation (STRIP) based run-time trojan attack detection system and focuses on vision system. We intentionally perturb the incoming input, for instance by superimposing various image patterns, and observe the randomness of predicted classes for perturbed inputs from a given deployed model-malicious or benign. A low entropy in predicted classes violates the input-dependence property of a benign model and implies the presence of a malicious input-a characteristic of a trojaned input. The high efficacy of our method is validated through case studies on three popular and contrasting datasets: MNIST, CIFAR10 and GTSRB. We achieve an overall false acceptance rate (FAR) of less than 1%, given a preset false rejection rate (FRR) of 1%, for different types of triggers. Using CIFAR10 and GTSRB, we have empirically achieved result of 0% for both FRR and FAR. We have also evaluated STRIP robustness against a number of trojan attack variants and adaptive attacks.

As a critical threat to deep neural networks (DNNs), backdoor attacks can be categorized into two types, i.e., source-agnostic backdoor attacks (SABAs) and source-specific backdoor attacks (SSBAs). Compared to traditional SABAs, SSBAs are more advanced in that they have superior stealthier in bypassing mainstream countermeasures that are effective against SABAs. Nonetheless, existing SSBAs suffer from two major limitations. First, they can hardly achieve a good trade-off between ASR (attack success rate) and FPR (false positive rate). Besides, they can be effectively detected by the state-of-the-art (SOTA) countermeasures (e.g., SCAn). To address the limitations above, we propose a new class of viable source-specific backdoor attacks, coined as CASSOCK. Our key insight is that trigger designs when creating poisoned data and cover data in SSBAs play a crucial role in demonstrating a viable source-specific attack, which has not been considered by existing SSBAs. With this insight, we focus on trigger transparency and content when crafting triggers for poisoned dataset where a sample has an attacker-targeted label and cover dataset where a sample has a ground-truth label. Specifically, we implement $CASSOCK_{Trans}$ and $CASSOCK_{Cont}$. While both they are orthogonal, they are complementary to each other, generating a more powerful attack, called $CASSOCK_{Comp}$, with further improved attack performance and stealthiness. We perform a comprehensive evaluation of the three $CASSOCK$-based attacks on four popular datasets and three SOTA defenses. Compared with a representative SSBA as a baseline ($SSBA_{Base}$), $CASSOCK$-based attacks have significantly advanced the attack performance, i.e., higher ASR and lower FPR with comparable CDA (clean data accuracy). Besides, $CASSOCK$-based attacks have effectively bypassed the SOTA defenses, and $SSBA_{Base}$ cannot.

Transforming off-the-shelf deep neural network (DNN) models into dynamic multi-exit architectures can achieve inference and transmission efficiency by fragmenting and distributing a large DNN model in edge computing scenarios (e.g., edge devices and cloud servers). In this paper, we propose a novel backdoor attack specifically on the dynamic multi-exit DNN models. Particularly, we inject a backdoor by poisoning one DNN model's shallow hidden layers targeting not this vanilla DNN model but only its dynamically deployed multi-exit architectures. Our backdoored vanilla model behaves normally on performance and cannot be activated even with the correct trigger. However, the backdoor will be activated when the victims acquire this model and transform it into a dynamic multi-exit architecture at their deployment. We conduct extensive experiments to prove the effectiveness of our attack on three structures (ResNet-56, VGG-16, and MobileNet) with four datasets (CIFAR-10, SVHN, GTSRB, and Tiny-ImageNet) and our backdoor is stealthy to evade multiple state-of-the-art backdoor detection or removal methods.

深度神经网络（DNNS）在训练过程中容易受到后门攻击的影响。该模型以这种方式损坏正常起作用，但是当输入中的某些模式触发时，会产生预定义的目标标签。现有防御通常依赖于通用后门设置的假设，其中有毒样品共享相同的均匀扳机。但是，最近的高级后门攻击表明，这种假设在动态后门中不再有效，在动态后门中，触发者因输入而异，从而击败了现有的防御。在这项工作中，我们提出了一种新颖的技术BEATRIX（通过革兰氏矩阵检测）。 BEATRIX利用革兰氏矩阵不仅捕获特征相关性，还可以捕获表示形式的适当高阶信息。通过从正常样本的激活模式中学习类条件统计，BEATRIX可以通过捕获激活模式中的异常来识别中毒样品。为了进一步提高识别目标标签的性能，BEATRIX利用基于内核的测试，而无需对表示分布进行任何先前的假设。我们通过与最先进的防御技术进行了广泛的评估和比较来证明我们的方法的有效性。实验结果表明，我们的方法在检测动态后门时达到了91.1％的F1得分，而最新技术只能达到36.9％。

计算能力和大型培训数据集的可用性增加，机器学习的成功助长了。假设它充分代表了在测试时遇到的数据，则使用培训数据来学习新模型或更新现有模型。这种假设受到中毒威胁的挑战，这种攻击会操纵训练数据，以损害模型在测试时的表现。尽管中毒已被认为是行业应用中的相关威胁，到目前为止，已经提出了各种不同的攻击和防御措施，但对该领域的完整系统化和批判性审查仍然缺失。在这项调查中，我们在机器学习中提供了中毒攻击和防御措施的全面系统化，审查了过去15年中该领域发表的100多篇论文。我们首先对当前的威胁模型和攻击进行分类，然后相应地组织现有防御。虽然我们主要关注计算机视觉应用程序，但我们认为我们的系统化还包括其他数据模式的最新攻击和防御。最后，我们讨论了中毒研究的现有资源，并阐明了当前的局限性和该研究领域的开放研究问题。

后门攻击已被证明是对深度学习系统的严重威胁，如生物识别认证和自主驾驶。有效的后门攻击可以在某些预定义条件下执行模型行为，即，触发器，但否则正常表现。然而，现有攻击的触发器直接注入像素空间，这往往可通过现有的防御和在训练和推理阶段进行视觉识别。在本文中，我们通过Trojaning频域提出了一个新的后门攻击ftrojan。关键的直觉是频域中的触发扰动对应于分散整个图像的小像素明智的扰动，打破了现有防御的底层假设，并使中毒图像从清洁的假设可视地无法区分。我们在几个数据集和任务中评估ftrojan，表明它实现了高攻击成功率，而不会显着降低良性输入的预测准确性。此外，中毒图像几乎看不见并保持高感性的质量。我们还评估FTROJAN，以防止最先进的防御以及在频域中设计的若干自适应防御。结果表明，FTROJAN可以强大地避开或显着降解这些防御的性能。

随着深度神经网络（DNN）的广泛应用，后门攻击逐渐引起了人们的关注。后门攻击是阴险的，中毒模型在良性样本上的表现良好，只有在给定特定输入时才会触发，这会导致神经网络产生不正确的输出。最先进的后门攻击工作是通过数据中毒（即攻击者注入中毒样品中的数据集中）实施的，并且用该数据集训练的模型被后门感染。但是，当前研究中使用的大多数触发因素都是在一小部分图像上修补的固定图案，并且经常被明显错误地标记，这很容易被人类或防御方法（例如神经清洁和前哨）检测到。同样，DNN很难在没有标记的情况下学习，因为它们可能会忽略小图案。在本文中，我们提出了一种基于频域的广义后门攻击方法，该方法可以实现后门植入而不会错标和访问训练过程。它是人类看不见的，能够逃避常用的防御方法。我们在三个数据集（CIFAR-10，STL-10和GTSRB）的无标签和清洁标签案例中评估了我们的方法。结果表明，我们的方法可以在所有任务上实现高攻击成功率（高于90％），而不会在主要任务上进行大量绩效降解。此外，我们评估了我们的方法的旁路性能，以进行各种防御措施，包括检测训练数据（即激活聚类），输入的预处理（即过滤），检测输入（即Sentinet）和检测模型（即神经清洁）。实验结果表明，我们的方法对这种防御能力表现出极好的鲁棒性。

神经网络的不透明度导致其脆弱性发生后门攻击，其中触发了感染神经元的隐藏注意力，以覆盖对攻击者选择的神经元的正常预测。在本文中，我们提出了一种新型的后门防御方法，以标记和净化后门神经网络中受感染的神经元。具体来说，我们首先定义了一个名为良性显着性的新指标。通过将一阶梯度组合以保持神经元之间的连接，良性显着性可以鉴定出比后门防御中常用度量的高精度的感染神经元。然后，提出了一种新的自适应正则化（AR）机制，以通过微调来帮助净化这些被鉴定的感染神经元。由于能够适应不同参数幅度的能力，与神经元纯化中的共同正则化机制相比，AR可以提供更快，更稳定的收敛性。广泛的实验结果表明，我们的方法可以消除具有可忽略的性能降解的神经网络中的后门。

In this paper, we present a simple yet surprisingly effective technique to induce "selective amnesia" on a backdoored model. Our approach, called SEAM, has been inspired by the problem of catastrophic forgetting (CF), a long standing issue in continual learning. Our idea is to retrain a given DNN model on randomly labeled clean data, to induce a CF on the model, leading to a sudden forget on both primary and backdoor tasks; then we recover the primary task by retraining the randomized model on correctly labeled clean data. We analyzed SEAM by modeling the unlearning process as continual learning and further approximating a DNN using Neural Tangent Kernel for measuring CF. Our analysis shows that our random-labeling approach actually maximizes the CF on an unknown backdoor in the absence of triggered inputs, and also preserves some feature extraction in the network to enable a fast revival of the primary task. We further evaluated SEAM on both image processing and Natural Language Processing tasks, under both data contamination and training manipulation attacks, over thousands of models either trained on popular image datasets or provided by the TrojAI competition. Our experiments show that SEAM vastly outperforms the state-of-the-art unlearning techniques, achieving a high Fidelity (measuring the gap between the accuracy of the primary task and that of the backdoor) within a few minutes (about 30 times faster than training a model from scratch using the MNIST dataset), with only a small amount of clean data (0.1% of training data for TrojAI models).

视觉变压器（VITS）具有与卷积神经网络相比，具有较小的感应偏置的根本不同的结构。随着绩效的提高，VIT的安全性和鲁棒性也非常重要。与许多最近利用VIT反对对抗性例子的鲁棒性的作品相反，本文调查了代表性的病因攻击，即后门。我们首先检查了VIT对各种后门攻击的脆弱性，发现VIT也很容易受到现有攻击的影响。但是，我们观察到，VIT的清洁数据准确性和后门攻击成功率在位置编码之前对补丁转换做出了明显的反应。然后，根据这一发现，我们为VIT提出了一种通过补丁处理来捍卫基于补丁的触发后门攻击的有效方法。在包括CIFAR10，GTSRB和Tinyimagenet在内的几个基准数据集上评估了这些表演，这些数据表明，该拟议的新颖防御在减轻VIT的后门攻击方面非常成功。据我们所知，本文提出了第一个防御性策略，该策略利用了反对后门攻击的VIT的独特特征。

最近的研究表明，深度神经网络（DNN）容易受到后门攻击的影响，后门攻击会导致DNN的恶意行为，当时特定的触发器附在输入图像上时。进一步证明，感染的DNN具有一系列通道，与正常通道相比，该通道对后门触发器更敏感。然后，将这些通道修剪可有效缓解后门行为。要定位这些通道，自然要考虑其Lipschitzness，这可以衡量他们对输入上最严重的扰动的敏感性。在这项工作中，我们介绍了一个名为Channel Lipschitz常数（CLC）的新颖概念，该概念定义为从输入图像到每个通道输出的映射的Lipschitz常数。然后，我们提供经验证据，以显示CLC（UCLC）上限与通道激活的触发激活变化之间的强相关性。由于可以从重量矩阵直接计算UCLC，因此我们可以以无数据的方式检测潜在的后门通道，并在感染的DNN上进行简单修剪以修复模型。提出的基于lipschitzness的通道修剪（CLP）方法非常快速，简单，无数据且可靠，可以选择修剪阈值。进行了广泛的实验来评估CLP的效率和有效性，CLP的效率和有效性也可以在主流防御方法中获得最新的结果。源代码可在https://github.com/rkteddy/channel-lipschitzness基于普通范围内获得。

最近的研究表明，深度神经网络（DNN）容易受到后门攻击的影响。感染的模型正常在良性输入上行为，而其预测将被迫对对抗数据进行攻击特定目标。已经开发了几种检测方法来区分投入来防御这种攻击。这些防御依赖的常见假设是受感染模型提取的清洁和对抗进口的潜在表示之间存在大的统计差异。但是，虽然缺乏假设是真实的重要性，但缺乏全面的研究。在本文中，我们专注于它并研究以下相关问题：1）统计差异的性质是什么？ 2）如何在不损害攻击强度的情况下有效减少它们？ 3）这种减少对基于差异的防御有何影响？我们的工作是在三个问题上进行的。首先，通过引入最大平均差异（MMD）作为指标，我们确定多级表示的统计差异都是大的，而不仅仅是最高级别。然后，我们通过在训练后门模型期间向损耗功能添加多级MMD约束来提出统计差异减少方法（SDRM），以有效降低差异。最后，检查了三种典型的基于差异的检测方法。这些防御的F1分数下降到定期训练的后门型号的90％-100％，在所有两个数据集，四个模型架构和四种攻击方法上用SDRM培训的型号上的60％-70％。结果表明，所提出的方法可用于增强现有攻击以逃避后门检测算法。