我们提出了三种新型的修剪技术，以提高推理意识到的可区分神经结构搜索（DNAS）的成本和结果。首先，我们介绍了DNA的随机双路构建块，它可以通过内存和计算复杂性在内部隐藏尺寸上进行搜索。其次，我们在搜索过程中提出了一种在超级网的随机层中修剪块的算法。第三，我们描述了一种在搜索过程中修剪不必要的随机层的新技术。由搜索产生的优化模型称为Prunet，并在Imagenet Top-1图像分类精度的推理潜伏期中为NVIDIA V100建立了新的最先进的Pareto边界。将Prunet作为骨架还优于COCO对象检测任务的GPUNET和EFIDENENET，相对于平均平均精度（MAP）。

深度学习技术在各种任务中都表现出了出色的有效性，并且深度学习具有推进多种应用程序（包括在边缘计算中）的潜力，其中将深层模型部署在边缘设备上，以实现即时的数据处理和响应。一个关键的挑战是，虽然深层模型的应用通常会产生大量的内存和计算成本，但Edge设备通常只提供非常有限的存储和计算功能，这些功能可能会在各个设备之间差异很大。这些特征使得难以构建深度学习解决方案，以释放边缘设备的潜力，同时遵守其约束。应对这一挑战的一种有希望的方法是自动化有效的深度学习模型的设计，这些模型轻巧，仅需少量存储，并且仅产生低计算开销。该调查提供了针对边缘计算的深度学习模型设计自动化技术的全面覆盖。它提供了关键指标的概述和比较，这些指标通常用于量化模型在有效性，轻度和计算成本方面的水平。然后，该调查涵盖了深层设计自动化技术的三类最新技术：自动化神经体系结构搜索，自动化模型压缩以及联合自动化设计和压缩。最后，调查涵盖了未来研究的开放问题和方向。

大多数对象检测框架都使用最初设计用于图像分类的主链体系结构，通常在Imagenet上具有预训练的参数。但是，图像分类和对象检测本质上是不同的任务，并且不能保证分类的最佳主链也适用于对象检测。最近的神经体系结构搜索（NAS）研究表明，自动设计专门用于对象检测的骨干有助于提高整体准确性。在本文中，我们引入了一种神经体系结构适应方法，该方法可以优化给定的主链以进行检测目的，同时仍允许使用预训练的参数。我们建议除了每个块的输出通道尺寸外，还通过搜索特定操作和层数来调整微体系结构。重要的是要找到最佳的通道深度，因为它极大地影响了特征表示功能和计算成本。我们使用搜索的主链进行对象检测进行实验，并证明我们的主链在可可数据集上的手动设计和搜索的最新骨干均优于手动设计和搜索的骨干。

Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS) methods are computationally expensive. ConvNet architecture optimality depends on factors such as input resolution and target devices. However, existing approaches are too resource demanding for case-by-case redesigns. Also, previous work focuses primarily on reducing FLOPs, but FLOP count does not always reflect actual latency. To address these, we propose a differentiable neural architecture search (DNAS) framework that uses gradient-based methods to optimize Con-vNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. FBNets (Facebook-Berkeley-Nets), a family of models discovered by DNAS surpass state-of-the-art models both designed manually and generated automatically. FBNet-B achieves 74.1% top-1 accuracy on ImageNet with 295M FLOPs and 23.1 ms latency on a Samsung S8 phone, 2.4x smaller and 1.5x faster than MobileNetV2-1.3[17] with similar accuracy. Despite higher accuracy and lower latency than MnasNet[20], we estimate FBNet-B's search cost is 420x smaller than MnasNet's, at only 216 GPUhours. Searched for different resolutions and channel sizes, FBNets achieve 1.5% to 6.4% higher accuracy than Mo-bileNetV2. The smallest FBNet achieves 50.2% accuracy and 2.9 ms latency (345 frames per second) on a Samsung S8. Over a Samsung-optimized FBNet, the iPhone-Xoptimized model achieves a 1.4x speedup on an iPhone X. FBNet models are open-sourced at https://github. com/facebookresearch/mobile-vision. * Work done while interning at Facebook.… Figure 1. Differentiable neural architecture search (DNAS) for ConvNet design. DNAS explores a layer-wise space that each layer of a ConvNet can choose a different block. The search space is represented by a stochastic super net. The search process trains the stochastic super net using SGD to optimize the architecture distribution. Optimal architectures are sampled from the trained distribution. The latency of each operator is measured on target devices and used to compute the loss for the super net.

我们介绍了延迟感知网络加速度（LANA） - 一种在神经结构上建立的方法，用于加速神经网络的神经结构搜索技术和教师学生蒸馏。 Lana由两个阶段组成：在第一阶段，它会使用层面特征映射蒸馏来列举每层教师网络的许多替代操作。在第二阶段，它解决了使用新颖的整数线性优化（ILP）方法的有效操作的组合选择。 ILP带来独特的属性，因为它（i）在几秒钟内执行NAS，（ii）轻松满足预算约束，（iii）在图层粒度上工作，（iv）支持巨大的搜索空间$ o（10 ^ { 100}）$，超越先前的搜索方法，效率和效率。在广泛的实验中，我们表明Lana产生了由目标潜伏期预算限制的有效和准确的模型，同时比其他技术明显快。我们分析了三个流行的网络架构：高效的网络，高效网络和reses，并在压缩较大模型的较小模型的延迟级别时，实现所有型号（高达3.0 \％$）的准确性改进。 Lana通过GPU和CPU实现显着的加速（高达5美元\倍），以没有准确性下降。代码将很快分享。

神经结构搜索（NAS）引起了日益增长的兴趣。为了降低搜索成本，最近的工作已经探讨了模型的重量分享，并在单枪NAS进行了重大进展。然而，已经观察到，单次模型精度较高的模型并不一定在独立培训时更好地执行更好。为了解决这个问题，本文提出了搜索空间的逐步自动设计，名为Pad-NAS。与超字幕中的所有层共享相同操作搜索空间的先前方法不同，我们根据操作修剪制定逐行搜索策略，并构建层面操作搜索空间。通过这种方式，Pad-NAS可以自动设计每层的操作，并在搜索空间质量和模型分集之间实现权衡。在搜索过程中，我们还考虑了高效神经网络模型部署的硬件平台约束。关于Imagenet的广泛实验表明我们的方法可以实现最先进的性能。

Neural architecture search (NAS) has a great impact by automatically designing effective neural network architectures. However, the prohibitive computational demand of conventional NAS algorithms (e.g. 10 4 GPU hours) makes it difficult to directly search the architectures on large-scale tasks (e.g. ImageNet). Differentiable NAS can reduce the cost of GPU hours via a continuous representation of network architecture but suffers from the high GPU memory consumption issue (grow linearly w.r.t. candidate set size). As a result, they need to utilize proxy tasks, such as training on a smaller dataset, or learning with only a few blocks, or training just for a few epochs. These architectures optimized on proxy tasks are not guaranteed to be optimal on the target task. In this paper, we present ProxylessNAS that can directly learn the architectures for large-scale target tasks and target hardware platforms. We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of directness and specialization. On CIFAR-10, our model achieves 2.08% test error with only 5.7M parameters, better than the previous state-of-the-art architecture AmoebaNet-B, while using 6× fewer parameters. On ImageNet, our model achieves 3.1% better top-1 accuracy than MobileNetV2, while being 1.2× faster with measured GPU latency. We also apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design. 1

While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation, and the vision of the Internet of Things fuel the interest in resource-efficient approaches. These approaches aim for a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. The development of such approaches is among the major challenges in current machine learning research and key to ensure a smooth transition of machine learning technology from a scientific environment with virtually unlimited computing resources into everyday's applications. In this article, we provide an overview of the current state of the art of machine learning techniques facilitating these real-world requirements. In particular, we focus on deep neural networks (DNNs), the predominant machine learning models of the past decade. We give a comprehensive overview of the vast literature that can be mainly split into three non-mutually exclusive categories: (i) quantized neural networks, (ii) network pruning, and (iii) structural efficiency. These techniques can be applied during training or as post-processing, and they are widely used to reduce the computational demands in terms of memory footprint, inference speed, and energy efficiency. We also briefly discuss different concepts of embedded hardware for DNNs and their compatibility with machine learning techniques as well as potential for energy and latency reduction. We substantiate our discussion with experiments on well-known benchmark datasets using compression techniques (quantization, pruning) for a set of resource-constrained embedded systems, such as CPUs, GPUs and FPGAs. The obtained results highlight the difficulty of finding good trade-offs between resource efficiency and predictive performance.

由于存储器和计算资源有限，部署在移动设备上的卷积神经网络（CNNS）是困难的。我们的目标是通过利用特征图中的冗余来设计包括CPU和GPU的异构设备的高效神经网络，这很少在神经结构设计中进行了研究。对于类似CPU的设备，我们提出了一种新颖的CPU高效的Ghost（C-Ghost）模块，以生成从廉价操作的更多特征映射。基于一组内在的特征映射，我们使用廉价的成本应用一系列线性变换，以生成许多幽灵特征图，可以完全揭示内在特征的信息。所提出的C-Ghost模块可以作为即插即用组件，以升级现有的卷积神经网络。 C-Ghost瓶颈旨在堆叠C-Ghost模块，然后可以轻松建立轻量级的C-Ghostnet。我们进一步考虑GPU设备的有效网络。在建筑阶段的情况下，不涉及太多的GPU效率（例如，深度明智的卷积），我们建议利用阶段明智的特征冗余来制定GPU高效的幽灵（G-GHOST）阶段结构。舞台中的特征被分成两个部分，其中使用具有较少输出通道的原始块处理第一部分，用于生成内在特征，另一个通过利用阶段明智的冗余来生成廉价的操作。在基准测试上进行的实验证明了所提出的C-Ghost模块和G-Ghost阶段的有效性。 C-Ghostnet和G-Ghostnet分别可以分别实现CPU和GPU的准确性和延迟的最佳权衡。代码可在https://github.com/huawei-noah/cv-backbones获得。

在这项工作中，我们提出了一种方法，以准确评估和比较有效的神经网络构建块的性能，以硬件感知方式进行计算机视觉。我们的比较使用了基于设计空间的随机采样网络的帕累托前沿来捕获潜在的准确性/复杂性权衡。我们表明，我们的方法允许通过以前的比较范例获得的信息匹配，但对硬件成本和准确性之间的关系提供了更多见解。我们使用我们的方法来分析不同的构件并评估其在一系列嵌入式硬件平台上的性能。这突出了基准构建块作为神经网络设计过程中的预选步骤的重要性。我们表明，选择合适的构件可以在特定硬件ML加速器上加快推理的速度2倍。

是否可以在深网络中重组非线性激活函数以创建硬件有效的模型？为了解决这个问题，我们提出了一个称为重组激活网络（RANS）的新范式，该范式操纵模型中的非线性数量以提高其硬件意识和效率。首先，我们提出了RAN-STHICER（RAN-E） - 一个新的硬件感知搜索空间和半自动搜索算法 - 用硬件感知的块替换效率低下的块。接下来，我们提出了一种称为RAN-IMPLICIC（RAN-I）的无训练模型缩放方法，从理论上讲，我们在非线性单元的数量方面证明了网络拓扑与其表现性之间的联系。我们证明，我们的网络在不同尺度和几种类型的硬件上实现最新的成像网结果。例如，与有效网络-lite-B0相比，RAN-E在ARM Micro-NPU上每秒（FPS）提高了1.5倍，同时提高了类似的精度。另一方面，ran-i以相似或更好的精度表现出#macs的#macs降低2倍。我们还表明，在基于ARM的数据中心CPU上，RAN-I的FPS比Convnext高40％。最后，与基于Convnext的模型相比，基于RAN-I的对象检测网络在数据中心CPU上获得了类似或更高的映射，并且在数据中心CPU上的fps高达33％。

不断需要在低容量设备上使用的图像超分辨率（SR）的高性能和计算有效的神经网络模型。获取此类模型的一种方法是压缩现有体系结构，例如量化。另一个选择是发现新的有效解决方案的神经体系结构搜索（NAS）。我们为专门设计的SR搜索空间提出了一种新颖的量化NAS程序。我们的方法执行NAS以找到量化友好的SR模型。搜索依赖于将量化噪声添加到参数和激活中，而不是直接量化参数。我们的Quontnas比固定体系结构的均匀或混合精度量化找到了具有更好的PSNR/BITOP权衡的体系结构。此外，我们对噪声过程的搜索比直接量化权重的速度快30％。

可区分架构搜索（飞镖）是基于解决双重优化问题的数据驱动神经网络设计的有效方法。尽管在许多体系结构搜索任务中取得了成功，但仍然担心一阶飞镖的准确性和二阶飞镖的效率。在本文中，我们制定了单个级别的替代方案和放松的体系结构搜索（RARTS）方法，该方法通过数据和网络拆分利用整个数据集在体系结构学习中，而无需涉及相应损失功能（如飞镖）的混合第二个衍生物。在我们制定网络拆分的过程中，两个具有不同但相关权重的网络在寻找共享体系结构时进行了合作。 RART比飞镖的优势通过收敛定理和可解析的模型证明是合理的。此外，RART在准确性和搜索效率方面优于飞镖及其变体，如足够的实验结果所示。对于搜索拓扑结构（即边缘和操作）的任务，RART获得了比CIFAR-10上的二阶Darts更高的精度和60 \％的计算成本降低。转移到Imagenet时，RART继续超越表演飞镖，并且与最近的飞镖变体相提并论，尽管我们的创新纯粹是在训练算法上，而无需修改搜索空间。对于搜索宽度的任务，即卷积层中的频道数量，RARTS还优于传统的网络修剪基准。关于公共体系结构搜索基准等NATS BENCH的进一步实验也支持RARTS的优势。

Recently, Neural architecture search has achieved great success on classification tasks for mobile devices. The backbone network for object detection is usually obtained on the image classification task. However, the architecture which is searched through the classification task is sub-optimal because of the gap between the task of image and object detection. As while work focuses on backbone network architecture search for mobile device object detection is limited, mainly because the backbone always requires expensive ImageNet pre-training. Accordingly, it is necessary to study the approach of network architecture search for mobile device object detection without expensive pre-training. In this work, we propose a mobile object detection backbone network architecture search algorithm which is a kind of evolutionary optimized method based on non-dominated sorting for NAS scenarios. It can quickly search to obtain the backbone network architecture within certain constraints. It better solves the problem of suboptimal linear combination accuracy and computational cost. The proposed approach can search the backbone networks with different depths, widths, or expansion sizes via a technique of weight mapping, making it possible to use NAS for mobile devices detection tasks a lot more efficiently. In our experiments, we verify the effectiveness of the proposed approach on YoloX-Lite, a lightweight version of the target detection framework. Under similar computational complexity, the accuracy of the backbone network architecture we search for is 2.0% mAP higher than MobileDet. Our improved backbone network can reduce the computational effort while improving the accuracy of the object detection network. To prove its effectiveness, a series of ablation studies have been carried out and the working mechanism has been analyzed in detail.

Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8× faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3× faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/ tree/master/models/official/mnasnet.

Structured channel pruning has been shown to significantly accelerate inference time for convolution neural networks (CNNs) on modern hardware, with a relatively minor loss of network accuracy. Recent works permanently zero these channels during training, which we observe to significantly hamper final accuracy, particularly as the fraction of the network being pruned increases. We propose Soft Masking for cost-constrained Channel Pruning (SMCP) to allow pruned channels to adaptively return to the network while simultaneously pruning towards a target cost constraint. By adding a soft mask re-parameterization of the weights and channel pruning from the perspective of removing input channels, we allow gradient updates to previously pruned channels and the opportunity for the channels to later return to the network. We then formulate input channel pruning as a global resource allocation problem. Our method outperforms prior works on both the ImageNet classification and PASCAL VOC detection datasets.

在过去几年中，已经制作了神经结构搜索领域的显着改进。然而，由于存在搜索的约束和实际推断时间之间的间隙，搜索有效网络仍然具有挑战性。为了搜索具有低推理时间的高性能网络，若干以前的作品为搜索算法设置了计算复杂性约束。然而，许多因素影响推理的速度（例如，拖鞋，MAC）。单个指示符与延迟之间的相关性并不强。目前，提出了一些重新参数化（REP）技术将多分支转换为对单路径架构进行推断友好的。然而，多分支架构仍然是人为定义和效率低下。在这项工作中，我们提出了一种适用于结构重新参数化技术的新搜索空间。 repnas是一种单级NAS方法，以便在分支号约束下有效地搜索每个层的最佳分支块（ODBB）。我们的实验结果表明，搜索的ODBB可以轻松超越手动各种分支块（DBB），高效培训。代码和型号将越早提供。

嵌入式和个人物联网设备由微控制器单元（MCU）供电，其极端资源稀缺是依赖于设备的深度学习推断的应用的主要障碍。与通常需要执行神经网络的内容相比，存储器，内存和计算能力较少的秩序，对网络架构上的严格结构约束并呼叫专业模型压缩方法。在这项工作中，我们为卷积神经网络提出了可分散的结构化网络修剪方法，它集成了模型的MCU特定的资源使用和参数重要性反馈，以获得高度压缩但准确的分类模型。我们的方法（a）提高了高达80倍的模型的关键资源使用; （b）在培训型号的同时迭代地修剪，导致没有开销甚至改善培训时间; （c）与现有MCU的特定方法相比，在比较多的时间内生产具有匹配或改进的资源使用的压缩模型。压缩模型可供下载。

混合精确的深神经网络达到了硬件部署所需的能源效率和吞吐量，尤其是在资源有限的情况下，而无需牺牲准确性。但是，不容易找到保留精度的最佳每层钻头精度，尤其是在创建巨大搜索空间的大量模型，数据集和量化技术中。为了解决这一困难，最近出现了一系列文献，并且已经提出了一些实现有希望的准确性结果的框架。在本文中，我们首先总结了文献中通常使用的量化技术。然后，我们对混合精液框架进行了彻底的调查，该调查是根据其优化技术进行分类的，例如增强学习和量化技术，例如确定性舍入。此外，讨论了每个框架的优势和缺点，我们在其中呈现并列。我们最终为未来的混合精液框架提供了指南。

知识蒸馏（KD）最近成为压缩神经网络的一种流行方法。在最近的研究中，已经提出了同时找到学生模型的参数和体系结构的广义蒸馏方法。尽管如此，这种搜索方法仍需要大量的计算来搜索体系结构，并且缺点是仅考虑其搜索空间中的卷积块。本文介绍了一种新的算法，认为是信任区域意识架构搜索以有效提炼知识（贸易），该算法迅速找到了使用信任区域贝叶斯优化方法从几种最先进的架构中找到有效的学生体系结构。实验结果表明，我们提出的贸易算法始终优于KD培训下的常规NAS方法和预定义的架构。