在这项工作中,我们提出了一种方法,以准确评估和比较有效的神经网络构建块的性能,以硬件感知方式进行计算机视觉。我们的比较使用了基于设计空间的随机采样网络的帕累托前沿来捕获潜在的准确性/复杂性权衡。我们表明,我们的方法允许通过以前的比较范例获得的信息匹配,但对硬件成本和准确性之间的关系提供了更多见解。我们使用我们的方法来分析不同的构件并评估其在一系列嵌入式硬件平台上的性能。这突出了基准构建块作为神经网络设计过程中的预选步骤的重要性。我们表明,选择合适的构件可以在特定硬件ML加速器上加快推理的速度2倍。
translated by 谷歌翻译
We present the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design. MobileNetV3 is tuned to mobile phone CPUs through a combination of hardwareaware network architecture search (NAS) complemented by the NetAdapt algorithm and then subsequently improved through novel architecture advances. This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art. Through this process we create two new MobileNet models for release: MobileNetV3-Large and MobileNetV3-Small which are targeted for high and low resource use cases. These models are then adapted and applied to the tasks of object detection and semantic segmentation. For the task of semantic segmentation (or any dense pixel prediction), we propose a new efficient segmentation decoder Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP). We achieve new state of the art results for mobile classification, detection and segmentation. MobileNetV3-Large is 3.2% more accurate on ImageNet classification while reducing latency by 20% compared to MobileNetV2. MobileNetV3-Small is 6.6% more accurate compared to a MobileNetV2 model with comparable latency. MobileNetV3-Large detection is over 25% faster at roughly the same accuracy as Mo-bileNetV2 on COCO detection. MobileNetV3-Large LR-ASPP is 34% faster than MobileNetV2 R-ASPP at similar accuracy for Cityscapes segmentation.
translated by 谷歌翻译
In this work, we present a new network design paradigm. Our goal is to help advance the understanding of network design and discover design principles that generalize across settings. Instead of focusing on designing individual network instances, we design network design spaces that parametrize populations of networks. The overall process is analogous to classic manual design of networks, but elevated to the design space level. Using our methodology we explore the structure aspect of network design and arrive at a low-dimensional design space consisting of simple, regular networks that we call RegNet. The core insight of the RegNet parametrization is surprisingly simple: widths and depths of good networks can be explained by a quantized linear function. We analyze the RegNet design space and arrive at interesting findings that do not match the current practice of network design. The RegNet design space provides simple and fast networks that work well across a wide range of flop regimes. Under comparable training settings and flops, the RegNet models outperform the popular Effi-cientNet models while being up to 5× faster on GPUs.
translated by 谷歌翻译
卷积神经网络(CNNS)用于许多现实世界应用,例如基于视觉的自主驾驶和视频内容分析。要在各种目标设备上运行CNN推断,硬件感知神经结构搜索(NAS)至关重要。有效的硬件感知NAS的关键要求是对推理延迟的快速评估,以便对不同的架构进行排名。在构建每个目标设备的延迟预测器的同时,在本领域中通常使用,这是一个非常耗时的过程,在极定的设备存在下缺乏可扩展性。在这项工作中,我们通过利用延迟单调性来解决可扩展性挑战 - 不同设备上的架构延迟排名通常相关。当存在强烈的延迟单调性时,我们可以重复使用在新目标设备上搜索一个代理设备的架构,而不会丢失最佳状态。在没有强烈的延迟单调性的情况下,我们提出了一种有效的代理适应技术,以显着提高延迟单调性。最后,我们验证了我们的方法,并在多个主流搜索空间上使用不同平台的设备进行实验,包括MobileNet-V2,MobileNet-V3,NAS-Bench-201,Proxylessnas和FBNet。我们的结果突出显示,通过仅使用一个代理设备,我们可以找到几乎与现有的每个设备NAS相同的帕累托最优架构,同时避免为每个设备构建延迟预测器的禁止成本。 github:https://github.com/ren-research/oneproxy.
translated by 谷歌翻译
深度学习技术在各种任务中都表现出了出色的有效性,并且深度学习具有推进多种应用程序(包括在边缘计算中)的潜力,其中将深层模型部署在边缘设备上,以实现即时的数据处理和响应。一个关键的挑战是,虽然深层模型的应用通常会产生大量的内存和计算成本,但Edge设备通常只提供非常有限的存储和计算功能,这些功能可能会在各个设备之间差异很大。这些特征使得难以构建深度学习解决方案,以释放边缘设备的潜力,同时遵守其约束。应对这一挑战的一种有希望的方法是自动化有效的深度学习模型的设计,这些模型轻巧,仅需少量存储,并且仅产生低计算开销。该调查提供了针对边缘计算的深度学习模型设计自动化技术的全面覆盖。它提供了关键指标的概述和比较,这些指标通常用于量化模型在有效性,轻度和计算成本方面的水平。然后,该调查涵盖了深层设计自动化技术的三类最新技术:自动化神经体系结构搜索,自动化模型压缩以及联合自动化设计和压缩。最后,调查涵盖了未来研究的开放问题和方向。
translated by 谷歌翻译
We introduce an extremely computation-efficient CNN architecture named ShuffleNet, which is designed specially for mobile devices with very limited computing power (e.g., 10-150 MFLOPs). The new architecture utilizes two new operations, pointwise group convolution and channel shuffle, to greatly reduce computation cost while maintaining accuracy. Experiments on ImageNet classification and MS COCO object detection demonstrate the superior performance of ShuffleNet over other structures, e.g. lower top-1 error (absolute 7.8%) than recent MobileNet [12] on Ima-geNet classification task, under the computation budget of 40 MFLOPs. On an ARM-based mobile device, ShuffleNet achieves ∼13× actual speedup over AlexNet while maintaining comparable accuracy.
translated by 谷歌翻译
While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation, and the vision of the Internet of Things fuel the interest in resource-efficient approaches. These approaches aim for a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. The development of such approaches is among the major challenges in current machine learning research and key to ensure a smooth transition of machine learning technology from a scientific environment with virtually unlimited computing resources into everyday's applications. In this article, we provide an overview of the current state of the art of machine learning techniques facilitating these real-world requirements. In particular, we focus on deep neural networks (DNNs), the predominant machine learning models of the past decade. We give a comprehensive overview of the vast literature that can be mainly split into three non-mutually exclusive categories: (i) quantized neural networks, (ii) network pruning, and (iii) structural efficiency. These techniques can be applied during training or as post-processing, and they are widely used to reduce the computational demands in terms of memory footprint, inference speed, and energy efficiency. We also briefly discuss different concepts of embedded hardware for DNNs and their compatibility with machine learning techniques as well as potential for energy and latency reduction. We substantiate our discussion with experiments on well-known benchmark datasets using compression techniques (quantization, pruning) for a set of resource-constrained embedded systems, such as CPUs, GPUs and FPGAs. The obtained results highlight the difficulty of finding good trade-offs between resource efficiency and predictive performance.
translated by 谷歌翻译
Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8× faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3× faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/ tree/master/models/official/mnasnet.
translated by 谷歌翻译
深神经网络(DNNS)在各种机器学习(ML)应用程序中取得了巨大成功,在计算机视觉,自然语言处理和虚拟现实等中提供了高质量的推理解决方案。但是,基于DNN的ML应用程序也带来计算和存储要求的增加了很多,对于具有有限的计算/存储资源,紧张的功率预算和较小形式的嵌入式系统而言,这尤其具有挑战性。挑战还来自各种特定应用的要求,包括实时响应,高通量性能和可靠的推理准确性。为了应对这些挑战,我们介绍了一系列有效的设计方法,包括有效的ML模型设计,定制的硬件加速器设计以及硬件/软件共同设计策略,以启用嵌入式系统上有效的ML应用程序。
translated by 谷歌翻译
Currently, the neural network architecture design is mostly guided by the indirect metric of computation complexity, i.e., FLOPs. However, the direct metric, e.g., speed, also depends on the other factors such as memory access cost and platform characterics. Thus, this work proposes to evaluate the direct metric on the target platform, beyond only considering FLOPs. Based on a series of controlled experiments, this work derives several practical guidelines for efficient network design. Accordingly, a new architecture is presented, called ShuffleNet V2.Comprehensive ablation experiments verify that our model is the stateof-the-art in terms of speed and accuracy tradeoff.
translated by 谷歌翻译
我们提出了三种新型的修剪技术,以提高推理意识到的可区分神经结构搜索(DNAS)的成本和结果。首先,我们介绍了DNA的随机双路构建块,它可以通过内存和计算复杂性在内部隐藏尺寸上进行搜索。其次,我们在搜索过程中提出了一种在超级网的随机层中修剪块的算法。第三,我们描述了一种在搜索过程中修剪不必要的随机层的新技术。由搜索产生的优化模型称为Prunet,并在Imagenet Top-1图像分类精度的推理潜伏期中为NVIDIA V100建立了新的最先进的Pareto边界。将Prunet作为骨架还优于COCO对象检测任务的GPUNET和EFIDENENET,相对于平均平均精度(MAP)。
translated by 谷歌翻译
In this paper we describe a new mobile architecture, MobileNetV2, that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes. We also describe efficient ways of applying these mobile models to object detection in a novel framework we call SSDLite. Additionally, we demonstrate how to build mobile semantic segmentation models through a reduced form of DeepLabv3 which we call Mobile DeepLabv3.is based on an inverted residual structure where the shortcut connections are between the thin bottleneck layers. The intermediate expansion layer uses lightweight depthwise convolutions to filter features as a source of non-linearity. Additionally, we find that it is important to remove non-linearities in the narrow layers in order to maintain representational power. We demonstrate that this improves performance and provide an intuition that led to this design.Finally, our approach allows decoupling of the input/output domains from the expressiveness of the transformation, which provides a convenient framework for further analysis. We measure our performance on ImageNet [1] classification, COCO object detection [2], VOC image segmentation [3]. We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as actual latency, and the number of parameters.
translated by 谷歌翻译
为了部署,神经架构搜索应该是硬件感知的,以满足设备特定的约束(例如,内存使用,延迟和能量消耗),并提高模型效率。硬件感知NAS的现有方法从目标设备收集大量样本(例如,精度和延迟),要么构建查找表或延迟估计器。然而,这种方法在现实世界方案中是不切实际的,因为存在具有不同硬件规格的许多器件,并从这些大量设备收集样本将需要禁止的计算和货币成本。为了克服这些限制,我们提出了硬件 - 自适应高效延迟预测器(帮助),其将设备特定的延迟估计问题交给了元学习问题,使得我们可以估计模型对给定任务的性能的延迟有一些样品的看不见的装置。为此,我们引入了新颖的硬件嵌入,将任何设备嵌入,将其视为输出延迟的黑盒功能,并使用硬件嵌入式以设备依赖方式学习硬件自适应延迟预测器。我们验证了在看不见的平台上实现了延迟估计性能的提议帮助,其中它达到了高估计性能,少于10个测量样本,优于所有相关基线。我们还验证了在没有它的帮助下使用帮助的端到端NAS框架,并表明它在很大程度上降低了基础NAS方法的总时间成本,在延迟约束的设置中。代码可在https://github.com/hayeonlee/help获得。
translated by 谷歌翻译
Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS) methods are computationally expensive. ConvNet architecture optimality depends on factors such as input resolution and target devices. However, existing approaches are too resource demanding for case-by-case redesigns. Also, previous work focuses primarily on reducing FLOPs, but FLOP count does not always reflect actual latency. To address these, we propose a differentiable neural architecture search (DNAS) framework that uses gradient-based methods to optimize Con-vNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. FBNets (Facebook-Berkeley-Nets), a family of models discovered by DNAS surpass state-of-the-art models both designed manually and generated automatically. FBNet-B achieves 74.1% top-1 accuracy on ImageNet with 295M FLOPs and 23.1 ms latency on a Samsung S8 phone, 2.4x smaller and 1.5x faster than MobileNetV2-1.3[17] with similar accuracy. Despite higher accuracy and lower latency than MnasNet[20], we estimate FBNet-B's search cost is 420x smaller than MnasNet's, at only 216 GPUhours. Searched for different resolutions and channel sizes, FBNets achieve 1.5% to 6.4% higher accuracy than Mo-bileNetV2. The smallest FBNet achieves 50.2% accuracy and 2.9 ms latency (345 frames per second) on a Samsung S8. Over a Samsung-optimized FBNet, the iPhone-Xoptimized model achieves a 1.4x speedup on an iPhone X. FBNet models are open-sourced at https://github. com/facebookresearch/mobile-vision. * Work done while interning at Facebook.… Figure 1. Differentiable neural architecture search (DNAS) for ConvNet design. DNAS explores a layer-wise space that each layer of a ConvNet can choose a different block. The search space is represented by a stochastic super net. The search process trains the stochastic super net using SGD to optimize the architecture distribution. Optimal architectures are sampled from the trained distribution. The latency of each operator is measured on target devices and used to compute the loss for the super net.
translated by 谷歌翻译
由于存储器和计算资源有限,部署在移动设备上的卷积神经网络(CNNS)是困难的。我们的目标是通过利用特征图中的冗余来设计包括CPU和GPU的异构设备的高效神经网络,这很少在神经结构设计中进行了研究。对于类似CPU的设备,我们提出了一种新颖的CPU高效的Ghost(C-Ghost)模块,以生成从廉价操作的更多特征映射。基于一组内在的特征映射,我们使用廉价的成本应用一系列线性变换,以生成许多幽灵特征图,可以完全揭示内在特征的信息。所提出的C-Ghost模块可以作为即插即用组件,以升级现有的卷积神经网络。 C-Ghost瓶颈旨在堆叠C-Ghost模块,然后可以轻松建立轻量级的C-Ghostnet。我们进一步考虑GPU设备的有效网络。在建筑阶段的情况下,不涉及太多的GPU效率(例如,深度明智的卷积),我们建议利用阶段明智的特征冗余来制定GPU高效的幽灵(G-GHOST)阶段结构。舞台中的特征被分成两个部分,其中使用具有较少输出通道的原始块处理第一部分,用于生成内在特征,另一个通过利用阶段明智的冗余来生成廉价的操作。在基准测试上进行的实验证明了所提出的C-Ghost模块和G-Ghost阶段的有效性。 C-Ghostnet和G-Ghostnet分别可以分别实现CPU和GPU的准确性和延迟的最佳权衡。代码可在https://github.com/huawei-noah/cv-backbones获得。
translated by 谷歌翻译
是否可以在深网络中重组非线性激活函数以创建硬件有效的模型?为了解决这个问题,我们提出了一个称为重组激活网络(RANS)的新范式,该范式操纵模型中的非线性数量以提高其硬件意识和效率。首先,我们提出了RAN-STHICER(RAN-E) - 一个新的硬件感知搜索空间和半自动搜索算法 - 用硬件感知的块替换效率低下的块。接下来,我们提出了一种称为RAN-IMPLICIC(RAN-I)的无训练模型缩放方法,从理论上讲,我们在非线性单元的数量方面证明了网络拓扑与其表现性之间的联系。我们证明,我们的网络在不同尺度和几种类型的硬件上实现最新的成像网结果。例如,与有效网络-lite-B0相比,RAN-E在ARM Micro-NPU上每秒(FPS)提高了1.5倍,同时提高了类似的精度。另一方面,ran-i以相似或更好的精度表现出#macs的#macs降低2倍。我们还表明,在基于ARM的数据中心CPU上,RAN-I的FPS比Convnext高40%。最后,与基于Convnext的模型相比,基于RAN-I的对象检测网络在数据中心CPU上获得了类似或更高的映射,并且在数据中心CPU上的fps高达33%。
translated by 谷歌翻译
There has been significant progress in developing neural network architectures that both achieve high predictive performance and that also achieve high application-level inference throughput (e.g., frames per second). Another metric of increasing importance is GPU utilization during inference: the measurement of how well a deployed neural network uses the computational capabilities of the GPU on which it runs. Achieving high GPU utilization is critical to increasing application-level throughput and ensuring a good return on investment for deploying GPUs. This paper analyzes the GPU utilization of convolutional neural network (CNN) inference. We first survey the GPU utilization of CNNs to show that there is room to improve the GPU utilization of many of these CNNs. We then investigate the GPU utilization of networks within a neural architecture search (NAS) search space, and explore how using GPU utilization as a metric could potentially be used to accelerate NAS itself. Our study makes the case that there is room to improve the inference-time GPU utilization of CNNs and that knowledge of GPU utilization has the potential to benefit even applications that do not target utilization itself. We hope that the results of this study will spur future innovation in designing GPU-efficient neural networks.
translated by 谷歌翻译
近年来,深度学习在许多实际问题中取得了重大结果,例如计算机视觉,自然语言处理,语音识别等。多年来,该研究的主要目标是提高模型的质量,即使复杂性不切实际。但是,对于通常需要实时工作的生产解决方案而言,该模型的延迟起着非常重要的作用。通过将模型复杂性考虑到神经体系结构搜索(NAS),发现了当前的最新架构。但是,设计适合特定硬件的搜索空间仍然是一项艰巨的任务。为了解决这个问题,我们提出了神经体系结构搜索空间的硬件效率的度量 - 矩阵效率度量(MEM);由硬件有效操作组成的搜索空间;一种延迟感知的缩放方法;和Isynet-一组架构,旨在在专门的神经加工单元(NPU)硬件上快速进行快速,同时准确。我们展示了NPU设备在ImageNet上设计的架构的优势以及下游分类和检测任务的概括能力。
translated by 谷歌翻译
基于自我注意力的模型,例如视觉变压器(VIT),已经成为计算机视觉中卷积神经网络(CNN)的一种非常有竞争力的建筑。尽管越来越高的变体具有更高的识别精度,但由于自我注意力的二次复杂性,现有的VIT通常在计算和模型大小中要求。尽管已重新引入了最近的CNN的几种成功设计选择(例如,卷积和分层多阶段结构)已重新引入最近的VIT,但它们仍然不足以满足移动设备的有限资源要求。这激发了最近根据最先进的Mobilenet-V2开发光线的尝试,但仍然留下了性能差距。在这项工作中,在这个研究不足的方向上进一步推动了Edgevits,这是一个新的轻巧vits家族,这首先使基于注意力的视觉模型能够与最佳轻巧的CNN竞争,这准确性和设备效率。这是通过基于自我注意力和卷积的最佳整合而引入高度成本效益的本地 - 全球局(LGL)信息交换瓶颈来实现的。对于设备青年的评估,我们不再依赖诸如拖船或参数的不准确代理,而是采用一种实用的方法来直接专注于设备延迟,以及首次首次提供能源效率。具体而言,我们表明,当考虑准确性的延迟和准确性 - 能量折衷时,我们的模型是帕累托最佳的,在几乎所有情况下都严格占据了其他VIT并与最有效的CNN竞争的严格优势。代码可从https://github.com/saic-fi/edgevit获得。
translated by 谷歌翻译
我们介绍了延迟感知网络加速度(LANA) - 一种在神经结构上建立的方法,用于加速神经网络的神经结构搜索技术和教师学生蒸馏。 Lana由两个阶段组成:在第一阶段,它会使用层面特征映射蒸馏来列举每层教师网络的许多替代操作。在第二阶段,它解决了使用新颖的整数线性优化(ILP)方法的有效操作的组合选择。 ILP带来独特的属性,因为它(i)在几秒钟内执行NAS,(ii)轻松满足预算约束,(iii)在图层粒度上工作,(iv)支持巨大的搜索空间$ o(10 ^ { 100})$,超越先前的搜索方法,效率和效率。在广泛的实验中,我们表明Lana产生了由目标潜伏期预算限制的有效和准确的模型,同时比其他技术明显快。我们分析了三个流行的网络架构:高效的网络,高效网络和reses,并在压缩较大模型的较小模型的延迟级别时,实现所有型号(高达3.0 \%$)的准确性改进。 Lana通过GPU和CPU实现显着的加速(高达5美元\倍),以没有准确性下降。代码将很快分享。
translated by 谷歌翻译