我们呈现Fouriermask，它采用傅立叶系列与隐式的神经表示结合起来，以产生实例分段掩模。我们将傅里叶映射（FM）应用于坐标位置，并利用映射特征作为隐式表示的输入（基于坐标的多层Perceptron（MLP））。 FouriMASK学习为特定实例预测FM的系数，因此将FM适应特定对象。这允许Fouriermask广泛化以预测来自自然图像的实例分段掩模。由于隐式功能在输入坐标的域中是连续的，因此我们说明通过对输入像素坐标进行分采样，因此我们可以在推理期间生成更高的分辨率掩码。此外，我们在Fourimask的不确定预测上培训渲染器MLP（Fourrierrend），并说明它显着提高了面具的质量。与在相同输出分辨率的基线掩模R-CNN相比，Fourimask在MS Coco DataSet上显示竞争结果，并在更高分辨率上超越它。

We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over-and undersampling challenges faced in pixel labeling tasks, we develop a unique perspective of image segmentation as a rendering problem. From this vantage, we present the PointRend (Point-based Rendering) neural network module: a module that performs point-based segmentation predictions at adaptively selected locations based on an iterative subdivision algorithm. PointRend can be flexibly applied to both instance and semantic segmentation tasks by building on top of existing state-ofthe-art models. While many concrete implementations of the general idea are possible, we show that a simple design already achieves excellent results. Qualitatively, PointRend outputs crisp object boundaries in regions that are oversmoothed by previous methods. Quantitatively, PointRend yields significant gains on COCO and Cityscapes, for both instance and semantic segmentation. PointRend's efficiency enables output resolutions that are otherwise impractical in terms of memory or computation compared to existing approaches. Code has been made available at https:// github.com/facebookresearch/detectron2/ tree/master/projects/PointRend.

We propose a simple yet effective instance segmentation framework, termed CondInst (conditional convolutions for instance segmentation). Top-performing instance segmentation methods such as Mask R-CNN rely on ROI operations (typically ROIPool or ROIAlign) to obtain the final instance masks. In contrast, we propose to solve instance segmentation from a new perspective. Instead of using instancewise ROIs as inputs to a network of fixed weights, we employ dynamic instance-aware networks, conditioned on instances. CondInst enjoys two advantages: 1) Instance segmentation is solved by a fully convolutional network, eliminating the need for ROI cropping and feature alignment.2) Due to the much improved capacity of dynamically-generated conditional convolutions, the mask head can be very compact (e.g., 3 conv. layers, each having only 8 channels), leading to significantly faster inference. We demonstrate a simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed. On the COCO dataset, we outperform a few recent methods including welltuned Mask R-CNN baselines, without longer training schedules needed.

In this paper, we introduce an anchor-box free and single shot instance segmentation method, which is conceptually simple, fully convolutional and can be used by easily embedding it into most off-the-shelf detection methods. Our method, termed PolarMask, formulates the instance segmentation problem as predicting contour of instance through instance center classification and dense distance regression in a polar coordinate. Moreover, we propose two effective approaches to deal with sampling high-quality center examples and optimization for dense distance regression, respectively, which can significantly improve the performance and simplify the training process. Without any bells and whistles, PolarMask achieves 32.9% in mask mAP with single-model and single-scale training/testing on the challenging COCO dataset.For the first time, we show that the complexity of instance segmentation, in terms of both design and computation complexity, can be the same as bounding box object detection and this much simpler and flexible instance segmentation framework can achieve competitive accuracy. We hope that the proposed PolarMask framework can serve as a fundamental and strong baseline for single shot instance segmentation task. Code is available at: github.com/xieenze/PolarMask.

We present a new, embarrassingly simple approach to instance segmentation. Compared to many other dense prediction tasks, e.g., semantic segmentation, it is the arbitrary number of instances that have made instance segmentation much more challenging. In order to predict a mask for each instance, mainstream approaches either follow the "detect-then-segment" strategy (e.g., Mask R-CNN), or predict embedding vectors first then use clustering techniques to group pixels into individual instances. We view the task of instance segmentation from a completely new perspective by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size, thus nicely converting instance segmentation into a single-shot classification-solvable problem. We demonstrate a much simpler and flexible instance segmentation framework with strong performance, achieving on par accuracy with Mask R-CNN and outperforming recent single-shot instance segmenters in accuracy. We hope that this simple and strong framework can serve as a baseline for many instance-level recognition tasks besides instance segmentation. Code is available at https://git.io/AdelaiDet

Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models, and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at https:// github.com/zjhuang22/maskscoring_rcnn. * The work was done when Zhaojin Huang was an intern in Horizon Robotics Inc.

当与输入的高维投影结合使用时，多层感知器（MLP）已被证明是有效的场景编码器，通常称为\ textit {位置{位置编码}。但是，频谱频谱的场景仍然是一个挑战：选择高频进行位置编码会引入低结构区域中的噪声，而低频率则导致详细区域的拟合不佳。为了解决这个问题，我们提出了一个渐进的位置编码，将分层MLP结构暴露于频率编码的增量集。我们的模型可以准确地使用广泛的频带重建场景，并以细节的渐进级别学习场景表示形式\ textit {没有明确的每级监督}。该体系结构是模块化的：每个级别都编码一个连续的隐式表示，可以分别利用其各自的分辨率，这意味着一个较小的网络来进行更粗糙的重建。与基线相比，几个2D和3D数据集的实验显示了重建精度，代表性能力和训练速度的提高。

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without tricks, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code will be made available.

两阶段和基于查询的实例分段方法取得了显着的结果。然而，他们的分段面具仍然非常粗糙。在本文中，我们呈现了用于高质量高效的实例分割的掩模转发器。我们的掩模转发器代替常规密集的张量，而不是在常规密集的张量上进行分解，并表示作为Quadtree的图像区域。我们基于变换器的方法仅处理检测到的错误易于树节点，并并行自我纠正其错误。虽然这些稀疏的像素仅构成总数的小比例，但它们对最终掩模质量至关重要。这允许掩模转换器以低计算成本预测高精度的实例掩模。广泛的实验表明，掩模转发器在三个流行的基准上优于当前实例分段方法，显着改善了COCO和BDD100K上的大型+3.0掩模AP的+3.0掩模AP的大余量和CityScapes上的+6.6边界AP。我们的代码和培训的型号将在http://vis.xyz/pub/transfiner提供。

隐式神经表示（INR）使用多层的感知来代表低维问题域中的高频函数。最近，这些表示在与复杂的3D对象和场景相关的任务上实现了最先进的结果。核心问题是高度详细信号的表示，其使用具有周期性激活功能（警报器）的网络来解决或将傅立叶映射应用于输入。这项工作分析了两种方法之间的连接，并表明傅里叶映射的Perceptron在结构上像一个隐藏层警报器。此外，我们确定先前提出的傅里叶映射与一般D维傅里叶系列之间的关系，导致整数晶格映射。此外，我们修改了渐进式培训策略，以便在任意傅里叶映射上工作，并表明它提高了插值任务的泛化。最后，我们比较图像回归和新颖观看综合任务的不同映射。我们确认前面发现映射性能的主要贡献者是其元素的嵌入和标准偏差的大小。

我们提出了一个令人尴尬的简单点注释方案，以收集弱监督，例如分割。除了边界框外，我们还收集了在每个边界框内均匀采样的一组点的二进制标签。我们表明，为完整的掩模监督开发的现有实例细分模型可以通过我们的方案收集基于点的监督而无缝培训。值得注意的是，接受了可可，Pascal VOC，CityScapes和LVI的面具R-CNN，每个物体只有10个带注释的随机点可实现94％ - 占其完全监督的性能的98％，为弱化的实例细分定下了强大的基线。新点注释方案的速度比注释完整的对象掩码快5倍，使高质量实例分割在实践中更容易访问。受基于点的注释形式的启发，我们提出了对Pointrend实例分割模块的修改。对于每个对象，称为隐式pointrend的新体系结构生成一个函数的参数，该函数可以使最终的点级掩码预测。隐式Pointrend更加简单，并使用单点级掩蔽丢失。我们的实验表明，新模块更适合基于点的监督。

We present a simple, fully-convolutional model for realtime instance segmentation that achieves 29.8 mAP on MS COCO at 33.5 fps evaluated on a single Titan Xp, which is significantly faster than any previous competitive approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. Finally, we also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty.

Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But recent deep learning object detectors have avoided pyramid representations, in part because they are compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A topdown architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using FPN in a basic Faster R-CNN system, our method achieves state-of-the-art singlemodel results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 6 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.

现有的实例分割方法已经达到了令人印象深刻的表现，但仍遭受了共同的困境：一个实例推断出冗余表示（例如，多个框，网格和锚点），这导致了多个重复的预测。因此，主流方法通常依赖于手工设计的非最大抑制（NMS）后处理步骤来选择最佳预测结果，这会阻碍端到端训练。为了解决此问题，我们建议一个称为Uniinst的无盒和无端机实例分割框架，该框架仅对每个实例产生一个唯一的表示。具体而言，我们设计了一种实例意识到的一对一分配方案，即仅产生一个表示（Oyor），该方案根据预测和地面真相之间的匹配质量，动态地为每个实例动态分配一个独特的表示。然后，一种新颖的预测重新排列策略被优雅地集成到框架中，以解决分类评分和掩盖质量之间的错位，从而使学习的表示形式更具歧视性。借助这些技术，我们的Uniinst，第一个基于FCN的盒子和无NMS实例分段框架，实现竞争性能，例如，使用Resnet-50-FPN和40.2 mask AP使用Resnet-101-FPN，使用Resnet-50-FPN和40.2 mask AP，使用Resnet-101-FPN，对抗AP可可测试-DEV的主流方法。此外，提出的实例感知方法对于遮挡场景是可靠的，在重锁定的ochuman基准上，通过杰出的掩码AP优于公共基线。我们的代码将在出版后提供。

在重建掩码的实例分段网络的设计中，分段通常是其文字定义 - 分配每个像素标签。这导致了将问题视为匹配一个问题，其中一个目标是最小化重建和地面真相像素之间的损耗。重新思考重建网络作为发电机，我们定义了预测掩模作为GAN游戏框架的问题：分割网络生成掩码，鉴别器网络决定掩码的质量。为了演示这个游戏，我们对掩模R-CNN的普通分段框架显示了有效修改。我们发现，在特征空间中播放游戏比导致鉴别器和发电机之间的稳定训练的像素空间更有效，应该通过预测对象的上下文区域来替换预测对象坐标，并且整体对抗性损失有助于性能和消除每个不同数据域的任何自定义设置都需要。我们在各个域中测试我们的框架并报告手机回收，自动驾驶，大规模对象检测和医用腺体。我们观察到一般的GANS产生掩模，该掩模占克里克里德界，杂乱，小物体和细节，处于规则形状或异质和聚结形状的领域。我们的再现结果的代码可公开提供。

本文提出了一种用于对象和场景的高质量图像分割的新方法。灵感来自于形态学图像处理技术中的扩张和侵蚀操作，像素级图像分割问题被视为挤压对象边界。从这个角度来看，提出了一种新颖且有效的\ textBF {边界挤压}模块。该模块用于从内侧和外侧方向挤压对象边界，这有助于精确掩模表示。提出了双向基于流的翘曲过程来产生这种挤压特征表示，并且设计了两个特定的损耗信号以监控挤压过程。边界挤压模块可以通过构建一些现有方法构建作为即插即用模块，可以轻松应用于实例和语义分段任务。此外，所提出的模块是重量的，因此具有实际使用的潜力。实验结果表明，我们简单但有效的设计可以在几个不同的数据集中产生高质量的结果。此外，边界上的其他几个指标用于证明我们对以前的工作中的方法的有效性。我们的方法对实例和语义分割的具有利于Coco和CityCapes数据集来产生重大改进，并且在相同的设置下以前的最先进的速度优于先前的最先进的速度。代码和模型将在\ url {https:/github.com/lxtgh/bsseg}发布。

我们提出了一个小说嵌入字段\ emph {pref}作为促进神经信号建模和重建任务的紧凑表示。基于纯的多层感知器（MLP）神经技术偏向低频信号，并依赖于深层或傅立叶编码以避免丢失细节。取而代之的是，基于傅立叶嵌入空间的相拟合公式，PREF采用了紧凑且物理上解释的编码场。我们进行全面的实验，以证明PERF比最新的空间嵌入技术的优势。然后，我们使用近似的逆傅里叶变换方案以及新型的parseval正常器来开发高效的频率学习框架。广泛的实验表明，我们的高效和紧凑的基于频率的神经信号处理技术与2D图像完成，3D SDF表面回归和5D辐射场现场重建相同，甚至比最新的。

分割高度重叠的图像对象是具有挑战性的，因为图像上的真实对象轮廓和遮挡边界之间通常没有区别。与先前的实例分割方法不同，我们将图像形成模拟为两个重叠层的组成，并提出了双层卷积网络（BCNET），其中顶层检测到遮挡对象（遮挡器），而底层则渗透到部分闭塞实例（胶囊）。遮挡关系与双层结构的显式建模自然地将遮挡和遮挡实例的边界解散，并在掩模回归过程中考虑了它们之间的相互作用。我们使用两种流行的卷积网络设计（即完全卷积网络（FCN）和图形卷积网络（GCN））研究了双层结构的功效。此外，我们通过将图像中的实例表示为单独的可学习封闭器和封闭者查询，从而使用视觉变压器（VIT）制定双层解耦。使用一个/两个阶段和基于查询的对象探测器具有各种骨架和网络层选择验证双层解耦合的概括能力，如图像实例分段基准（可可，亲戚，可可）和视频所示实例分割基准（YTVIS，OVIS，BDD100K MOTS），特别是对于重闭塞病例。代码和数据可在https://github.com/lkeab/bcnet上找到。

We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions in lowdimensional problem domains. These results shed light on recent advances in computer vision and graphics that achieve state-of-the-art results by using MLPs to represent complex 3D objects and scenes. Using tools from the neural tangent kernel (NTK) literature, we show that a standard MLP fails to learn high frequencies both in theory and in practice. To overcome this spectral bias, we use a Fourier feature mapping to transform the effective NTK into a stationary kernel with a tunable bandwidth. We suggest an approach for selecting problem-specific Fourier features that greatly improves the performance of MLPs for low-dimensional regression tasks relevant to the computer vision and graphics communities.

大多数最先进的实例级人类解析模型都采用了两阶段的基于锚的探测器，因此无法避免启发式锚盒设计和像素级别缺乏分析。为了解决这两个问题，我们设计了一个实例级人类解析网络，该网络在像素级别上无锚固且可解决。它由两个简单的子网络组成：一个用于边界框预测的无锚检测头和一个用于人体分割的边缘引导解析头。无锚探测器的头继承了像素样的优点，并有效地避免了对象检测应用中证明的超参数的敏感性。通过引入部分感知的边界线索，边缘引导的解析头能够将相邻的人类部分与彼此区分开，最多可在一个人类实例中，甚至重叠的实例。同时，利用了精炼的头部整合盒子级别的分数和部分分析质量，以提高解析结果的质量。在两个多个人类解析数据集（即CIHP和LV-MHP-V2.0）和一个视频实例级人类解析数据集（即VIP）上进行实验，表明我们的方法实现了超过全球级别和实例级别的性能最新的一阶段自上而下的替代方案。