我们提出切碎，这是一种3D形状区域分解的方法。 Shred将3D点云作为输入，并使用学习的本地操作来产生近似细粒零件实例的分割。我们将切碎的分解操作赋予了三个分解操作：分裂区域，固定区域之间的边界，并将区域合并在一起。模块经过独立和本地培训，使切碎可以为在培训过程中未见的类别生成高质量的细分。我们通过Partnet的细粒细分进行训练和评估切碎；使用其合并 - 阈值超参数，我们表明，在任何所需的分解粒度下，切碎的分割可以更好地尊重与基线方法相比，更好地尊重地面真相的注释。最后，我们证明切碎对于下游应用非常有用，在零弹药细粒的零件实例分割上的所有基准都超过了所有基准，并且当与学习标记形状区域的方法结合使用时，几乎没有发射细粒的语义分割。

我们提出了神经引导的形状解析器（NGSP），一种方法，该方法学习如何将细粒度语义标签分配给3D形状的区域。 NGSP通过MAP推断解决了这个问题，在输入形状上建模了标签分配的后验概率，其具有学习的似然函数。为了使这次搜索易于进行，NGSP采用神经指南网络，了解近似后部。 NGSP通过使用引导网络的第一次采样提案找到高概率标签分配，然后在完全可能性下评估每个提案。我们评估NGSP从Partnet的制造3D形状的细粒度语义分割任务，其中形状被分解成对应于零件实例过分分割的区域。我们发现NGSP通过比较方法提供显着的性能改进，（i）使用区域对分组每点预测，（ii）使用区域作为自我监督信号或（iii）将标签分配给替代配方下的区域。此外，我们表明，即使具有有限的标记数据或作为形状区域经历人为腐败，NGSP即使具有有限的人为腐败，也会保持强劲的性能。最后，我们证明了NGSP可以直接应用于在线存储库中的CAD形状，并验证其效力与感知研究。

We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. Our dataset consists of 573,585 part instances over 26,671 3D models covering 24 object categories. This dataset enables and serves as a catalyst for many tasks such as shape analysis, dynamic 3D scene modeling and simulation, affordance analysis, and others. Using our dataset, we establish three benchmarking tasks for evaluating 3D part recognition: fine-grained semantic segmentation, hierarchical semantic segmentation, and instance segmentation. We benchmark four state-ofthe-art 3D deep learning algorithms for fine-grained semantic segmentation and three baseline methods for hierarchical semantic segmentation. We also propose a novel method for part instance segmentation and demonstrate its superior performance over existing methods.

我们呈现FURTIT，这是一种简单的3D形状分割网络的高效学习方法。FURTIT基于自我监督的任务，可以将3D形状的表面分解成几何基元。可以很容易地应用于用于3D形状分割的现有网络架构，并提高了几张拍摄设置中的性能，因为我们在广泛使用的ShapEnet和Partnet基准中展示。FISHIT在这种环境中优于现有的现有技术，表明对基元的分解是在学习对语义部分预测的陈述之前的有用。我们提出了许多实验，改变了几何基元和下游任务的选择，以证明该方法的有效性。

Generalizable 3D part segmentation is important but challenging in vision and robotics. Training deep models via conventional supervised methods requires large-scale 3D datasets with fine-grained part annotations, which are costly to collect. This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP, which achieves superior performance on open-vocabulary 2D detection. We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm. We also utilize multi-view 3D priors and few-shot prompt tuning to boost performance significantly. Extensive evaluation on PartNet and PartNet-Mobility datasets shows that our method enables excellent zero-shot 3D part segmentation. Our few-shot version not only outperforms existing few-shot approaches by a large margin but also achieves highly competitive results compared to the fully supervised counterpart. Furthermore, we demonstrate that our method can be directly applied to iPhone-scanned point clouds without significant domain gaps.

大多数现有的点云实例和语义分割方法在很大程度上依赖于强大的监督信号，这需要场景中每个点的点级标签。但是，这种强大的监督遭受了巨大的注释成本，引起了研究有效注释的需求。在本文中，我们发现实例的位置对实例和语义3D场景细分都很重要。通过充分利用位置，我们设计了一种弱监督的点云分割算法，该算法仅需要单击每个实例以指示其注释的位置。通过进行预处理过度分割，我们将这些位置注释扩展到seg级标签中。我们通过将未标记的片段分组分组到相关的附近标签段中，进一步设计一个段分组网络（SEGGROUP），以在SEG级标签下生成点级伪标签，以便现有的点级监督的分段模型可以直接消耗这些PSEUDO标签为了训练。实验结果表明，我们的SEG级监督方法（SEGGROUP）通过完全注释的点级监督方法获得了可比的结果。此外，在固定注释预算的情况下，它的表现优于最近弱监督的方法。

我们建议在2D域中利用自我监督的技术来实现细粒度的3D形状分割任务。这是受到观察的启发：基于视图的表面表示比基于点云或体素占用率的3D对应物更有效地建模高分辨率表面细节和纹理。具体而言，给定3D形状，我们将其从多个视图中渲染，并在对比度学习框架内建立密集的对应学习任务。结果，与仅在2D或3D中使用自学的替代方案相比，学到的2D表示是视图不变和几何一致的，在对有限的标记形状进行培训时，可以更好地概括概括。对纹理（渲染peple）和未纹理（partnet）3D数据集的实验表明，我们的方法在细粒部分分割中优于最先进的替代方案。当仅一组稀疏的视图可供训练或形状纹理时，对基准的改进就会更大，这表明MVDecor受益于2D处理和3D几何推理。

我们介绍了PartGolot，神经框架和相关架构，用于学习3D形几何的语义部分分割，仅基于部分参照语言。我们利用形状的语言描述可以提供形状的部分的前瞻性 - 因为自然语言已经发展，以反映对物体的组成结构的人类感知，对其认可和使用至关重要。对于培训，我们使用Shapeglot工作中收集的成对几何/语言数据来为其参考游戏，其中扬声器创建话语以区分从两个牵引器的目标形状，并且听众必须基于这种话语找到目标。我们的网络旨在解决此目标辨别问题，仔细介绍基于变压器的注意模块，以便输出注意力可以精确地突出显示语言中描述的语义部件或零件。此外，网络在3D几何形状本身上没有任何直接监督。令人惊讶的是，我们进一步证明学习部分信息是概括的，可以在训练期间形状看不见。我们的方法打开了单独从语言学习3D形状的可能性，而无需大规模部分几何注释，从而促进注释采集。

基于学习的3D形状分割通常被配制为语义标记问题，假设训练形状的所有部分都用给定的一组标签注释。然而，这种假设对于学习细粒度的细分来说是不切实际的。虽然大多数现成的CAD模型是由施工组成的细粒度，但它们通常会错过语义标签并标记那些细粒度的部分非常乏味。我们接近深群体的问题，其中关键的想法是从带有细粒度分割的形状数据集中学习部分前导者，但没有部分标签。给定点采样3D形状，我们通过相似矩阵模拟点的聚类前沿，通过最小化新的低级损失来实现部分分割。为了处理高度密集的采样点集，我们采用了分裂和征服策略。我们将大点分区设置为多个块。每个块使用以类别 - 不可知方式培训的基于深度基于集群的基于网络的部分进行分段。然后，我们会培训图形卷积网络以合并所有块的段以形成最终的分段结果。我们的方法是用细粒细分的具有挑战性的基准进行评估，显示出最先进的性能。

从3D点云中识别3D零件实例对于3D结构和场景理解至关重要。几种基于学习的方法使用语义细分和实例中心预测作为培训任务，并且无法进一步利用形状语义和部分实例之间的固有关系。在本文中，我们提出了一种用于3D份实例分割的新方法。我们的方法将语义分割利用为融合非本地实例特征（例如中心预测），并以多种和跨层次的方式进一步增强了融合方案。我们还提出了一个语义区域中心预测任务，以训练和利用预测结果来改善实例点的聚类。我们的方法优于现有方法，在Partnet基准测试方面有大幅度的改进。我们还证明，我们的功能融合方案可以应用于其他现有方法，以提高其在室内场景实例细分任务中的性能。

We introduce Similarity Group Proposal Network (SGPN), a simple and intuitive deep learning framework for 3D object instance segmentation on point clouds. SGPN uses a single network to predict point grouping proposals and a corresponding semantic class for each proposal, from which we can directly extract instance segmentation results. Important to the effectiveness of SGPN is its novel representation of 3D instance segmentation results in the form of a similarity matrix that indicates the similarity between each pair of points in embedded feature space, thus producing an accurate grouping proposal for each point. Experimental results on various 3D scenes show the effectiveness of our method on 3D instance segmentation, and we also evaluate the capability of SGPN to improve 3D object detection and semantic segmentation results. We also demonstrate its flexibility by seamlessly incorporating 2D CNN features into the framework to boost performance.

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.

我们通过执行基于接触的推理，提供了一种形状部分插槽机，一种用于组装来自现有部件的新型3D形状。我们的方法表示每个形状作为“槽”的图形，其中每个槽是两个形状部件之间的接触区域。基于此表示，我们设计了一种基于图形 - 神经网络的模型，用于生成新的插槽图和检索兼容部分，以及基于梯度 - 下降的优化方案，用于将检索到的部分组装成尊重所生成的完整形状插槽图。这种方法不需要任何语义部分标签;有趣的是，它还不需要完整的部分几何形状 - 推理零件连接的区域足以产生新颖的，高质量的3D形状。我们展示了我们的方法在质量，多样性和结构复杂性方面产生了优于现有的逐个拟合方法的形状。

Segmenting humans in 3D indoor scenes has become increasingly important with the rise of human-centered robotics and AR/VR applications. In this direction, we explore the tasks of 3D human semantic-, instance- and multi-human body-part segmentation. Few works have attempted to directly segment humans in point clouds (or depth maps), which is largely due to the lack of training data on humans interacting with 3D scenes. We address this challenge and propose a framework for synthesizing virtual humans in realistic 3D scenes. Synthetic point cloud data is attractive since the domain gap between real and synthetic depth is small compared to images. Our analysis of different training schemes using a combination of synthetic and realistic data shows that synthetic data for pre-training improves performance in a wide variety of segmentation tasks and models. We further propose the first end-to-end model for 3D multi-human body-part segmentation, called Human3D, that performs all the above segmentation tasks in a unified manner. Remarkably, Human3D even outperforms previous task-specific state-of-the-art methods. Finally, we manually annotate humans in test scenes from EgoBody to compare the proposed training schemes and segmentation models.

我们提出了一个Point2cyl，一个监督网络将原始3D点云变换到一组挤出缸。从原始几何到CAD模型的逆向工程是能够在形状编辑软件中操纵3D数据的重要任务，从而在许多下游应用中扩展其使用。特别地，具有挤出圆柱序列的CAD模型的形式 - 2D草图加上挤出轴和范围 - 以及它们的布尔组合不仅广泛应用于CAD社区/软件，而且相比具有很大的形状表现性具有有限类型的基元（例如，平面，球形和汽缸）。在这项工作中，我们介绍了一种神经网络，通过首先学习底层几何代理来解决挤出汽缸分解问题的挤出圆柱分解问题。精确地，我们的方法首先预测每点分割，基础/桶标签和法线，然后估计可分离和闭合形式配方中的底层挤出参数。我们的实验表明，我们的方法展示了两个最近CAD数据集，融合画廊和Deepcad上的最佳性能，我们进一步展示了逆向工程和编辑的方法。

零件代表不同对象的几何和语义相似性的基本单位。我们争辩说，部分知识应与观察到的对象课程中有款组合。对此，我们将3D组成零射击学习作为从看作识的零件泛化的问题，从而看成了语义分割。我们通过将任务与所提出的组成部分数据集进行基准测试，提供结构化研究。该数据集是通过处理原始PartNet来创建的，以最大化不同对象的部分重叠。现有点云部分段方法未能在此设置中概括到未遵守的对象类。作为解决方案，我们提出了分解共识，其将零件分割网络与部分评分网络相结合。我们方法的关键直觉是某些部件的分割掩码应该具有与其部分分数分开的零件分数的共识。在生成最合适的分割掩模之前在每个对象部分中定义的不同部分组合的两个网络原因。我们展示了我们的方法允许组成零射分段和广义零拍分类，并在两个任务中建立最先进的状态。

Current supervised visual detectors, though impressive within their training distribution, often fail to segment out-of-distribution scenes into their constituent entities. Recent test-time adaptation methods use auxiliary self-supervised losses to adapt the network parameters to each test example independently and have shown promising results towards generalization outside the training distribution for the task of image classification. In our work, we find evidence that these losses can be insufficient for instance segmentation tasks, without also considering architectural inductive biases. For image segmentation, recent slot-centric generative models break such dependence on supervision by attempting to segment scenes into entities in a self-supervised manner by reconstructing pixels. Drawing upon these two lines of work, we propose Slot-TTA, a semi-supervised instance segmentation model equipped with a slot-centric inductive bias, that is adapted per scene at test time through gradient descent on reconstruction or novel view synthesis objectives. We show that test-time adaptation in Slot-TTA greatly improves instance segmentation in out-of-distribution scenes. We evaluate Slot-TTA in several 3D and 2D scene instance segmentation benchmarks and show substantial out-of-distribution performance improvements against state-of-the-art supervised feed-forward detectors and self-supervised test-time adaptation methods.

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset. Code, data and trained models are available at https://wentaoyuan.github.io/pcn.

我们提出了一种基于动态卷积的3D点云的实例分割方法。这使其能够在推断时适应变化的功能和对象尺度。这样做避免了一些自下而上的方法的陷阱，包括对超参数调整和启发式后处理管道的依赖，以弥补物体大小的不可避免的可变性，即使在单个场景中也是如此。通过收集具有相同语义类别并为几何质心进行仔细投票的均匀点，网络的表示能力大大提高了。然后通过几个简单的卷积层解码实例，其中参数是在输入上生成的。所提出的方法是无建议的，而是利用适应每个实例的空间和语义特征的卷积过程。建立在瓶颈层上的轻重量变压器使模型可以捕获远程依赖性，并具有有限的计算开销。结果是一种简单，高效且健壮的方法，可以在各种数据集上产生强大的性能：ScannETV2，S3DIS和Partnet。基于体素和点的体系结构的一致改进意味着提出的方法的有效性。代码可在以下网址找到：https：//git.io/dyco3d