我们提出了一个分散的视图跨度识别框架，该框架在不需要参考3D地图的情况下自由移动的摄像机运行。每个摄像机都独立提取，汇总为层次结构，并随着时间的推移共享特征点描述符。通过视图匹配和几何验证来验证视图重叠，以丢弃错误匹配的视图。提出的框架是通用的，可以与不同的描述符一起使用。我们对公共可用序列以及使用手持相机收集的新序列进行实验。我们表明，与NetVlad，Rootsift和Superglue相比，在提议的框架内带有二进制单词的快速和旋转简短（ORB）特征，可提高精度和更高或类似的精度。

This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

在本文中，我们建议超越建立的基于视觉的本地化方法，该方法依赖于查询图像和3D点云之间的视觉描述符匹配。尽管通过视觉描述符匹配关键点使本地化高度准确，但它具有重大的存储需求，提出了隐私问题，并需要长期对描述符进行更新。为了优雅地应对大规模定位的实用挑战，我们提出了Gomatch，这是基于视觉的匹配的替代方法，仅依靠几何信息来匹配图像键点与地图的匹配，这是轴承矢量集。我们的新型轴承矢量表示3D点，可显着缓解基于几何的匹配中的跨模式挑战，这阻止了先前的工作在现实环境中解决本地化。凭借额外的仔细建筑设计，Gomatch在先前的基于几何的匹配工作中改善了（1067m，95.7升）和（1.43m，34.7摄氏度），平均中位数姿势错误，同时需要7个尺寸，同时需要7片。与最佳基于视觉的匹配方法相比，几乎1.5/1.7％的存储容量。这证实了其对现实世界本地化的潜力和可行性，并为不需要存储视觉描述符的城市规模的视觉定位方法打开了未来努力的大门。

近年来，机器人社区已经广泛检查了关于同时定位和映射应用范围内的地点识别任务的方法。这篇文章提出了一种基于外观的循环闭合检测管道，命名为“fild ++”（快速和增量环闭合检测） .First，系统由连续图像馈送，并且通过通过单个卷积神经网络通过两次，通过单个卷积神经网络来提取全局和局部深度特征。灵活，分级导航的小世界图逐步构建表示机器人遍历路径的可视数据库基于计算的全局特征。最后，每个时间步骤抓取查询映像，被设置为在遍历的路线上检索类似的位置。遵循的图像到图像配对，它利用本地特征来评估空间信息。因此，在拟议的文章中，我们向全球和本地特征提取提出了一个网络与我们之前的一个网络工作（FILD），而在生成的深度本地特征上采用了彻底搜索验证过程，避免利用哈希代码。关于11个公共数据集的详尽实验表现出系统的高性能（实现其中八个的最高召回得分）和低执行时间（在新学院平均22.05毫秒，这是与其他国家相比包含52480图像的最大版本） - 最艺术方法。

We introduce an approach for recovering the 6D pose of multiple known objects in a scene captured by a set of input images with unknown camera viewpoints. First, we present a single-view single-object 6D pose estimation method, which we use to generate 6D object pose hypotheses. Second, we develop a robust method for matching individual 6D object pose hypotheses across different input images in order to jointly estimate camera viewpoints and 6D poses of all objects in a single consistent scene. Our approach explicitly handles object symmetries, does not require depth measurements, is robust to missing or incorrect object hypotheses, and automatically recovers the number of objects in the scene. Third, we develop a method for global scene refinement given multiple object hypotheses and their correspondences across views. This is achieved by solving an object-level bundle adjustment problem that refines the poses of cameras and objects to minimize the reprojection error in all views. We demonstrate that the proposed method, dubbed Cosy-Pose, outperforms current state-of-the-art results for single-view and multi-view 6D object pose estimation by a large margin on two challenging benchmarks: the YCB-Video and T-LESS datasets. Code and pre-trained models are available on the project webpage. 5

地理定位的概念是指确定地球上的某些“实体”的位置的过程，通常使用全球定位系统（GPS）坐标。感兴趣的实体可以是图像，图像序列，视频，卫星图像，甚至图像中可见的物体。由于GPS标记媒体的大规模数据集由于智能手机和互联网而迅速变得可用，而深入学习已经上升以提高机器学习模型的性能能力，因此由于其显着影响而出现了视觉和对象地理定位的领域广泛的应用，如增强现实，机器人，自驾驶车辆，道路维护和3D重建。本文提供了对涉及图像的地理定位的全面调查，其涉及从捕获图像（图像地理定位）或图像内的地理定位对象（对象地理定位）的地理定位的综合调查。我们将提供深入的研究，包括流行算法的摘要，对所提出的数据集的描述以及性能结果的分析来说明每个字段的当前状态。

Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformation synchronization algorithm. Our key insight is that self-supervised multiview registration allows us to obtain correspondences over longer time frames; increasing both the diversity and difficulty of sampled pairs. We evaluate our approach on indoor scenes for correspondence estimation and RGB-D pointcloud registration and find that we perform on-par with supervised approaches.

This paper introduces SuperGlue, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points. Assignments are estimated by solving a differentiable optimal transport problem, whose costs are predicted by a graph neural network. We introduce a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly. Compared to traditional, hand-designed heuristics, our technique learns priors over geometric transformations and regularities of the 3D world through end-to-end training from image pairs. SuperGlue outperforms other learned approaches and achieves state-of-the-art results on the task of pose estimation in challenging real-world indoor and outdoor environments. The proposed method performs matching in real-time on a modern GPU and can be readily integrated into modern SfM or SLAM systems. The code and trained weights are publicly available at github.com/magicleap/SuperGluePretrainedNetwork.

尽管提取了通过手工制作和基于学习的描述符实现的本地特征的进步，但它们仍然受到不符合非刚性转换的不变性的限制。在本文中，我们提出了一种计算来自静止图像的特征的新方法，该特征对于非刚性变形稳健，以避免匹配可变形表面和物体的问题。我们的变形感知当地描述符，命名优惠，利用极性采样和空间变压器翘曲，以提供旋转，尺度和图像变形的不变性。我们通过将等距非刚性变形应用于模拟环境中的对象作为指导来提供高度辨别的本地特征来培训模型架构端到端。该实验表明，我们的方法优于静止图像中的实际和现实合成可变形对象的不同数据集中的最先进的手工制作，基于学习的图像和RGB-D描述符。描述符的源代码和培训模型在https://www.verlab.dcc.ufmg.br/descriptors/neUrips2021上公开可用。

现在，基于视觉的本地化方法为来自机器人技术到辅助技术的无数用例提供了新出现的导航管道。与基于传感器的解决方案相比，基于视觉的定位不需要预安装的传感器基础架构，这是昂贵，耗时和/或通常不可行的。本文中，我们为特定用例提出了一个基于视觉的本地化管道：针对失明和低视力的最终用户的导航支持。给定最终用户在移动应用程序上拍摄的查询图像，该管道利用视觉位置识别（VPR）算法在目标空间的参考图像数据库中找到相似的图像。这些相似图像的地理位置用于采用加权平均方法来估计最终用户的位置和透视N点（PNP）算法的下游任务中，以估计最终用户的方向。此外，该系统实现了Dijkstra的算法，以根据包括Trip Origin和目的地的可通航地图计算最短路径。用于本地化和导航的层压映射是使用定制的图形用户界面构建的，该图形用户界面投影了3D重建的稀疏映射，从一系列图像构建到相应的先验2D楼平面图。用于地图构造的顺序图像可以在预映射步骤中收集，也可以通过公共数据库/公民科学清除。端到端系统可以使用带有自定义移动应用程序的相机安装在任何可互联网的设备上。出于评估目的，在复杂的医院环境中测试了映射和定位。评估结果表明，我们的系统可以以少于1米的平均误差来实现本地化，而无需了解摄像机的固有参数，例如焦距。

我们介绍了一种简单而有效的方法，可以使用本地3D深度描述符（L3DS）同时定位和映射解决循环闭合检测。 L3DS正在采用深度学习算法从数据从数据中学到的点云提取的斑块的紧凑型表示。通过在通过其估计的相对姿势向循环候选点云登记之后计算对应于相互最近邻接描述符的点之间的度量误差，提出了一种用于循环检测的新颖重叠度量。这种新方法使我们能够在小重叠的情况下精确地检测环并估计六个自由度。我们将基于L3D的循环闭合方法与最近的LIDAR数据的方法进行比较，实现最先进的环路闭合检测精度。此外，我们嵌入了我们在最近的基于边缘的SLAM系统中的循环闭合方法，并对现实世界RGBD-TUM和合成ICL数据集进行了评估。与其原始环路闭合策略相比，我们的方法能够实现更好的本地化准确性。

小天体的任务在很大程度上依赖于光学特征跟踪，以表征和相对导航。尽管深度学习导致了功能检测和描述方面的巨大进步，但由于大规模，带注释的数据集的可用性有限，因此培训和验证了空间应用程序的数据驱动模型具有挑战性。本文介绍了Astrovision，这是一个大规模数据集，由115,970个密集注释的，真实的图像组成，这些图像是过去和正在进行的任务中捕获的16个不同物体的真实图像。我们利用Astrovision开发一组标准化基准，并对手工和数据驱动的功能检测和描述方法进行详尽的评估。接下来，我们采用Astrovision对最先进的，深刻的功能检测和描述网络进行端到端培训，并在多个基准测试中表现出改善的性能。将公开使用完整的基准管道和数据集，以促进用于空间应用程序的计算机视觉算法的发展。

本文介绍了一种用于水下车辆机械手系统（UVMS）的新型视野映射方法，具有特定强调自然海底环境中的鲁棒映射。水下场景映射的先前方法通常会离线处理数据，而实时运行的现有水下SLAM方法通常会集中在本地化上而不是映射。我们的方法使用GPU加速SIFT功能在图形优化框架中构建一个特征映射。地图刻度由车辆安装的立体声相机的特征约束，我们通过将机械手系统的动态定位能力从手腕安装的Fisheye摄像机融合到地图中，以将其延伸到车辆安装摄像机的有限视点之外。我们的混合SLAM方法是在Costa rican Continental Shelf级别的自然深海环境中采用UVMS收集的挑战性图像序列，我们还在浅礁调查数据集中评估立体声的立体声。这些数据集的结果证明了我们的系统的高准确性，适合于在不同的自然海底环境中运营。

由于其对环境变化的鲁棒性，视觉猛感的间接方法是受欢迎的。 ORB-SLAM2 \ CITE {ORBSLM2}是该域中的基准方法，但是，除非选择帧作为关键帧，否则它会消耗从未被重用的描述符。轻量级和高效，因为它跟踪相邻帧之间的关键点而不计算描述符。为此，基于稀疏光流提出了一种两个级粗到微小描述符独立的Keypoint匹配方法。在第一阶段，我们通过简单但有效的运动模型预测初始关键点对应，然后通过基于金字塔的稀疏光流跟踪鲁棒地建立了对应关系。在第二阶段，我们利用运动平滑度和末端几何形状的约束来改进对应关系。特别是，我们的方法仅计算关键帧的描述符。我们在\ texit {tum}和\ texit {icl-nuim} RGB-D数据集上测试Fastorb-Slam，并将其准确性和效率与九种现有的RGB-D SLAM方法进行比较。定性和定量结果表明，我们的方法实现了最先进的准确性，并且大约是ORB-SLAM2的两倍。

Incremental Structure-from-Motion is a prevalent strategy for 3D reconstruction from unordered image collections. While incremental reconstruction systems have tremendously advanced in all regards, robustness, accuracy, completeness, and scalability remain the key problems towards building a truly general-purpose pipeline. We propose a new SfM technique that improves upon the state of the art to make a further step towards this ultimate goal. The full reconstruction pipeline is released to the public as an open-source implementation.

This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models.The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real time, in small and large, indoor and outdoor environments, and is two to ten times more accurate than previous approaches.The second main novelty is a multiple map system that relies on a new place recognition method with improved recall. Thanks to it, ORB-SLAM3 is able to survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting mapped areas. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information. This allows to include in bundle adjustment co-visible keyframes, that provide high parallax observations boosting accuracy, even if they are widely separated in time or if they come from a previous mapping session.Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.5 cm in the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting representative of AR/VR scenarios. For the benefit of the community we make public the source code.

跨场型模型适应对于在实际场景中的摄像机重新定位至关重要。通常，最好将预学的模型快速适应新颖的场景，并尽可能少地训练样本。但是，由于图像特征提取和场景坐标回归的纠缠，现有的最新方法几乎不能支持如此少的场景适应。为了解决此问题，我们使用解耦的解决方案接近摄像机重新定位，在该解决方案中，分别执行特征提取，坐标回归和姿势估计。我们的关键见解是，应通过删除坐标系的分心因子来学习用于坐标回归的功能编码器，从而从多个场景中学到了特征编码器，以获得一般特征表示和更重要的，不敏感的功能。具有此功能先验，并与坐标回归器结合使用，与现有集成解决方案相比，新场景中几乎没有射击的观测比3D世界更容易。实验表明，与最先进的方法相比，我们的方法的优越性，在具有不同的视觉外观和观点分布的几个场景上产生了更高的精度。

We propose a 6D RGB-D odometry approach that finds the relative camera pose between consecutive RGB-D frames by keypoint extraction and feature matching both on the RGB and depth image planes. Furthermore, we feed the estimated pose to the highly accurate KinectFusion algorithm, which uses a fast ICP (Iterative Closest Point) to fine-tune the frame-to-frame relative pose and fuse the depth data into a global implicit surface. We evaluate our method on a publicly available RGB-D SLAM benchmark dataset by Sturm et al. The experimental results show that our proposed reconstruction method solely based on visual odometry and KinectFusion outperforms the state-of-the-art RGB-D SLAM system accuracy. Moreover, our algorithm outputs a ready-to-use polygon mesh (highly suitable for creating 3D virtual worlds) without any postprocessing steps.

我们调查来自两个或更多重叠的网络摄像头流的3D场景重建的可能性。大量，增长，网络摄像头数目观察兴趣的地方，并可公开访问。自然出现的问题：我们可以使用此免费数据源进行3D计算机愿景吗？事实证明，从网络摄像头流中重建场景结构的任务与标准结构 - 从 - 动作（SFM）非常不同，传统的SFM管道失败。在网络摄像头设置中，在大多数情况下，相同场景的观点很少，只有两个。这些观点通常具有大的基线和/或比例差异，它们的重叠相当有限，除了未知的内部和外部校准之外，它们的时间同步也未知。另一方面，它们在长期跨越时不断录制相当大的视野，因此他们定期观察通过场景的动态对象。我们展示了如何利用最近的计算机愿景领域的进步，以适应SFM重建对此特定场景并重建未知的相机姿势，3D场景结构和动态对象的3D轨迹。

Visual localization is the task of estimating camera pose in a known scene, which is an essential problem in robotics and computer vision. However, long-term visual localization is still a challenge due to the environmental appearance changes caused by lighting and seasons. While techniques exist to address appearance changes using neural networks, these methods typically require ground-truth pose information to generate accurate image correspondences or act as a supervisory signal during training. In this paper, we present a novel self-supervised feature learning framework for metric visual localization. We use a sequence-based image matching algorithm across different sequences of images (i.e., experiences) to generate image correspondences without ground-truth labels. We can then sample image pairs to train a deep neural network that learns sparse features with associated descriptors and scores without ground-truth pose supervision. The learned features can be used together with a classical pose estimator for visual stereo localization. We validate the learned features by integrating with an existing Visual Teach & Repeat pipeline to perform closed-loop localization experiments under different lighting conditions for a total of 22.4 km.