The task of reconstructing 3D human motion has wideranging applications. The gold standard Motion capture (MoCap) systems are accurate but inaccessible to the general public due to their cost, hardware and space constraints. In contrast, monocular human mesh recovery (HMR) methods are much more accessible than MoCap as they take single-view videos as inputs. Replacing the multi-view Mo- Cap systems with a monocular HMR method would break the current barriers to collecting accurate 3D motion thus making exciting applications like motion analysis and motiondriven animation accessible to the general public. However, performance of existing HMR methods degrade when the video contains challenging and dynamic motion that is not in existing MoCap datasets used for training. This reduces its appeal as dynamic motion is frequently the target in 3D motion recovery in the aforementioned applications. Our study aims to bridge the gap between monocular HMR and multi-view MoCap systems by leveraging information shared across multiple video instances of the same action. We introduce the Neural Motion (NeMo) field. It is optimized to represent the underlying 3D motions across a set of videos of the same action. Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection. To further validate NeMo using 3D metrics, we collected a small MoCap dataset mimicking actions in Penn Action,and show that NeMo achieves better 3D reconstruction compared to various baselines.
translated by 谷歌翻译
Open World Object Detection (OWOD) is a new and challenging computer vision task that bridges the gap between classic object detection (OD) benchmarks and object detection in the real world. In addition to detecting and classifying seen/labeled objects, OWOD algorithms are expected to detect novel/unknown objects - which can be classified and incrementally learned. In standard OD, object proposals not overlapping with a labeled object are automatically classified as background. Therefore, simply applying OD methods to OWOD fails as unknown objects would be predicted as background. The challenge of detecting unknown objects stems from the lack of supervision in distinguishing unknown objects and background object proposals. Previous OWOD methods have attempted to overcome this issue by generating supervision using pseudo-labeling - however, unknown object detection has remained low. Probabilistic/generative models may provide a solution for this challenge. Herein, we introduce a novel probabilistic framework for objectness estimation, where we alternate between probability distribution estimation and objectness likelihood maximization of known objects in the embedded feature space - ultimately allowing us to estimate the objectness probability of different proposals. The resulting Probabilistic Objectness transformer-based open-world detector, PROB, integrates our framework into traditional object detection models, adapting them for the open-world setting. Comprehensive experiments on OWOD benchmarks show that PROB outperforms all existing OWOD methods in both unknown object detection ($\sim 2\times$ unknown recall) and known object detection ($\sim 10\%$ mAP). Our code will be made available upon publication at https://github.com/orrzohar/PROB.
translated by 谷歌翻译
机器学习(ML)技术在教育方面越来越普遍,从预测学生辍学,到协助大学入学以及促进MOOC的兴起。考虑到这些新颖用途的快速增长,迫切需要调查ML技术如何支持长期以来的教育原则和目标。在这项工作中,我们阐明了这一复杂的景观绘制,以对教育专家的访谈进行定性见解。这些访谈包括对过去十年中著名应用ML会议上发表的ML教育(ML4ED)论文的深入评估。我们的中心研究目标是批判性地研究这些论文的陈述或暗示教育和社会目标如何与他们解决的ML问题保持一致。也就是说,技术问题的提出,目标,方法和解释结果与手头的教育问题保持一致。我们发现,在ML生命周期的两个部分中存在跨学科的差距,并且尤其突出:从教育目标和将预测转换为干预措施的ML问题的提出。我们使用这些见解来提出扩展的ML生命周期,这也可能适用于在其他领域中使用ML。我们的工作加入了越来越多的跨教育和ML研究的荟萃分析研究,以及对ML社会影响的批判性分析。具体而言,它填补了对机器学习的主要技术理解与与学生合作和政策合作的教育研究人员的观点之间的差距。
translated by 谷歌翻译
从单个图像中感知3D人体的能力具有多种应用,从娱乐和机器人技术到神经科学和医疗保健。人类网格恢复中的一个基本挑战是收集训练所需的地面真相3D网格目标,这需要负担重大的运动捕获系统,并且通常仅限于室内实验室。结果,尽管在这些限制性设置中收集的基准数据集上取得了进展,但由于分配变化,模型无法推广到现实世界中的``野外''方案。我们提出了域自适应3D姿势增强(DAPA),这是一种数据增强方法,可增强模型在野外场景中的概括能力。 DAPA通过从综合网格中获得直接监督,并通过使用目标数据集的地面真相2D关键点来结合基于合成数据集的方法的强度。我们定量地表明,使用DAPA的填充有效地改善了基准3DPW和Agora的结果。我们进一步证明了DAPA在一个充满挑战的数据集中,该数据集从现实世界中亲子互动的视频中策划了。
translated by 谷歌翻译
事实证明,知识蒸馏是使用教师模型的预测来改善学生模型的一项有效技术。但是,最近的工作表明,在数据中的亚组中,平均效率的提高并不统一,尤其是在稀有亚组和类别上的准确性通常可能以准确性为代价。为了在可能遵循长尾分配的课程中保持强劲的表现,我们开发了蒸馏技术,这些技术是为了改善学生最差的级别表现而定制的。具体来说,我们为教师和学生介绍了不同组合的强大优化目标,并进一步允许在整体准确性和强大的最差目标之间进行任何权衡训练。我们从经验上表明,与其他基线方法相比,我们强大的蒸馏技术不仅可以实现更好的最差级别性能,而且还可以改善整体性能和最差的级别性能之间的权衡。从理论上讲,我们提供有关在目标培训健壮学生时使一名好老师的见解。
translated by 谷歌翻译
Decentralized bilevel optimization has received increasing attention recently due to its foundational role in many emerging multi-agent learning paradigms (e.g., multi-agent meta-learning and multi-agent reinforcement learning) over peer-to-peer edge networks. However, to work with the limited computation and communication capabilities of edge networks, a major challenge in developing decentralized bilevel optimization techniques is to lower sample and communication complexities. This motivates us to develop a new decentralized bilevel optimization called DIAMOND (decentralized single-timescale stochastic approximation with momentum and gradient-tracking). The contributions of this paper are as follows: i) our DIAMOND algorithm adopts a single-loop structure rather than following the natural double-loop structure of bilevel optimization, which offers low computation and implementation complexity; ii) compared to existing approaches, the DIAMOND algorithm does not require any full gradient evaluations, which further reduces both sample and computational complexities; iii) through a careful integration of momentum information and gradient tracking techniques, we show that the DIAMOND algorithm enjoys $\mathcal{O}(\epsilon^{-3/2})$ in sample and communication complexities for achieving an $\epsilon$-stationary solution, both of which are independent of the dataset sizes and significantly outperform existing works. Extensive experiments also verify our theoretical findings.
translated by 谷歌翻译
In this new computing paradigm, named quantum computing, researchers from all over the world are taking their first steps in designing quantum circuits for image processing, through a difficult process of knowledge transfer. This effort is named Quantum Image Processing, an emerging research field pushed by powerful parallel computing capabilities of quantum computers. This work goes in this direction and proposes the challenging development of a powerful method of image denoising, such as the Total Variation (TV) model, in a quantum environment. The proposed Quantum TV is described and its sub-components are analysed. Despite the natural limitations of the current capabilities of quantum devices, the experimental results show a competitive denoising performance compared to the classical variational TV counterpart.
translated by 谷歌翻译
机器学习(ML)研究通常集中在模型上,而最突出的数据集已用于日常的ML任务,而不考虑这些数据集对基本问题的广度,困难和忠诚。忽略数据集的基本重要性已引起了重大问题,该问题涉及现实世界中的数据级联以及数据集驱动标准的模型质量饱和,并阻碍了研究的增长。为了解决此问题,我们提出Dataperf,这是用于评估ML数据集和数据集工作算法的基准软件包。我们打算启用“数据棘轮”,其中培训集将有助于评估相同问题的测试集,反之亦然。这种反馈驱动的策略将产生一个良性的循环,该循环将加速以数据为中心的AI。MLCommons协会将维护Dataperf。
translated by 谷歌翻译
自动手术活动识别可以实现更智能的手术设备和更有效的工作流程。这种技术在新手术室中的整合有可能改善对患者的护理服务并降低成本。最近的作品在手术活动识别方面取得了有希望的表现。但是,这些模型缺乏普遍性是该技术广泛采用的关键障碍之一。在这项工作中,我们研究了手术室跨手术活动识别模型的普遍性。我们提出了一种新的域适应方法,以在新手术室中提高手术活动识别模型的性能,而我们只有未标记的视频。我们的方法生成了伪标签,用于对其有信心的未标记视频剪辑,并在剪辑的增强版本上训练该模型。我们将方法扩展到半监督域的适应设置,其中还标记了目标域的一小部分。在我们的实验中,我们提出的方法始终优于从两个手术室收集的480多个长手术视频的数据集上的基准。
translated by 谷歌翻译
人形机器人可以在危险情况下取代人类,但大多数此类情况对他们来说同样危险,这意味着他们有很大的损害和下降的机会。我们假设人形机器人主要用于建筑物,这使它们可能靠近墙壁。为了避免跌倒,他们可以像人类那样靠在最接近的墙上,只要他们在几毫秒内找到手放手的地方。本文介绍了一种称为D-Reflex的方法,该方法学习了一个神经网络,该神经网络在墙壁方向,墙壁距离和机器人的姿势下选择此接触位置。然后,全身控制器使用此接触位置来达到稳定的姿势。我们表明,D-Reflex允许模拟的Talos机器人(1.75m,100kg,30自由度)避免了超过75%的可避免跌倒,并且可以在真正的机器人上工作。
translated by 谷歌翻译