人类自然有效地在复杂的场景中找到突出区域。通过这种观察的动机，引入了计算机视觉中的注意力机制，目的是模仿人类视觉系统的这一方面。这种注意机制可以基于输入图像的特征被视为动态权重调整过程。注意机制在许多视觉任务中取得了巨大的成功，包括图像分类，对象检测，语义分割，视频理解，图像生成，3D视觉，多模态任务和自我监督的学习。在本调查中，我们对计算机愿景中的各种关注机制进行了全面的审查，并根据渠道注意，空间关注，暂时关注和分支注意力进行分类。相关的存储库https：//github.com/menghaoguo/awesome-vision-tions致力于收集相关的工作。我们还建议了未来的注意机制研究方向。

空中图像中的物体比典型图像中的刻度和方向具有更大的变化，因此检测更加困难。卷积神经网络使用各种频率和方向特定的内核来识别受不同变换的对象;这些需要许多参数。采样等式网络可以根据对象的转换从输入特征映射调整采样，允许内核在不同的变换下提取对象的特征。这样做需要较少的参数，并且使网络更适合于代表可变形对象，如空中图像中的那些。然而，像可变形卷积网络一样的方法只能在某些情况下提供采样设备，因为用于采样的位置。我们提出了采样的等式自我关注网络，其认为自我关注限制在本地图像补丁中，因为用掩模而不是位置的卷积采样，以及设计变换嵌入模块，以进一步提高等值的采样能力。我们还使用新颖的随机标准化模块来应付由于空中图像数据有限的原因。我们表明，我们的型号（i）提供了比现有方法更好的采样量规范，而无需额外监督，（ii）提供对Imagenet的改进分类，并且（iii）在没有增加的情况下实现最先进的结果计算。

卷积神经网络（CNNS）在2D计算机视觉中取得了很大的突破。然而，它们的不规则结构使得难以在网格上直接利用CNNS的潜力。细分表面提供分层多分辨率结构，其中闭合的2 - 歧管三角网格中的每个面正恰好邻近三个面。本文推出了这两种观察，介绍了具有环形细分序列连接的3D三角形网格的创新和多功能CNN框架。在2D图像中的网格面和像素之间进行类比允许我们呈现网状卷积操作者以聚合附近面的局部特征。通过利用面部街区，这种卷积可以支持标准的2D卷积网络概念，例如，可变内核大小，步幅和扩张。基于多分辨率层次结构，我们利用汇集层，将四个面均匀地合并成一个和上采样方法，该方法将一个面分为四个。因此，许多流行的2D CNN架构可以容易地适应处理3D网格。可以通过自我参数化来回收具有任意连接的网格，以使循环细分序列连接，使子变量是一般的方法。广泛的评估和各种应用展示了SubDIVNet的有效性和效率。

The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation, semantic segmentation and normal estimation tasks.

The goal of this paper is to detect objects by exploiting their interrelationships. Rather than relying on predefined and labeled graph structures, we infer a graph prior from object co-occurrence statistics. The key idea of our paper is to model object relations as a function of initial class predictions and co-occurrence priors to generate a graph representation of an image for improved classification and bounding box regression. We additionally learn the object-relation joint distribution via energy based modeling. Sampling from this distribution generates a refined graph representation of the image which in turn produces improved detection performance. Experiments on the Visual Genome and MS-COCO datasets demonstrate our method is detector agnostic, end-to-end trainable, and especially beneficial for rare object classes. What is more, we establish a consistent improvement over object detectors like DETR and Faster-RCNN, as well as state-of-the-art methods modeling object interrelationships.

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Line segments are ubiquitous in our human-made world and are increasingly used in vision tasks. They are complementary to feature points thanks to their spatial extent and the structural information they provide. Traditional line detectors based on the image gradient are extremely fast and accurate, but lack robustness in noisy images and challenging conditions. Their learned counterparts are more repeatable and can handle challenging images, but at the cost of a lower accuracy and a bias towards wireframe lines. We propose to combine traditional and learned approaches to get the best of both worlds: an accurate and robust line detector that can be trained in the wild without ground truth lines. Our new line segment detector, DeepLSD, processes images with a deep network to generate a line attraction field, before converting it to a surrogate image gradient magnitude and angle, which is then fed to any existing handcrafted line detector. Additionally, we propose a new optimization tool to refine line segments based on the attraction field and vanishing points. This refinement improves the accuracy of current deep detectors by a large margin. We demonstrate the performance of our method on low-level line detection metrics, as well as on several downstream tasks using multiple challenging datasets. The source code and models are available at https://github.com/cvg/DeepLSD.

Recent work in large language modeling (LLMs) has used fine-tuning to align outputs with the preferences of a prototypical user. This work assumes that human preferences are static and homogeneous across individuals, so that aligning to a a single "generic" user will confer more general alignment. Here, we embrace the heterogeneity of human preferences to consider a different challenge: how might a machine help people with diverse views find agreement? We fine-tune a 70 billion parameter LLM to generate statements that maximize the expected approval for a group of people with potentially diverse opinions. Human participants provide written opinions on thousands of questions touching on moral and political issues (e.g., "should we raise taxes on the rich?"), and rate the LLM's generated candidate consensus statements for agreement and quality. A reward model is then trained to predict individual preferences, enabling it to quantify and rank consensus statements in terms of their appeal to the overall group, defined according to different aggregation (social welfare) functions. The model produces consensus statements that are preferred by human users over those from prompted LLMs (>70%) and significantly outperforms a tight fine-tuned baseline that lacks the final ranking step. Further, our best model's consensus statements are preferred over the best human-generated opinions (>65%). We find that when we silently constructed consensus statements from only a subset of group members, those who were excluded were more likely to dissent, revealing the sensitivity of the consensus to individual contributions. These results highlight the potential to use LLMs to help groups of humans align their values with one another.

在原始文本中训练的语言模型（LMS）无法直接访问物理世界。 Gordon和Van Durme（2013）指出，LMS因此可能会遭受报告偏见的困扰：文本很少报告常见事实，而是关注情况的异常方面。如果LMS仅接受文本语料库的培训，并天真地记住当地的同时出现统计数据，那么他们自然会学会对物理世界的偏见。虽然先前的研究反复验证了较小尺度的LM（例如Roberta，GPT-2）放大了报告偏差，但在模型扩展时，这种趋势是否继续。我们从较大语言模型（LLM）（例如Palm和GPT-3）中从颜色的角度研究报告偏见。具体而言，我们查询llms对物体的典型颜色，这是一种简单的感知扎根的物理常识。令人惊讶的是，我们发现LLM在确定对象的典型颜色和更紧密地跟踪人类判断方面的表现明显优于较小的LMS，而不是过于适应文本中存储的表面图案。这表明，仅凭语言的大型语言模型就能克服以局部共发生为特征的某些类型的报告偏差。