Vision-language models (VLMs) that are pre-trained on large-scale image-text pairs have demonstrated impressive transferability on a wide range of visual tasks. Transferring knowledge from such powerful pre-trained VLMs is emerging as a promising direction for building effective video recognition models. However, the current exploration is still limited. In our opinion, the greatest charm of pre-trained vision-language models is to build a bridge between visual and textual domains. In this paper, we present a novel framework called BIKE which utilizes the cross-modal bridge to explore bidirectional knowledge: i) We propose a Video Attribute Association mechanism which leverages the Video-to-Text knowledge to generate textual auxiliary attributes to complement video recognition. ii) We also present a Temporal Concept Spotting mechanism which uses the Text-to-Video expertise to capture temporal saliency in a parameter-free manner to yield enhanced video representation. The extensive studies on popular video datasets (ie, Kinetics-400 & 600, UCF-101, HMDB-51 and ActivityNet) show that our method achieves state-of-the-art performance in most recognition scenarios, eg, general, zero-shot, and few-shot video recognition. To the best of our knowledge, our best model achieves a state-of-the-art accuracy of 88.4% on challenging Kinetics-400 with the released CLIP pre-trained model.

基于图像的3D检测是自主驾驶感知系统的必不可少的组成部分。但是，它仍然受到不满意的表现，这是有限的培训数据的主要原因之一。不幸的是，在3D空间中注释对象是极度时间/资源消耗的，这使得很难任意扩展训练集。在这项工作中，我们专注于半监督的方式，并探索更便宜的替代方案（即伪标记）的可行性，以利用未标记的数据。为此，我们进行了广泛的实验，以研究伪标签是否可以在不同环境下为基线模型提供有效的监督。实验结果不仅证明了基于图像的3D检测的伪标记机制的有效性（例如，在单眼设置下，我们在没有铃铛和哨声的Kitti-3D测试集上实现了20.23 AP，用于中等水平，从6.03 AP），但还显示了几个有趣且值得注意的发现（例如，经过伪标签训练的模型的性能要比基于相同培训数据的地面真相注释训练的表现更好）。我们希望这项工作可以在半监督环境下为基于图像的3D检测社区提供见解。代码，伪标签和预培训模型将公开可用。

时间序列与许多其他机器学习领域一样，从统计学到深度学习进行了过渡。尽管随着模型在许多公开可用的数据集中的更新时，似乎精度一直在提高，但通常只会将比例尺增加几倍，以换取准确性的略有差异。通过该实验，我们指出了不同的思维方式，时间序列，尤其是长期预测，可能与其他领域有所不同。不必使用广泛而复杂的模型来掌握时间序列的所有方面，而是使用纯模型来掌握时间序列的核心规则。有了这个简单但有效的想法，我们创建了Purets，这是一个具有三个纯线性层的网络，在80％的长序列预测任务中实现了最新的，同时几乎是最轻的模型，并且运行速度最快。在此基础上，我们讨论了纯线性层在现象和本质中的潜力。理解核心法律的能力有助于长距离预测的高精度，并且合理的波动可以防止其扭曲多步预测中的曲线，例如主流深度学习模型，该模型总结为纯粹的线性神经网络，避免了范围 - 覆盖。最后，我们建议轻巧长时间时间序列任务的基本设计标准：输入和输出应尝试具有相同的维度，并且结构避免了碎片化和复杂的操作。

知识蒸馏（KD）在将学习表征从大型模型（教师）转移到小型模型（学生）方面表现出非常有希望的能力。但是，随着学生和教师之间的容量差距变得更大，现有的KD方法无法获得更好的结果。我们的工作表明，“先验知识”对KD至关重要，尤其是在应用大型老师时。特别是，我们提出了动态的先验知识（DPK），该知识将教师特征的一部分作为特征蒸馏之前的先验知识。这意味着我们的方法还将教师的功能视为“输入”，而不仅仅是``目标''。此外，我们根据特征差距动态调整训练阶段的先验知识比率，从而引导学生在适当的困难中。为了评估所提出的方法，我们对两个图像分类基准（即CIFAR100和Imagenet）和一个对象检测基准（即MS Coco）进行了广泛的实验。结果表明，在不同的设置下，我们方法在性能方面具有优势。更重要的是，我们的DPK使学生模型的表现与教师模型的表现呈正相关，这意味着我们可以通过应用更大的教师进一步提高学生的准确性。我们的代码将公开用于可重复性。

我们重新审视汤普森采样算法以控制Ouyang等人最近提出的未知线性二次（LQ）系统（Arxiv：1709.04047）。该算法的遗憾是根据封闭环系统的诱导规范的技术假设得出的。在此技术说明中，我们表明，通过在算法中进行较小的修改（特别是确保发作不会太早结束），可以根据光谱来代替诱发规范的技术假设。闭环系统的半径。修改后的算法与$ \ tilde {\ Mathcal {o}}（\ sqrt {t}）$具有相同的贝叶斯遗憾，其中$ t $是Time-Horizon和$ \ tilde {\ tilde {\ Mathcal {o}}（O}}（O}}（O}}（O}}（O}））（\ cdot）$ note法将对数术语隐藏在〜$ t $中。

Most existing text-video retrieval methods focus on cross-modal matching between the visual content of offline videos and textual query sentences. However, in real scenarios, online videos are frequently accompanied by relevant text information such as titles, tags, and even subtitles, which can be utilized to match textual queries. This inspires us to generate associated captions from offline videos to help with existing text-video retrieval methods. To do so, we propose to use the zero-shot video captioner with knowledge of pre-trained web-scale models (e.g., CLIP and GPT-2) to generate captions for offline videos without any training. Given the captions, one question naturally arises: what can auxiliary captions do for text-video retrieval? In this paper, we present a novel framework Cap4Video, which makes use of captions from three aspects: i) Input data: The video and captions can form new video-caption pairs as data augmentation for training. ii) Feature interaction: We perform feature interaction between video and caption to yield enhanced video representations. iii) Output score: The Query-Caption matching branch can be complementary to the original Query-Video matching branch for text-video retrieval. We conduct thorough ablation studies to demonstrate the effectiveness of our method. Without any post-processing, our Cap4Video achieves state-of-the-art performance on MSR-VTT (51.4%), VATEX (66.6%), MSVD (51.8%), and DiDeMo (52.0%).

We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering. Motivated by the fact that informative point cloud features should be able to encode rich geometry and appearance cues and render realistic images, we train a point-cloud encoder within a devised point-based neural renderer by comparing the rendered images with real images on massive RGB-D data. The learned point-cloud encoder can be easily integrated into various downstream tasks, including not only high-level tasks like 3D detection and segmentation, but low-level tasks like 3D reconstruction and image synthesis. Extensive experiments on various tasks demonstrate the superiority of our approach compared to existing pre-training methods.

Automatic image colorization is a particularly challenging problem. Due to the high illness of the problem and multi-modal uncertainty, directly training a deep neural network usually leads to incorrect semantic colors and low color richness. Existing transformer-based methods can deliver better results but highly depend on hand-crafted dataset-level empirical distribution priors. In this work, we propose DDColor, a new end-to-end method with dual decoders, for image colorization. More specifically, we design a multi-scale image decoder and a transformer-based color decoder. The former manages to restore the spatial resolution of the image, while the latter establishes the correlation between semantic representations and color queries via cross-attention. The two decoders incorporate to learn semantic-aware color embedding by leveraging the multi-scale visual features. With the help of these two decoders, our method succeeds in producing semantically consistent and visually plausible colorization results without any additional priors. In addition, a simple but effective colorfulness loss is introduced to further improve the color richness of generated results. Our extensive experiments demonstrate that the proposed DDColor achieves significantly superior performance to existing state-of-the-art works both quantitatively and qualitatively. Codes will be made publicly available.

Masked Modeling (MM) has demonstrated widespread success in various vision challenges, by reconstructing masked visual patches. Yet, applying MM for large-scale 3D scenes remains an open problem due to the data sparsity and scene complexity. The conventional random masking paradigm used in 2D images often causes a high risk of ambiguity when recovering the masked region of 3D scenes. To this end, we propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points, effectively enhancing the pretext masking task for 3D scene understanding. Integrated with a progressive reconstruction manner, our method can concentrate on modeling regional geometry and enjoy less ambiguity for masked reconstruction. Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas. By elegantly combining informative-preserved reconstruction on masked areas and consistency self-distillation from unmasked areas, a unified framework called MM-3DScene is yielded. We conduct comprehensive experiments on a host of downstream tasks. The consistent improvement (e.g., +6.1 mAP@0.5 on object detection and +2.2% mIoU on semantic segmentation) demonstrates the superiority of our approach.

End-to-end Speech Translation (E2E ST) aims to translate source speech into target translation without generating the intermediate transcript. However, existing approaches for E2E ST degrade considerably when only limited ST data are available. We observe that an ST model's performance strongly correlates with its embedding similarity from speech and transcript. In this paper, we propose Word-Aligned COntrastive learning (WACO), a novel method for few-shot speech-to-text translation. Our key idea is bridging word-level representations for both modalities via contrastive learning. We evaluate WACO and other methods on the MuST-C dataset, a widely used ST benchmark. Our experiments demonstrate that WACO outperforms the best baseline methods by 0.7-8.5 BLEU points with only 1-hour parallel data. Code is available at https://anonymous.4open.science/r/WACO .