由于它们在现实世界中的广泛采用，提高深神经网络（DNN）的运行时性能至关重要。现有的优化DNN的张量代数表达的方法仅考虑由固定的预定义运算符表示的表达式，在一般表达式之间缺少可能的优化机会。我们提出了Ollie，这是第一个基于衍生的张量程序优化器。 Ollie通过利用一般张量代数表达式之间的转换来优化张量程序，从而实现了一个更大的表达搜索空间，其中包括由先前工作作为特殊情况支持的搜索空间。 Ollie使用基于混合衍生的优化器，该优化器有效地结合了探索性和指导性推导，以快速发现高度优化的表达式。对七个DNN的评估表明，Ollie可以在A100 GPU上胜过2.73 $ \ times $（平均为1.46美元$ \ times $），在V100上最多可超过2.68 $ \ times $（1.51 $ \ times $） GPU分别。

域的概括旨在学习一个通用模型，该模型在看不见的目标域上表现良好，并结合了来自多个源域的知识。在这项研究中，我们考虑了以下场景，在不同类别跨领域的条件分布之间发生不同的领域变化。当源域中的标记样品受到限制时，现有方法不足以鲁棒。为了解决这个问题，我们提出了一个新型的域泛化框架，称为Wasserstein分布在鲁棒域的概括（WDRDG），灵感来自分布稳健优化的概念。我们鼓励对特定于类的Wasserstein不确定性集中有条件分布的鲁棒性，并优化分类器在这些不确定性集上的最差性能。我们进一步开发了一个测试时间适应模块，利用最佳运输来量化未见目标域和源域之间的关系，以使目标数据适应性推断。旋转MNIST，PACS和VLCS数据集的实验表明，我们的方法可以有效地平衡挑战性概括场景中的鲁棒性和可区分性。

最近，疾病控制和预防中心（CDC）与其他联邦机构合作，以鉴定冠心病疾病2019年（Covid-19）发病率（热点）的县，并为当地卫生部门提供支持，以限制疾病的传播。了解热点事件的时空动态非常重视支持政策决策并防止大规模爆发。本文提出了一种时空贝叶斯框架，用于早期检测美国Covid-19热点（在县级）。我们假设观察到的病例和热点都依赖于一类潜随机变量，其编码Covid-19传输的底层时空动态。这种潜在的变量遵循零均值高斯过程，其协方差由非静止内核功能指定。我们内核功能的最突出的特征是引入深度神经网络，以增强模型的代表性，同时仍然享有内核的可解释性。我们得出了一种稀疏的模型，并使用变分的学习策略适合模型，以规避大数据集的计算诡计。与其他基线方法相比，我们的模型展示了更好的解释性和优越的热点检测性能。

从嘈杂的观察结果中检测网络社区结构的突然变化是统计和机器学习中的一个基本问题。本文介绍了一种称为Spectral-Cusum的在线变更检测算法，以通过广义似然比统计量来检测未知网络结构的变化。我们表征了光谱库过程的平均运行长度（ARL）和预期检测延迟（EDD），并证明其渐近最佳性。最后，我们证明了光谱曲线过程的良好性能，并使用传感器网络数据进行地震事件检测的模拟和真实数据示例将其与多种基线方法进行比较。

Driven by improved architectures and better representation learning frameworks, the field of visual recognition has enjoyed rapid modernization and performance boost in the early 2020s. For example, modern ConvNets, represented by ConvNeXt, have demonstrated strong performance in various scenarios. While these models were originally designed for supervised learning with ImageNet labels, they can also potentially benefit from self-supervised learning techniques such as masked autoencoders (MAE). However, we found that simply combining these two approaches leads to subpar performance. In this paper, we propose a fully convolutional masked autoencoder framework and a new Global Response Normalization (GRN) layer that can be added to the ConvNeXt architecture to enhance inter-channel feature competition. This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation. We also provide pre-trained ConvNeXt V2 models of various sizes, ranging from an efficient 3.7M-parameter Atto model with 76.7% top-1 accuracy on ImageNet, to a 650M Huge model that achieves a state-of-the-art 88.9% accuracy using only public training data.

A step-search sequential quadratic programming method is proposed for solving nonlinear equality constrained stochastic optimization problems. It is assumed that constraint function values and derivatives are available, but only stochastic approximations of the objective function and its associated derivatives can be computed via inexact probabilistic zeroth- and first-order oracles. Under reasonable assumptions, a high-probability bound on the iteration complexity of the algorithm to approximate first-order stationarity is derived. Numerical results on standard nonlinear optimization test problems illustrate the advantages and limitations of our proposed method.

Masked image modeling (MIM) has shown great promise for self-supervised learning (SSL) yet been criticized for learning inefficiency. We believe the insufficient utilization of training signals should be responsible. To alleviate this issue, we introduce a conceptually simple yet learning-efficient MIM training scheme, termed Disjoint Masking with Joint Distillation (DMJD). For disjoint masking (DM), we sequentially sample multiple masked views per image in a mini-batch with the disjoint regulation to raise the usage of tokens for reconstruction in each image while keeping the masking rate of each view. For joint distillation (JD), we adopt a dual branch architecture to respectively predict invisible (masked) and visible (unmasked) tokens with superior learning targets. Rooting in orthogonal perspectives for training efficiency improvement, DM and JD cooperatively accelerate the training convergence yet not sacrificing the model generalization ability. Concretely, DM can train ViT with half of the effective training epochs (3.7 times less time-consuming) to report competitive performance. With JD, our DMJD clearly improves the linear probing classification accuracy over ConvMAE by 5.8%. On fine-grained downstream tasks like semantic segmentation, object detection, etc., our DMJD also presents superior generalization compared with state-of-the-art SSL methods. The code and model will be made public at https://github.com/mx-mark/DMJD.

Considering the computation complexity, we propose a Guided Hybrid Quantization with One-to-one Self-Teaching (GHOST}) framework. More concretely, we first design a structure called guided quantization self-distillation (GQSD), which is an innovative idea for realizing lightweight through the synergy of quantization and distillation. The training process of the quantization model is guided by its full-precision model, which is time-saving and cost-saving without preparing a huge pre-trained model in advance. Second, we put forward a hybrid quantization (HQ) module to obtain the optimal bit width automatically under a constrained condition where a threshold for distribution distance between the center and samples is applied in the weight value search space. Third, in order to improve information transformation, we propose a one-to-one self-teaching (OST) module to give the student network a ability of self-judgment. A switch control machine (SCM) builds a bridge between the student network and teacher network in the same location to help the teacher to reduce wrong guidance and impart vital knowledge to the student. This distillation method allows a model to learn from itself and gain substantial improvement without any additional supervision. Extensive experiments on a multimodal dataset (VEDAI) and single-modality datasets (DOTA, NWPU, and DIOR) show that object detection based on GHOST outperforms the existing detectors. The tiny parameters (<9.7 MB) and Bit-Operations (BOPs) (<2158 G) compared with any remote sensing-based, lightweight or distillation-based algorithms demonstrate the superiority in the lightweight design domain. Our code and model will be released at https://github.com/icey-zhang/GHOST.

Automatic font generation without human experts is a practical and significant problem, especially for some languages that consist of a large number of characters. Existing methods for font generation are often in supervised learning. They require a large number of paired data, which are labor-intensive and expensive to collect. In contrast, common unsupervised image-to-image translation methods are not applicable to font generation, as they often define style as the set of textures and colors. In this work, we propose a robust deformable generative network for unsupervised font generation (abbreviated as DGFont++). We introduce a feature deformation skip connection (FDSC) to learn local patterns and geometric transformations between fonts. The FDSC predicts pairs of displacement maps and employs the predicted maps to apply deformable convolution to the low-level content feature maps. The outputs of FDSC are fed into a mixer to generate final results. Moreover, we introduce contrastive self-supervised learning to learn a robust style representation for fonts by understanding the similarity and dissimilarities of fonts. To distinguish different styles, we train our model with a multi-task discriminator, which ensures that each style can be discriminated independently. In addition to adversarial loss, another two reconstruction losses are adopted to constrain the domain-invariant characteristics between generated images and content images. Taking advantage of FDSC and the adopted loss functions, our model is able to maintain spatial information and generates high-quality character images in an unsupervised manner. Experiments demonstrate that our model is able to generate character images of higher quality than state-of-the-art methods.

Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.