优化器的高效和自动化设计在全栈自动系统中起着至关重要的作用。但是，优化器搜索中的先前方法通常受其可扩展性，生成性或样品效率的限制。为了将优化器搜索的研究和应用民主化，我们提出了第一个有效，可扩展和可推广的框架，可以直接搜索感兴趣的任务。我们首先观察到优化器更新从根本上是数学表达式应用于梯度。受到基础数学表达式的先天树结构的启发，我们将优化器的空间重新安排到一个超树中，每个路径都编码优化器。这样，优化器搜索可以自然地作为路径找到问题，从而使各种建立的树遍历方法可以用作搜索算法。我们采用蒙特卡洛方法的改编来进行树木搜索，配备拒绝采样和等效形式检测，以利用优化器更新规则的特征来进一步提高样本效率。我们提供了一套多种任务，以基于我们的算法进行基准测试，并证明，只有128个评估，提出的框架可以发现超过人类设计的对应方和先前的优化器搜索方法的优化器。

联邦学习〜（FL）最近引起了学术界和行业的越来越多的关注，其最终目标是在隐私和沟通限制下进行协作培训。现有的基于FL算法的现有迭代模型需要大量的通信回合，以获得良好的模型，这是由于不同客户之间的极为不平衡和非平衡的I.D数据分配。因此，我们建议FedDM从多个本地替代功能中构建全球培训目标，这使服务器能够获得对损失格局的更全球视野。详细说明，我们在每个客户端构建了合成数据集，以在本地匹配从原始数据到分发匹配的损失景观。与笨拙的模型权重相比，FedDM通过传输更多信息和较小的合成数据来降低通信回合并提高模型质量。我们对三个图像分类数据集进行了广泛的实验，结果表明，在效率和模型性能方面，我们的方法可以优于其他FL的实验。此外，我们证明，FedDM可以适应使用高斯机制来保护差异隐私，并在相同的隐私预算下训练更好的模型。

极端多标签文本分类（XMC）问题问题是从大型标签集查找输入文本实例的大多数相关标签。但是，XMC设置面临两个挑战：（1）不允许在动态环境中预测看不见的标签，（2）它需要大量监督（实例，标签）对，这可能难以获得新兴域名。最近，已经研究了广义零拍XMC（GZ-XMC）设置，并相应地提出了Zestxml以处理未经调整的标签，这仍需要大量注释（实例，标签）对。在本文中，我们考虑了一个更实际的场景，称为极端零拍摄XMC（EZ-XMC），其中不需要监督，并且只能访问实例的原始文本和标签。少量XMC（FS-XMC），还调查了具有有限监督的EZ-XMC的扩展。要学习实例的语义嵌入和标签与原始文本，我们建议预先列车基于变压器的编码器，具有自我监督的对比损失。具体而言，我们开发了一种预训练方法MACLR，它彻底利用了使用多尺度自适应聚类，标签正则化和具有伪正对的自我训练的技术的原始文本。四个公共EZ-XMC数据集的实验结果表明，与所有其他领先的基线方法相比，MaclR达到了卓越的性能，特别是平均精度和召回的预测约为5-10％。此外，我们还表明，当在训练中存在有限数量的地面真相阳性对时，我们的预训练编码器可以进一步提高FS-XMC。通过在这样的几滴子集中进行微调，Maclr仍然显着优于其他极端分类器。

现有的转移攻击方法通常假定攻击者知道黑盒受害者模型的训练集（例如标签集，输入大小），这通常是不现实的，因为在某些情况下，攻击者不知道此信息。在本文中，我们定义了一个通用的可转移攻击（GTA）问题，在该问题中，攻击者不知道此信息，并获得攻击可能来自未知数据集的任何随机遇到的图像。为了解决GTA问题，我们提出了一种新颖的图像分类橡皮擦（ICE），该图像分类（ICE）训练特定的攻击者从任意数据集中擦除任何图像的分类信息。几个数据集的实验表明，ICE在GTA上的现有转移攻击极大地胜过了转移攻击，并表明ICE使用类似纹理的噪声来扰动不同数据集的不同图像。此外，快速傅立叶变换分析表明，每个冰噪声中的主要成分是R，G和B图像通道的三个正弦波。受这个有趣的发现的启发，我们设计了一种新颖的正弦攻击方法（SA），以优化三个正弦波。实验表明，SA的性能与冰相当，表明这三个正弦波是有效的，足以打破GTA设置下的DNN。

Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.

Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.

Time-series anomaly detection is an important task and has been widely applied in the industry. Since manual data annotation is expensive and inefficient, most applications adopt unsupervised anomaly detection methods, but the results are usually sub-optimal and unsatisfactory to end customers. Weak supervision is a promising paradigm for obtaining considerable labels in a low-cost way, which enables the customers to label data by writing heuristic rules rather than annotating each instance individually. However, in the time-series domain, it is hard for people to write reasonable labeling functions as the time-series data is numerically continuous and difficult to be understood. In this paper, we propose a Label-Efficient Interactive Time-Series Anomaly Detection (LEIAD) system, which enables a user to improve the results of unsupervised anomaly detection by performing only a small amount of interactions with the system. To achieve this goal, the system integrates weak supervision and active learning collaboratively while generating labeling functions automatically using only a few labeled data. All of these techniques are complementary and can promote each other in a reinforced manner. We conduct experiments on three time-series anomaly detection datasets, demonstrating that the proposed system is superior to existing solutions in both weak supervision and active learning areas. Also, the system has been tested in a real scenario in industry to show its practicality.

This paper investigates the use of artificial neural networks (ANNs) to solve differential equations (DEs) and the construction of the loss function which meets both differential equation and its initial/boundary condition of a certain DE. In section 2, the loss function is generalized to $n^\text{th}$ order ordinary differential equation(ODE). Other methods of construction are examined in Section 3 and applied to three different models to assess their effectiveness.

Kernels are efficient in representing nonlocal dependence and they are widely used to design operators between function spaces. Thus, learning kernels in operators from data is an inverse problem of general interest. Due to the nonlocal dependence, the inverse problem can be severely ill-posed with a data-dependent singular inversion operator. The Bayesian approach overcomes the ill-posedness through a non-degenerate prior. However, a fixed non-degenerate prior leads to a divergent posterior mean when the observation noise becomes small, if the data induces a perturbation in the eigenspace of zero eigenvalues of the inversion operator. We introduce a data-adaptive prior to achieve a stable posterior whose mean always has a small noise limit. The data-adaptive prior's covariance is the inversion operator with a hyper-parameter selected adaptive to data by the L-curve method. Furthermore, we provide a detailed analysis on the computational practice of the data-adaptive prior, and demonstrate it on Toeplitz matrices and integral operators. Numerical tests show that a fixed prior can lead to a divergent posterior mean in the presence of any of the four types of errors: discretization error, model error, partial observation and wrong noise assumption. In contrast, the data-adaptive prior always attains posterior means with small noise limits.

Deep learning has been widely used for protein engineering. However, it is limited by the lack of sufficient experimental data to train an accurate model for predicting the functional fitness of high-order mutants. Here, we develop SESNet, a supervised deep-learning model to predict the fitness for protein mutants by leveraging both sequence and structure information, and exploiting attention mechanism. Our model integrates local evolutionary context from homologous sequences, the global evolutionary context encoding rich semantic from the universal protein sequence space and the structure information accounting for the microenvironment around each residue in a protein. We show that SESNet outperforms state-of-the-art models for predicting the sequence-function relationship on 26 deep mutational scanning datasets. More importantly, we propose a data augmentation strategy by leveraging the data from unsupervised models to pre-train our model. After that, our model can achieve strikingly high accuracy in prediction of the fitness of protein mutants, especially for the higher order variants (> 4 mutation sites), when finetuned by using only a small number of experimental mutation data (<50). The strategy proposed is of great practical value as the required experimental effort, i.e., producing a few tens of experimental mutation data on a given protein, is generally affordable by an ordinary biochemical group and can be applied on almost any protein.