该文档的目的是对变压器体系结构和算法的独立，数学精确的概述（*非*结果）。它涵盖了变压器是什么，他们的训练方式，使用的方式，其关键架构组件以及最突出的模型的预览。假定读者熟悉基本的ML术语和更简单的神经网络体系结构，例如MLP。

Symmetry arises in many optimization and decision-making problems, and has attracted considerable attention from the optimization community: By utilizing the existence of such symmetries, the process of searching for optimal solutions can be improved significantly. Despite its success in (offline) optimization, the utilization of symmetries has not been well examined within the online optimization settings, especially in the bandit literature. As such, in this paper we study the invariant Lipschitz bandit setting, a subclass of the Lipschitz bandits where the reward function and the set of arms are preserved under a group of transformations. We introduce an algorithm named \texttt{UniformMesh-N}, which naturally integrates side observations using group orbits into the \texttt{UniformMesh} algorithm (\cite{Kleinberg2005_UniformMesh}), which uniformly discretizes the set of arms. Using the side-observation approach, we prove an improved regret upper bound, which depends on the cardinality of the group, given that the group is finite. We also prove a matching regret's lower bound for the invariant Lipschitz bandit class (up to logarithmic factors). We hope that our work will ignite further investigation of symmetry in bandit theory and sequential decision-making theory in general.

Classical differential private DP-SGD implements individual clipping with random subsampling, which forces a mini-batch SGD approach. We provide a general differential private algorithmic framework that goes beyond DP-SGD and allows any possible first order optimizers (e.g., classical SGD and momentum based SGD approaches) in combination with batch clipping, which clips an aggregate of computed gradients rather than summing clipped gradients (as is done in individual clipping). The framework also admits sampling techniques beyond random subsampling such as shuffling. Our DP analysis follows the $f$-DP approach and introduces a new proof technique which allows us to also analyse group privacy. In particular, for $E$ epochs work and groups of size $g$, we show a $\sqrt{g E}$ DP dependency for batch clipping with shuffling. This is much better than the previously anticipated linear dependency in $g$ and is much better than the previously expected square root dependency on the total number of rounds within $E$ epochs which is generally much more than $\sqrt{E}$.

The deployment of robots in uncontrolled environments requires them to operate robustly under previously unseen scenarios, like irregular terrain and wind conditions. Unfortunately, while rigorous safety frameworks from robust optimal control theory scale poorly to high-dimensional nonlinear dynamics, control policies computed by more tractable "deep" methods lack guarantees and tend to exhibit little robustness to uncertain operating conditions. This work introduces a novel approach enabling scalable synthesis of robust safety-preserving controllers for robotic systems with general nonlinear dynamics subject to bounded modeling error by combining game-theoretic safety analysis with adversarial reinforcement learning in simulation. Following a soft actor-critic scheme, a safety-seeking fallback policy is co-trained with an adversarial "disturbance" agent that aims to invoke the worst-case realization of model error and training-to-deployment discrepancy allowed by the designer's uncertainty. While the learned control policy does not intrinsically guarantee safety, it is used to construct a real-time safety filter (or shield) with robust safety guarantees based on forward reachability rollouts. This shield can be used in conjunction with a safety-agnostic control policy, precluding any task-driven actions that could result in loss of safety. We evaluate our learning-based safety approach in a 5D race car simulator, compare the learned safety policy to the numerically obtained optimal solution, and empirically validate the robust safety guarantee of our proposed safety shield against worst-case model discrepancy.

Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-independent and identically distributed~(Non-IID) private data and unevenly distributed computational resources. Preserving user data privacy while optimizing AI/ML models in a heterogeneous federated network requires us to address data heterogeneity and system/resource heterogeneity. Hence, we propose \underline{R}esource-\underline{a}ware \underline{F}ederated \underline{L}earning~(RaFL) to address these challenges. RaFL allocates resource-aware models to edge devices using Neural Architecture Search~(NAS) and allows heterogeneous model architecture deployment by knowledge extraction and fusion. Integrating NAS into FL enables on-demand customized model deployment for resource-diverse edge devices. Furthermore, we propose a multi-model architecture fusion scheme allowing the aggregation of the distributed learning results. Results demonstrate RaFL's superior resource efficiency compared to SoTA.

We introduce efficient deep learning-based methods for legal document processing including Legal Document Retrieval and Legal Question Answering tasks in the Automated Legal Question Answering Competition (ALQAC 2022). In this competition, we achieve 1\textsuperscript{st} place in the first task and 3\textsuperscript{rd} place in the second task. Our method is based on the XLM-RoBERTa model that is pre-trained from a large amount of unlabeled corpus before fine-tuning to the specific tasks. The experimental results showed that our method works well in legal retrieval information tasks with limited labeled data. Besides, this method can be applied to other information retrieval tasks in low-resource languages.

当植物天然产物与药物共容纳时，就会发生药代动力学天然产物 - 药物相互作用（NPDIS）。了解NPDI的机制是防止不良事件的关键。我们构建了一个知识图框架NP-KG，作为迈向药代动力学NPDIS的计算发现的一步。 NP-KG是一个具有生物医学本体论，链接数据和科学文献的全文，由表型知识翻译框架和语义关系提取系统，SEMREP和集成网络和动态推理组成的构建的科学文献的全文。通过路径搜索和元路径发现对药代动力学绿茶和kratom-prug相互作用的案例研究评估NP-KG，以确定与地面真实数据相比的一致性和矛盾信息。完全集成的NP-KG由745,512个节点和7,249,576个边缘组成。 NP-KG的评估导致了一致（绿茶的38.98％，kratom的50％），矛盾（绿茶的15.25％，21.43％，Kratom的21.43％），同等和矛盾的（15.25％）（21.43％，21.43％，21.43％ kratom）信息。几种声称的NPDI的潜在药代动力学机制，包括绿茶 - 茶氧化烯，绿茶 - 纳多洛尔，Kratom-Midazolam，Kratom-Quetiapine和Kratom-Venlafaxine相互作用，与已出版的文献一致。 NP-KG是第一个将生物医学本体论与专注于天然产品的科学文献的全文相结合的公斤。我们证明了NP-KG在鉴定涉及酶，转运蛋白和药物的药代动力学相互作用的应用。我们设想NP-KG将有助于改善人机合作，以指导研究人员将来对药代动力学NPDIS进行研究。 NP-KG框架可在https://doi.org/10.5281/zenodo.6814507和https://github.com/sanyabt/np-kg上公开获得。

我们的粮食安全建立在土壤的基础上。如果土壤不健康，农民将无法用纤维，食物和燃料喂养我们。准确预测土壤的类型有助于规划土壤的使用，从而提高生产率。这项研究采用了最先进的视觉变压器，并与SVM，Alexnet，Resnet和CNN等不同模型进行了比较。此外，这项研究还着重于区分不同的视觉变压器体系结构。对于土壤类型的分类，数据集由4种不同类型的土壤样品组成，例如冲积，红色，黑色和粘土。 Visual Transformer模型在测试和测试时达到98.13％的训练和93.62％的范围，在测试和训练精度方面都优于其他模型。视觉变压器的性能超过了其他模型的性能至少2％。因此，新颖的视觉变压器可用于计算机视觉任务，包括土壤分类。

由于欺诈模式随着时间的流逝而变化，并且欺诈示例的可用性有限，以学习这种复杂的模式，因此欺诈检测是一项具有挑战性的任务。因此，借助智能版本的机器学习（ML）工具的欺诈检测对于确保安全至关重要。欺诈检测是主要的ML分类任务；但是，相应的ML工具的最佳性能取决于最佳的超参数值的使用。此外，在不平衡类中的分类非常具有挑战性，因为它在少数群体中导致绩效差，大多数ML分类技术都忽略了。因此，我们研究了四种最先进的ML技术，即逻辑回归，决策树，随机森林和极端梯度提升，它们适用于处理不平衡类别以最大程度地提高精度并同时降低假阳性。首先，这些分类器经过两个原始基准测试不平衡检测数据集的培训，即网站网站URL和欺诈性信用卡交易。然后，通过实现采样框架，即RandomundSampler，Smote和Smoteenn，为每个原始数据集生产了三个合成平衡的数据集。使用RandomzedSearchCV方法揭示了所有16个实验的最佳超参数。使用两个基准性能指标比较了欺诈检测中16种方法的有效性，即接收器操作特性（AUC ROC）和精度和召回曲线下的面积（AUC PR）（AUC PR）。对于网络钓鱼网站URL和信用卡欺诈事务数据集，结果表明，对原始数据的极端梯度提升显示了不平衡数据集中值得信赖的性能，并以AUC ROC和AUC PR来超越其他三种方法。

我们介绍了贴片采样时间表（PSS）的概念，该概念在训练过程中每批次使用的视觉变压器（VIT）贴片的数量变化。由于对于大多数视觉目标（例如，分类），所有补丁都不同样重要，因此我们认为，不太重要的补丁可以用于较少的训练迭代中，从而导致较短的训练时间，对性能的影响最小。此外，我们观察到，使用PSS的训练可以使VIT在推理过程中对更宽的贴片采样范围更强。这允许在推理过程中进行吞吐量和准确性之间的细粒度，动态的权衡。我们使用PSSS在VIT上评估Imagenet的VIT，均通过从头开始训练并使用重建损耗函数进行了预训练。对于预训练的模型，与使用所有斑块相比，我们的分类准确性降低了0.26％（从25小时到17小时）降低了0.26％。代码，模型检查点和日志可在https://github.com/bradmcdanel/pss上找到。