集中式培训（CT）是许多受欢迎的多代理增强学习（MARL）方法的基础，因为它允许代理商快速学习高性能的政策。但是，CT依靠代理人从对特定州对其他代理商的行为的一次性观察中学习。由于MARL代理商在培训期间探索和更新其政策，因此这些观察结果通常会为其他代理商的行为和预期的给定行动回报提供不良的预测。因此，CT方法患有较高的差异和容易出错的估计，从而损害了学习。除非施加了强大的分解限制，否则CT方法还遭受了复杂性爆炸性增长（例如，QMIX的单调奖励函数）。我们通过一个新的半居中的MAL框架来应对这些挑战，该框架执行政策安装的培训和分散的执行。我们的方法是嵌入式增强学习算法（PERLA），是参与者批评的MARL算法的增强工具，它利用了一种新型参数共享协议和策略嵌入方法来维持对其他代理商的行为的估计。我们的理论证明，佩拉大大降低了价值估计的差异。与各种CT方法不同，Perla无缝地采用MARL算法，它可以轻松地与代理数量缩放，而无需限制性分解假设。我们展示了Perla在基准环境中的出色经验表现和有效的缩放，包括Starcraft Micromagement II和Multi-Agent Mujoco

几乎可以肯定（或使用概率）满足安全限制对于在现实生活中的增强学习（RL）的部署至关重要。例如，理想情况下，平面降落和起飞应以概率为单位发生。我们通过引入安全增强（SAUTE）马尔可夫决策过程（MDP）来解决该问题，在该过程中，通过将其扩大到州空间并重塑目标来消除安全限制。我们表明，Saute MDP满足了Bellman方程，并使我们更加接近解决安全的RL，几乎可以肯定地满足。我们认为，Saute MDP允许从不同的角度查看安全的RL问题，从而实现新功能。例如，我们的方法具有插件的性质，即任何RL算法都可以“炒”。此外，国家扩展允许跨安全限制进行政策概括。我们最终表明，当约束满意度非常重要时，SAUTE RL算法的表现可以胜过其最先进的对应物。

奖励成型（RS）是克服稀疏或不信息奖励问题的强大方法（RL）。但是，RS通常依赖于手动设计的成型奖励功能，其构造耗时且容易出错。它还需要与自主学习目标相反的领域知识。我们介绍了增强学习优化塑造算法（ROSA）的增强型，这是一个自动化的RS框架，其中塑造奖励函数是在两个代理之间的新型马尔可夫游戏中构建的。奖励塑料代理（Shaper）使用切换控件来确定在其他代理（控制器）使用这些形状奖励的任务中学习任务的最佳策略，以确定要添加形状奖励及其最佳值的状态。我们证明，Rosa很容易采用现有的RL算法，学会了构建针对任务的塑造奖励功能，从而确保有效地收敛到高性能策略。我们在三个经过精心设计的实验中展示了罗莎（Rosa）在挑战稀疏奖励环境中对最先进的RS算法的优越性能。

A Complete Computer vision system can be divided into two main categories: detection and classification. The Lane detection algorithm is a part of the computer vision detection category and has been applied in autonomous driving and smart vehicle systems. The lane detection system is responsible for lane marking in a complex road environment. At the same time, lane detection plays a crucial role in the warning system for a car when departs the lane. The implemented lane detection algorithm is mainly divided into two steps: edge detection and line detection. In this paper, we will compare the state-of-the-art implementation performance obtained with both FPGA and GPU to evaluate the trade-off for latency, power consumption, and utilization. Our comparison emphasises the advantages and disadvantages of the two systems.

Robustness of different pattern recognition methods is one of the key challenges in autonomous driving, especially when driving in the high variety of road environments and weather conditions, such as gravel roads and snowfall. Although one can collect data from these adverse conditions using cars equipped with sensors, it is quite tedious to annotate the data for training. In this work, we address this limitation and propose a CNN-based method that can leverage the steering wheel angle information to improve the road area semantic segmentation. As the steering wheel angle data can be easily acquired with the associated images, one could improve the accuracy of road area semantic segmentation by collecting data in new road environments without manual data annotation. We demonstrate the effectiveness of the proposed approach on two challenging data sets for autonomous driving and show that when the steering task is used in our segmentation model training, it leads to a 0.1-2.9% gain in the road area mIoU (mean Intersection over Union) compared to the corresponding reference transfer learning model.

Traffic forecasting has attracted widespread attention recently. In reality, traffic data usually contains missing values due to sensor or communication errors. The Spatio-temporal feature in traffic data brings more challenges for processing such missing values, for which the classic techniques (e.g., data imputations) are limited: 1) in temporal axis, the values can be randomly or consecutively missing; 2) in spatial axis, the missing values can happen on one single sensor or on multiple sensors simultaneously. Recent models powered by Graph Neural Networks achieved satisfying performance on traffic forecasting tasks. However, few of them are applicable to such a complex missing-value context. To this end, we propose GCN-M, a Graph Convolutional Network model with the ability to handle the complex missing values in the Spatio-temporal context. Particularly, we jointly model the missing value processing and traffic forecasting tasks, considering both local Spatio-temporal features and global historical patterns in an attention-based memory network. We propose as well a dynamic graph learning module based on the learned local-global features. The experimental results on real-life datasets show the reliability of our proposed method.

Current pre-trained language models rely on large datasets for achieving state-of-the-art performance. However, past research has shown that not all examples in a dataset are equally important during training. In fact, it is sometimes possible to prune a considerable fraction of the training set while maintaining the test performance. Established on standard vision benchmarks, two gradient-based scoring metrics for finding important examples are GraNd and its estimated version, EL2N. In this work, we employ these two metrics for the first time in NLP. We demonstrate that these metrics need to be computed after at least one epoch of fine-tuning and they are not reliable in early steps. Furthermore, we show that by pruning a small portion of the examples with the highest GraNd/EL2N scores, we can not only preserve the test accuracy, but also surpass it. This paper details adjustments and implementation choices which enable GraNd and EL2N to be applied to NLP.

It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.

In recent years, spammers are now trying to obfuscate their intents by introducing hybrid spam e-mail combining both image and text parts, which is more challenging to detect in comparison to e-mails containing text or image only. The motivation behind this research is to design an effective approach filtering out hybrid spam e-mails to avoid situations where traditional text-based or image-baesd only filters fail to detect hybrid spam e-mails. To the best of our knowledge, a few studies have been conducted with the goal of detecting hybrid spam e-mails. Ordinarily, Optical Character Recognition (OCR) technology is used to eliminate the image parts of spam by transforming images into text. However, the research questions are that although OCR scanning is a very successful technique in processing text-and-image hybrid spam, it is not an effective solution for dealing with huge quantities due to the CPU power required and the execution time it takes to scan e-mail files. And the OCR techniques are not always reliable in the transformation processes. To address such problems, we propose new late multi-modal fusion training frameworks for a text-and-image hybrid spam e-mail filtering system compared to the classical early fusion detection frameworks based on the OCR method. Convolutional Neural Network (CNN) and Continuous Bag of Words were implemented to extract features from image and text parts of hybrid spam respectively, whereas generated features were fed to sigmoid layer and Machine Learning based classifiers including Random Forest (RF), Decision Tree (DT), Naive Bayes (NB) and Support Vector Machine (SVM) to determine the e-mail ham or spam.

近年来，全球医学事物（IOMT）行业已经以极大的速度发展。由于IOMT网络的庞大规模和部署，安全和隐私是IOMT的关键问题。机器学习（ML）和区块链（BC）技术已大大提高了Healthcare 5.0的功能和设施，并产生了一个名为“ Smart Healthcare”的新领域。通过早期确定问题，智能医疗保健系统可以帮助避免长期损害。这将提高患者的生活质量，同时减少压力和医疗保健费用。 IOMT在信息技术领域中启用了一系列功能，其中之一是智能和互动的医疗保健。但是，将医疗数据合并到单个存储位置以训练强大的机器学习模型，这引起了人们对隐私，所有权和更加集中的遵守的担忧。联合学习（FL）通过利用集中式聚合服务器来传播全球学习模型，从而克服了前面的困难。同时，本地参与者可以控制患者信息，从而确保数据机密性和安全性。本文对与医疗保健中联邦学习纠缠的区块链技术的发现进行了全面分析。 5.0。这项研究的目的是利用区块链技术和入侵检测系统（IDS）在医疗保健5.0中构建安全的健康监测系统，以检测医疗保健网络中的任何恶意活动，并使医生能够通过医疗传感器监控患者并采取必要的措施。定期通过预测疾病。