The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community.

We present temporally layered architecture (TLA), a biologically inspired system for temporally adaptive distributed control. TLA layers a fast and a slow controller together to achieve temporal abstraction that allows each layer to focus on a different time-scale. Our design is biologically inspired and draws on the architecture of the human brain which executes actions at different timescales depending on the environment's demands. Such distributed control design is widespread across biological systems because it increases survivability and accuracy in certain and uncertain environments. We demonstrate that TLA can provide many advantages over existing approaches, including persistent exploration, adaptive control, explainable temporal behavior, compute efficiency and distributed control. We present two different algorithms for training TLA: (a) Closed-loop control, where the fast controller is trained over a pre-trained slow controller, allowing better exploration for the fast controller and closed-loop control where the fast controller decides whether to "act-or-not" at each timestep; and (b) Partially open loop control, where the slow controller is trained over a pre-trained fast controller, allowing for open loop-control where the slow controller picks a temporally extended action or defers the next n-actions to the fast controller. We evaluated our method on a suite of continuous control tasks and demonstrate the advantages of TLA over several strong baselines.

Social insects such as ants communicate via pheromones which allows them to coordinate their activity and solve complex tasks as a swarm, e.g. foraging for food. This behaviour was shaped through evolutionary processes. In computational models, self-coordination in swarms has been implemented using probabilistic or action rules to shape the decision of each agent and the collective behaviour. However, manual tuned decision rules may limit the behaviour of the swarm. In this work we investigate the emergence of self-coordination and communication in evolved swarms without defining any rule. We evolve a swarm of agents representing an ant colony. We use a genetic algorithm to optimize a spiking neural network (SNN) which serves as an artificial brain to control the behaviour of each agent. The goal of the colony is to find optimal ways to forage for food in the shortest amount of time. In the evolutionary phase, the ants are able to learn to collaborate by depositing pheromone near food piles and near the nest to guide its cohorts. The pheromone usage is not encoded into the network; instead, this behaviour is established through the optimization procedure. We observe that pheromone-based communication enables the ants to perform better in comparison to colonies where communication did not emerge. We assess the foraging performance by comparing the SNN based model to a rule based system. Our results show that the SNN based model can complete the foraging task more efficiently in a shorter time. Our approach illustrates that even in the absence of pre-defined rules, self coordination via pheromone emerges as a result of the network optimization. This work serves as a proof of concept for the possibility of creating complex applications utilizing SNNs as underlying architectures for multi-agent interactions where communication and self-coordination is desired.

This chapter sheds light on the synaptic organization of the brain from the perspective of computational neuroscience. It provides an introductory overview on how to account for empirical data in mathematical models, implement them in software, and perform simulations reflecting experiments. This path is demonstrated with respect to four key aspects of synaptic signaling: the connectivity of brain networks, synaptic transmission, synaptic plasticity, and the heterogeneity across synapses. Each step and aspect of the modeling and simulation workflow comes with its own challenges and pitfalls, which are highlighted and addressed in detail.

In recent years, multilingual pre-trained language models have gained prominence due to their remarkable performance on numerous downstream Natural Language Processing tasks (NLP). However, pre-training these large multilingual language models requires a lot of training data, which is not available for African Languages. Active learning is a semi-supervised learning algorithm, in which a model consistently and dynamically learns to identify the most beneficial samples to train itself on, in order to achieve better optimization and performance on downstream tasks. Furthermore, active learning effectively and practically addresses real-world data scarcity. Despite all its benefits, active learning, in the context of NLP and especially multilingual language models pretraining, has received little consideration. In this paper, we present AfroLM, a multilingual language model pretrained from scratch on 23 African languages (the largest effort to date) using our novel self-active learning framework. Pretrained on a dataset significantly (14x) smaller than existing baselines, AfroLM outperforms many multilingual pretrained language models (AfriBERTa, XLMR-base, mBERT) on various NLP downstream tasks (NER, text classification, and sentiment analysis). Additional out-of-domain sentiment analysis experiments show that \textbf{AfroLM} is able to generalize well across various domains. We release the code source, and our datasets used in our framework at https://github.com/bonaventuredossou/MLM_AL.

我们介绍了Sparrow，这是一个寻求信息的对话代理，与提示的语言模型基线相比，训练有素，更有帮助，正确和无害。我们使用从人类反馈中的强化学习来培训我们的模型，以帮助人类评估者判断代理人的行为。首先，为了使我们的代理人更有帮助和无害，我们将良好对话的要求分解为代理人应遵循的自然语言规则，并分别向评估者询问每个规则。我们证明，这种崩溃使我们能够收集对代理行为的更多针对性的人类判断，并允许更有效的规则条件奖励模型。其次，我们的代理商在收集对模型声明的偏好判决时提供了支持事实主张的来源的证据。对于事实问题，麻雀提供的证据支持了78％的时间。比基线比基线更享受麻雀，同时对人类的对抗性探测更具弹性，在探测时只有8％的时间违反了我们的规则。最后，我们进行了广泛的分析，表明尽管我们的模型学会遵守我们的规则，但它可以表现出分布偏见。

数据质量是发展医疗保健中值得信赖的AI的关键因素。大量具有控制混杂因素的策划数据集可以帮助提高下游AI算法的准确性，鲁棒性和隐私性。但是，访问高质量的数据集受数据获取的技术难度的限制，并且严格的道德限制阻碍了医疗保健数据的大规模共享。数据合成算法生成具有与真实临床数据相似的分布的数据，可以作为解决可信度AI的发展过程中缺乏优质数据的潜在解决方案。然而，最新的数据合成算法，尤其是深度学习算法，更多地集中于成像数据，同时忽略了非成像医疗保健数据的综合，包括临床测量，医疗信号和波形以及电子保健记录（EHRS）（EHRS） 。因此，在本文中，我们将回顾合成算法，尤其是对于非成像医学数据，目的是在该领域提供可信赖的AI。本教程风格的审查论文将对包括算法，评估，局限性和未来研究方向在内的各个方面进行全面描述。

气道分割对于检查，诊断和预后的肺部疾病至关重要，而其手动描述则不当。为了减轻这种耗时且潜在的主观手动程序，研究人员提出了从计算机断层扫描（CT）图像自动分割气道的方法。但是，一些小型气道分支（例如，支气管和终末支气管）显着加剧了通过机器学习模型的自动分割难度。特别是，气道分支中体素值和严重的数据失衡的方差使计算模块容易导致不连续和假阴性预测。注意机制表明了分割复杂结构的能力，而模糊逻辑可以减少特征表示的不确定性。因此，由模糊注意力层给出的深度注意力网络和模糊理论的整合应该是升级的解决方案。本文提出了一种有效的气道分割方法，包括一个新型的模糊注意力神经网络和全面的损失函数，以增强气道分割的空间连续性。深层模糊集由特征图中的一组体素和可学习的高斯成员功能制定。与现有的注意机制不同，所提出的特异性模糊注意力解决了不同渠道中异质特征的问题。此外，提出了一种新的评估指标来评估气道结构的连续性和完整性。该方法的效率已通过在包括精确的09和LIDC数据集在内的开放数据集上进行测试，以及我们的内部Covid-19和纤维化肺病数据集证明了这一建议的效率。

我们提出了一种基于事件的降雪算法，称为EBSNOR。我们开发了一种技术，可以使用基于事件的相机数据来测量像素上雪花的停留时间，该数据用于进行Neyman-Pearson假设测试，以将事件流分为雪花和背景事件。在一个名为udayton22ebsnow的新数据集上验证了拟议的EBSNOR的有效性，该数据集由前面事件的摄像机组成，该相机在汽车中驾驶雪中，并在周围车辆周围手动注释的边界盒。在定性上，Ebsnor正确地标识了与雪花相对应的事件；并且在定量上，EBSNOR预处理的事件数据改善了基于事件的CAR检测算法的性能。

在人工智能的许多应用中，细粒度的变化检测和回归分析至关重要。实际上，由于缺乏可靠的基础真理信息和复杂性，因此这项任务通常是有挑战性的。因此，开发一个可以代表多个信息源的相关性和可靠性至关重要的框架。在本文中，我们调查了如何将多任务指标学习中的技术应用于实际数据中的细粒度变化。关键思想是，如果我们将一个单个对象的特定实例之间的兴趣指标中的增量变化纳入作为多任务指标学习框架中的一项任务，然后解释该限制将使用户被警报以对整体度量的整体度量不变。研究的技术是专门针对处理异质数据源的专门量身定制的。每个任务的输入数据可能包含缺失的值，该值的比例和分辨率在任务之间不存在，并且数据包含非独立且相同分布的（非IID）实例。根据我们最初的实验实施结果的结果，并讨论了该域中的相关研究，这可能为进一步的研究提供了方向。