Geologic cores are rock samples that are extracted from deep under the ground during the well drilling process. They are used for petroleum reservoirs' performance characterization. Traditionally, physical studies of cores are carried out by the means of manual time-consuming experiments. With the development of deep learning, scientists actively started working on developing machine-learning-based approaches to identify physical properties without any manual experiments. Several previous works used machine learning to determine the porosity and permeability of the rocks, but either method was inaccurate or computationally expensive. We are proposing to use self-supervised pretraining of the very small CNN-transformer-based model to predict the physical properties of the rocks with high accuracy in a time-efficient manner. We show that this technique prevents overfitting even for extremely small datasets. Github: https://github.com/Shahbozjon/porosity-and-permeability-prediction

与RGB图像相比，高光谱图像包含更多数量的通道，因此包含有关图像中实体的更多信息。卷积神经网络（CNN）和多层感知器（MLP）已被证明是一种有效的图像分类方法。但是，他们遭受了长期培训时间和大量标记数据的要求，以达到预期的结果。在处理高光谱图像时，这些问题变得更加复杂。为了减少训练时间并减少对大型标记数据集的依赖性，我们建议使用转移学习方法。使用PCA将高光谱数据集预处理到较低的维度，然后将深度学习模型应用于分类。然后，转移学习模型使用该模型学到的功能来解决看不见的数据集上的新分类问题。进行了CNN和多个MLP体系结构模型的详细比较，以确定最适合目标的最佳体系结构。结果表明，层的缩放并不总是会导致准确性的提高，但通常会导致过度拟合，并增加训练时间。通过应用转移学习方法而不仅仅是解决问题，训练时间更大程度地减少了。通过直接在大型数据集上训练新模型，而不会影响准确性。

海洋生态系统及其鱼类栖息地越来越重要，因为它们在提供有价值的食物来源和保护效果方面的重要作用。由于它们的偏僻且难以接近自然，因此通常使用水下摄像头对海洋环境和鱼类栖息地进行监测。这些相机产生了大量数字数据，这些数据无法通过当前的手动处理方法有效地分析，这些方法涉及人类观察者。 DL是一种尖端的AI技术，在分析视觉数据时表现出了前所未有的性能。尽管它应用于无数领域，但仍在探索其在水下鱼类栖息地监测中的使用。在本文中，我们提供了一个涵盖DL的关键概念的教程，该教程可帮助读者了解对DL的工作原理的高级理解。该教程还解释了一个逐步的程序，讲述了如何为诸如水下鱼类监测等挑战性应用开发DL算法。此外，我们还提供了针对鱼类栖息地监测的关键深度学习技术的全面调查，包括分类，计数，定位和细分。此外，我们对水下鱼类数据集进行了公开调查，并比较水下鱼类监测域中的各种DL技术。我们还讨论了鱼类栖息地加工深度学习的新兴领域的一些挑战和机遇。本文是为了作为希望掌握对DL的高级了解，通过遵循我们的分步教程而为其应用开发的海洋科学家的教程，并了解如何发展其研究，以促进他们的研究。努力。同时，它适用于希望调查基于DL的最先进方法的计算机科学家，以进行鱼类栖息地监测。

Remaining Useful Life (RUL) estimation plays a critical role in Prognostics and Health Management (PHM). Traditional machine health maintenance systems are often costly, requiring sufficient prior expertise, and are difficult to fit into highly complex and changing industrial scenarios. With the widespread deployment of sensors on industrial equipment, building the Industrial Internet of Things (IIoT) to interconnect these devices has become an inexorable trend in the development of the digital factory. Using the device's real-time operational data collected by IIoT to get the estimated RUL through the RUL prediction algorithm, the PHM system can develop proactive maintenance measures for the device, thus, reducing maintenance costs and decreasing failure times during operation. This paper carries out research into the remaining useful life prediction model for multi-sensor devices in the IIoT scenario. We investigated the mainstream RUL prediction models and summarized the basic steps of RUL prediction modeling in this scenario. On this basis, a data-driven approach for RUL estimation is proposed in this paper. It employs a Multi-Head Attention Mechanism to fuse the multi-dimensional time-series data output from multiple sensors, in which the attention on features is used to capture the interactions between features and attention on sequences is used to learn the weights of time steps. Then, the Long Short-Term Memory Network is applied to learn the features of time series. We evaluate the proposed model on two benchmark datasets (C-MAPSS and PHM08), and the results demonstrate that it outperforms the state-of-art models. Moreover, through the interpretability of the multi-head attention mechanism, the proposed model can provide a preliminary explanation of engine degradation. Therefore, this approach is promising for predictive maintenance in IIoT scenarios.

在这项工作中，我们提出了一种超大形态器，一种基于变压器的模型，用于几次学习，直接从支持样品产生卷积神经网络（CNN）的权重。由于小生成的CNN模型对特定任务的依赖性由高容量变压器模型编码，因此我们有效地将大型任务空间的复杂性与各个任务的复杂性分离。我们的方法对于小目标CNN架构特别有效，其中学习固定的通用任务无关的嵌入不是最佳的，并且在关于任务的信息可以调制所有模型参数时实现更好的性能。对于较大的模型，我们发现单独生成最后一层允许我们产生比使用最先进的方法获得的竞争或更好的结果，同时端到端可分辨率。最后，我们将我们的方法扩展到一个半监督的政权，利用支持集中的未标记样本，进一步提高少量射击性能。

从自然语言嵌入中汲取灵感，我们提出了Astromer，这是一种基于变压器的模型，以创建光曲线的表示。Astromer接受了数以百万计的Macho R波段样品的培训，并且很容易对其进行微调以匹配与下游任务相关的特定域。例如，本文显示了使用预训练的表示形式对变量恒星进行分类的好处。此外，我们还提供了一个Python库，其中包括这项工作中使用的所有功能。我们的图书馆包括预先培训的模型，可用于增强深度学习模型的性能，减少计算资源，同时获得最新的结果。

Practical applications of mechanical metamaterials often involve solving inverse problems where the objective is to find the (multiple) microarchitectures that give rise to a given set of properties. The limited resolution of additive manufacturing techniques often requires solving such inverse problems for specific sizes. One should, therefore, find multiple microarchitectural designs that exhibit the desired properties for a specimen with given dimensions. Moreover, the candidate microarchitectures should be resistant to fatigue and fracture, meaning that peak stresses should be minimized as well. Such a multi-objective inverse design problem is formidably difficult to solve but its solution is the key to real-world applications of mechanical metamaterials. Here, we propose a modular approach titled 'Deep-DRAM' that combines four decoupled models, including two deep learning models (DLM), a deep generative model (DGM) based on conditional variational autoencoders (CVAE), and direct finite element (FE) simulations. Deep-DRAM (deep learning for the design of random-network metamaterials) integrates these models into a unified framework capable of finding many solutions to the multi-objective inverse design problem posed here. The integrated framework first introduces the desired elastic properties to the DGM, which returns a set of candidate designs. The candidate designs, together with the target specimen dimensions are then passed to the DLM which predicts their actual elastic properties considering the specimen size. After a filtering step based on the closeness of the actual properties to the desired ones, the last step uses direct FE simulations to identify the designs with the minimum peak stresses.

In recent years, deep learning has infiltrated every field it has touched, reducing the need for specialist knowledge and automating the process of knowledge discovery from data. This review argues that astronomy is no different, and that we are currently in the midst of a deep learning revolution that is transforming the way we do astronomy. We trace the history of astronomical connectionism from the early days of multilayer perceptrons, through the second wave of convolutional and recurrent neural networks, to the current third wave of self-supervised and unsupervised deep learning. We then predict that we will soon enter a fourth wave of astronomical connectionism, in which finetuned versions of an all-encompassing 'foundation' model will replace expertly crafted deep learning models. We argue that such a model can only be brought about through a symbiotic relationship between astronomy and connectionism, whereby astronomy provides high quality multimodal data to train the foundation model, and in turn the foundation model is used to advance astronomical research.

作为自然现象的地震，历史上不断造成伤害和人类生活的损失。地震预测是任何社会计划的重要方面，可以增加公共准备，并在很大程度上减少损坏。然而，由于地震的随机特征以及实现了地震预测的有效和可靠模型的挑战，迄今为止努力一直不足，需要新的方法来解决这个问题。本文意识到​​这些问题，提出了一种基于注意机制（AM），卷积神经网络（CNN）和双向长短期存储器（BILSTM）模型的新型预测方法，其可以预测数量和最大幅度中国大陆各地区的地震为基于该地区的地震目录。该模型利用LSTM和CNN具有注意机制，以更好地关注有效的地震特性并产生更准确的预测。首先，将零阶保持技术应用于地震数据上的预处理，使得模型的输入数据更适当。其次，为了有效地使用空间信息并减少输入数据的维度，CNN用于捕获地震数据之间的空间依赖性。第三，使用Bi-LSTM层来捕获时间依赖性。第四，引入了AM层以突出其重要的特征来实现更好的预测性能。结果表明，该方法具有比其他预测方法更好的性能和概括能力。

X-ray imaging technology has been used for decades in clinical tasks to reveal the internal condition of different organs, and in recent years, it has become more common in other areas such as industry, security, and geography. The recent development of computer vision and machine learning techniques has also made it easier to automatically process X-ray images and several machine learning-based object (anomaly) detection, classification, and segmentation methods have been recently employed in X-ray image analysis. Due to the high potential of deep learning in related image processing applications, it has been used in most of the studies. This survey reviews the recent research on using computer vision and machine learning for X-ray analysis in industrial production and security applications and covers the applications, techniques, evaluation metrics, datasets, and performance comparison of those techniques on publicly available datasets. We also highlight some drawbacks in the published research and give recommendations for future research in computer vision-based X-ray analysis.

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-ofthe-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.

我们开发了卷积神经网络（CNNS），快速，直接从无线电尘埃连续图像中推断出行星质量。在原始板块中的年轻行星引起的子结构可用于推断潜在的年轻行星属性。流体动力模拟已被用于研究地球属性与这些磁盘特征之间的关系。然而，这些尝试了微调的数值模拟，以一次适合一个原始磁盘，这是耗时的，或者四方平均模拟结果，以导出间隙宽度/深度和行星质量之间的一些线性关系，这丢失了信息磁盘中的不对称功能。为了应对这些缺点，我们开发了行星间隙神经网络（PGNET），以推断出2D图像的行星质量。我们首先符合张等人的网格数据。 （2018）作为分类问题。然后，通过使用近随机采样参数运行额外的模拟来分布数据集，并将行星质量和磁盘粘度一起作为回归问题衍生在一起。分类方法可以达到92 \％的准确性，而回归方法可以达到1 $ \ Sigma $ AS 0.16 DEX，用于行星质量和0.23°D磁盘粘度。我们可以在线性拟合方法中重现退化缩放$ \ alpha $ $ \ propto $ $ m_p ^ 3 $。这意味着CNN方法甚至可以用于寻找退化关系。梯度加权类激活映射有效地确认PGNETS使用适当的磁盘特征来限制行星质量。我们为张等人提供了PGNETS和传统配件方法的计划。 （2018），并讨论各种方法的优缺点。

现代时间域的光度测验收集了许多天文学对象的观察结果，大规模调查的即将到来的时代将提供更多信息。大多数对象从未接受过光谱随访，这对于瞬态尤其至关重要。超新星。在这种情况下，观察到的光曲线可以提供负担得起的替代方案。时间序列被积极用于光度分类和表征，例如峰值和光度下降估计。但是，收集的时间序列是多维的，不规则地采样，包含异常值，并且没有明确定义的系统不确定性。机器学习方法有助于以最有效的方式从可用数据中提取有用的信息。我们考虑了基于神经网络的几种光曲线近似方法：多层感知，贝叶斯神经网络以及使流量正常化，以近似单光曲线观察。使用模拟的Parperc和Real Zwicky瞬态设施数据样本的测试表明，即使很少有观察值足以拟合网络并获得比其他最新方法更好的近似质量。我们表明，这项工作中描述的方法具有比高斯流程更快的计算复杂性和更快的工作速度。我们分析了旨在填补光曲线观察中空白的近似技术的性能，并表明使用适当的技术会提高峰值发现和超新星分类的准确性。此外，研究结果是在GitHub上可用的Fulu Python库中组织的，该库可以很容易地由社区使用。

冠状病毒疾病或Covid-19是由SARS-COV-2病毒引起的一种传染病。该病毒引起的第一个确认病例是在2019年12月底在中国武汉市发现的。然后，此案遍布全球，包括印度尼西亚。因此，联合19案被WHO指定为全球大流行。可以使用多种方法（例如深神经网络（DNN））预测COVID-19病例的增长，尤其是在印度尼西亚。可以使用的DNN模型之一是可以预测时间序列的深变压器。该模型经过多种测试方案的培训，以获取最佳模型。评估是找到最佳的超参数。然后，使用预测天数，优化器，功能数量以及与长期短期记忆（LSTM）（LSTM）和复发性神经网络（RNN）的先前模型进行比较的最佳超参数设置进行了进一步的评估。 。所有评估均使用平均绝对百分比误差（MAPE）的度量。基于评估的结果，深层变压器在使用前层归一化时会产生最佳的结果，并预测有一天的MAPE值为18.83。此外，接受Adamax优化器训练的模型在其他测试优化器中获得了最佳性能。 Deep Transformer的性能还超过了其他测试模型，即LSTM和RNN。

Over the years, Machine Learning models have been successfully employed on neuroimaging data for accurately predicting brain age. Deviations from the healthy brain aging pattern are associated to the accelerated brain aging and brain abnormalities. Hence, efficient and accurate diagnosis techniques are required for eliciting accurate brain age estimations. Several contributions have been reported in the past for this purpose, resorting to different data-driven modeling methods. Recently, deep neural networks (also referred to as deep learning) have become prevalent in manifold neuroimaging studies, including brain age estimation. In this review, we offer a comprehensive analysis of the literature related to the adoption of deep learning for brain age estimation with neuroimaging data. We detail and analyze different deep learning architectures used for this application, pausing at research works published to date quantitatively exploring their application. We also examine different brain age estimation frameworks, comparatively exposing their advantages and weaknesses. Finally, the review concludes with an outlook towards future directions that should be followed by prospective studies. The ultimate goal of this paper is to establish a common and informed reference for newcomers and experienced researchers willing to approach brain age estimation by using deep learning models

根据诊断各种疾病的胸部X射线图像的可观增长，以及收集广泛的数据集，使用深神经网络进行了自动诊断程序，已经占据了专家的思想。计算机视觉中的大多数可用方法都使用CNN主链来获得分类问题的高精度。然而，最近的研究表明，在NLP中成为事实上方法的变压器也可以优于许多基于CNN的模型。本文提出了一个基于SWIN变压器的多标签分类深模型，作为实现最新诊断分类的骨干。它利用了头部体系结构来利用多层感知器（也称为MLP）。我们评估了我们的模型，该模型称为“ Chest X-Ray14”，最广泛，最大的X射线数据集之一，该数据集由30,000多名14例著名胸部疾病的患者组成100,000多个额叶/背景图像。我们的模型已经用几个数量的MLP层用于头部设置，每个模型都在所有类别上都达到了竞争性的AUC分数。胸部X射线14的全面实验表明，与以前的SOTA平均AUC为0.799相比，三层头的平均AUC得分为0.810，其平均AUC得分为0.810。我们建议对现有方法进行公平基准测试的实验设置，该设置可以用作未来研究的基础。最后，我们通过确认所提出的方法参与胸部的病理相关区域，从而跟进了结果。

通过卫星摄像机获取关于地球表面的大面积的信息使我们能够看到远远超过我们在地面上看到的更多。这有助于我们在检测和监测土地使用模式，大气条件，森林覆盖和许多非上市方面的地区的物理特征。所获得的图像不仅跟踪连续的自然现象，而且对解决严重森林砍伐的全球挑战也至关重要。其中亚马逊盆地每年占最大份额。适当的数据分析将有助于利用可持续健康的氛围来限制对生态系统和生物多样性的不利影响。本报告旨在通过不同的机器学习和优越的深度学习模型用大气和各种陆地覆盖或土地使用亚马逊雨林的卫星图像芯片。评估是基于F2度量完成的，而用于损耗函数，我们都有S形跨熵以及Softmax交叉熵。在使用预先训练的ImageNet架构中仅提取功能之后，图像被间接馈送到机器学习分类器。鉴于深度学习模型，通过传输学习使用微调Imagenet预训练模型的集合。到目前为止，我们的最佳分数与F2度量为0.927。

Pneumonia, a respiratory infection brought on by bacteria or viruses, affects a large number of people, especially in developing and impoverished countries where high levels of pollution, unclean living conditions, and overcrowding are frequently observed, along with insufficient medical infrastructure. Pleural effusion, a condition in which fluids fill the lung and complicate breathing, is brought on by pneumonia. Early detection of pneumonia is essential for ensuring curative care and boosting survival rates. The approach most usually used to diagnose pneumonia is chest X-ray imaging. The purpose of this work is to develop a method for the automatic diagnosis of bacterial and viral pneumonia in digital x-ray pictures. This article first presents the authors' technique, and then gives a comprehensive report on recent developments in the field of reliable diagnosis of pneumonia. In this study, here tuned a state-of-the-art deep convolutional neural network to classify plant diseases based on images and tested its performance. Deep learning architecture is compared empirically. VGG19, ResNet with 152v2, Resnext101, Seresnet152, Mobilenettv2, and DenseNet with 201 layers are among the architectures tested. Experiment data consists of two groups, sick and healthy X-ray pictures. To take appropriate action against plant diseases as soon as possible, rapid disease identification models are preferred. DenseNet201 has shown no overfitting or performance degradation in our experiments, and its accuracy tends to increase as the number of epochs increases. Further, DenseNet201 achieves state-of-the-art performance with a significantly a smaller number of parameters and within a reasonable computing time. This architecture outperforms the competition in terms of testing accuracy, scoring 95%. Each architecture was trained using Keras, using Theano as the backend.

我们提出了TABPFN，这是一种与小型表格数据集上的最新技术竞争性的自动化方法，而更快的速度超过1,000美元。我们的方法非常简单：它完全符合单个神经网络的权重，而单个正向通行证直接产生了对新数据集的预测。我们的AutoML方法是使用基于变压器的先验数据拟合网络（PFN）体系结构进行元学习的，并近似贝叶斯推断，其先验是基于简单性和因果结构的假设。先验包含庞大的结构性因果模型和贝叶斯神经网络，其偏见是小体系结构，因此复杂性较低。此外，我们扩展了PFN方法以在实际数据上校准Prior的超参数。通过这样做，我们将抽象先前的假设与对真实数据的启发式校准分开。之后，修复了校准的超参数，并在按钮按钮时可以将TABPFN应用于任何新的表格数据集。最后，在OpenML-CC18套件的30个数据集上，我们表明我们的方法优于树木，并与复杂的最新Automl系统相同，并且在不到一秒钟内产生的预测。我们在补充材料中提供所有代码和最终训练的TABPFN。

流感病毒迅速变异，可能对公共卫生构成威胁，尤其是对弱势群体的人。在整个历史中，流感A病毒在不同物种之间引起了大流行病。重要的是要识别病毒的起源，以防止爆发的传播。最近，人们对使用机器学习算法来为病毒序列提供快速准确的预测一直引起人们的兴趣。在这项研究中，使用真实的测试数据集和各种评估指标用于评估不同分类学水平的机器学习算法。由于血凝素是免疫反应中的主要蛋白质，因此仅使用血凝素序列并由位置特异性评分基质和单词嵌入来表示。结果表明，5-grams-transformer神经网络是预测病毒序列起源的最有效算法，大约99.54％的AUCPR，98.01％的F1分数和96.60％的MCC，在较高的分类水平上，约94.74％AUCPR，87.41％，87.41％，87.41％ ％F1分数％和80.79％的MCC在较低的分类水平下。