可以将一组个人或组织之间的战略互动建模为在网络上玩的游戏，在网络上，玩家的回报不仅取决于他们的行动，还取决于邻居的行动。从观察到的游戏结果（平衡动作）中推断网络结构是一个重要的问题，对于经济学和社会科学中的许多潜在应用。现有方法主要需要与游戏相关的效用函数的知识，在现实世界中，这通常是不现实的。我们采用类似变压器的体系结构，该体系结构正确说明了问题的对称性，并在没有明确了解效用功能的情况下学习了从平衡动作到游戏网络结构的映射。我们使用合成和现实世界数据在三种不同类型的网络游戏上测试我们的方法，并证明其在网络结构推理中的有效性和优于现有方法的卓越性能。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

Research in Graph Signal Processing (GSP) aims to develop tools for processing data defined on irregular graph domains. In this paper we first provide an overview of core ideas in GSP and their connection to conventional digital signal processing, along with a brief historical perspective to highlight how concepts recently developed in GSP build on top of prior research in other areas. We then summarize recent advances in developing basic GSP tools, including methods for sampling, filtering or graph learning. Next, we review progress in several application areas using GSP, including processing and analysis of sensor network data, biological data, and applications to image processing and machine learning.

Graph AutoCododers（GAE）和变分图自动编码器（VGAE）作为链接预测的强大方法出现。他们的表现对社区探测问题的印象不那么令人印象深刻，根据最近和同意的实验评估，它们的表现通常超过了诸如louvain方法之类的简单替代方案。目前尚不清楚可以通过GAE和VGAE改善社区检测的程度，尤其是在没有节点功能的情况下。此外，不确定是否可以在链接预测上同时保留良好的性能。在本文中，我们表明，可以高精度地共同解决这两个任务。为此，我们介绍和理论上研究了一个社区保留的消息传递方案，通过在计算嵌入空间时考虑初始图形结构和基于模块化的先验社区来掺杂我们的GAE和VGAE编码器。我们还提出了新颖的培训和优化策略，包括引入一个模块化的正规器，以补充联合链路预测和社区检测的现有重建损失。我们通过对各种现实世界图的深入实验验证，证明了方法的经验有效性，称为模块化感知的GAE和VGAE。

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

网络完成是一个比链接预测更难的问题，因为它不仅尝试推断丢失的链接，还要推断节点。已经提出了不同的方法来解决此问题，但是很少有人使用结构信息 - 局部连接模式的相似性。在本文中，我们提出了一个名为C-GIN的模型，以根据图形自动编码器框架从网络的观察到的部分捕获局部结构模式，该框架配备了图形同构网络模型，并将这些模式推广到完成整个图形。对来自不同领域的合成和现实世界网络的实验和分析表明，C-Gin可以实现竞争性能，而所需的信息较少，并且在大多数情况下，与基线预测模型相比，可以获得更高的准确性。我们进一步提出了一个基于网络结构的“可达聚类系数（CC）”。实验表明，我们的模型在具有较高可及的CC的网络上表现更好。

Clustering is a fundamental problem in network analysis that finds closely connected groups of nodes and separates them from other nodes in the graph, while link prediction is to predict whether two nodes in a network are likely to have a link. The definition of both naturally determines that clustering must play a positive role in obtaining accurate link prediction tasks. Yet researchers have long ignored or used inappropriate ways to undermine this positive relationship. In this article, We construct a simple but efficient clustering-driven link prediction framework(ClusterLP), with the goal of directly exploiting the cluster structures to obtain connections between nodes as accurately as possible in both undirected graphs and directed graphs. Specifically, we propose that it is easier to establish links between nodes with similar representation vectors and cluster tendencies in undirected graphs, while nodes in a directed graphs can more easily point to nodes similar to their representation vectors and have greater influence in their own cluster. We customized the implementation of ClusterLP for undirected and directed graphs, respectively, and the experimental results using multiple real-world networks on the link prediction task showed that our models is highly competitive with existing baseline models. The code implementation of ClusterLP and baselines we use are available at https://github.com/ZINUX1998/ClusterLP.

我们考虑了从节点观测值估算多个网络拓扑的问题，其中假定这些网络是从相同（未知）随机图模型中绘制的。我们采用图形作为我们的随机图模型，这是一个非参数模型，可以从中绘制出潜在不同大小的图形。图形子的多功能性使我们能够解决关节推理问题，即使对于要恢复的图形包含不同数量的节点并且缺乏整个图形的精确比对的情况。我们的解决方案是基于将最大似然惩罚与Graphon估计方案结合在一起，可用于增强现有网络推理方法。通过引入嘈杂图抽样信息的强大方法，进一步增强了所提出的联合网络和图形估计。我们通过将其性能与合成和实际数据集中的竞争方法进行比较来验证我们提出的方法。

最新提出的基于变压器的图形模型的作品证明了香草变压器用于图形表示学习的不足。要了解这种不足，需要研究变压器的光谱分析是否会揭示其对其表现力的见解。类似的研究已经确定，图神经网络（GNN）的光谱分析为其表现力提供了额外的观点。在这项工作中，我们系统地研究并建立了变压器领域中的空间和光谱域之间的联系。我们进一步提供了理论分析，并证明了变压器中的空间注意机制无法有效捕获所需的频率响应，因此，固有地限制了其在光谱空间中的表现力。因此，我们提出了feta，该框架旨在在整个图形频谱（即图形的实际频率成分）上进行注意力类似于空间空间中的注意力。经验结果表明，FETA在标准基准的所有任务中为香草变压器提供均匀的性能增益，并且可以轻松地扩展到具有低通特性的基于GNN的模型（例如GAT）。

在现实世界中，签名的定向网络无处不在。但是，对于分析此类网络的方法，较少的工作提出了频谱图神经网络（GNN）方法。在这里，我们介绍了一个签名的定向拉普拉斯矩阵，我们称之为磁性签名的laplacian，作为在签名的图表上签名的laplacian的自然概括，在有向图上的磁Laplacian。然后，我们使用此矩阵来构建一种新型的光谱GNN结构，并在节点聚类和链接预测任务上进行广泛的实验。在这些实验中，我们考虑了与签名信息有关的任务，与定向信息相关的任务以及与签名和定向信息有关的任务。我们证明，我们提出的光谱GNN有效地合并了签名和定向信息，并在广泛的数据集中获得领先的性能。此外，我们提供了一种新颖的合成网络模型，我们称之为签名的定向随机块模型，以及许多基于财务时间序列中铅滞后关系的新型现实世界数据集。

图形神经网络（GNN）在许多基于图的学​​习任务中表现出很大的优势，但通常无法准确预测基于任务的节点集，例如链接/主题预测等。最近，许多作品通过使用随机节点功能或节点距离特征来解决此问题。但是，它们的收敛速度缓慢，预测不准确或高复杂性。在这项工作中，我们重新访问允许使用位置编码（PE）技术（例如Laplacian eigenmap，deepwalk等）的节点的位置特征。 。在这里，我们以原则性的方式研究了这些问题，并提出了一种可证明的解决方案，这是一类用严格数学分析的钉子的GNN层。 PEG使用单独的频道来更新原始节点功能和位置功能。 PEG施加置换量比W.R.T.原始节点功能并施加$ O（P）$（正交组）均值W.R.T.位置特征同时特征，其中$ p $是二手位置特征的维度。在8个现实世界网络上进行的广泛链接预测实验证明了PEG在概括和可伸缩性方面的优势。

Estimating the structure of directed acyclic graphs (DAGs) of features (variables) plays a vital role in revealing the latent data generation process and providing causal insights in various applications. Although there have been many studies on structure learning with various types of data, the structure learning on the dynamic graph has not been explored yet, and thus we study the learning problem of node feature generation mechanism on such ubiquitous dynamic graph data. In a dynamic graph, we propose to simultaneously estimate contemporaneous relationships and time-lagged interaction relationships between the node features. These two kinds of relationships form a DAG, which could effectively characterize the feature generation process in a concise way. To learn such a DAG, we cast the learning problem as a continuous score-based optimization problem, which consists of a differentiable score function to measure the validity of the learned DAGs and a smooth acyclicity constraint to ensure the acyclicity of the learned DAGs. These two components are translated into an unconstraint augmented Lagrangian objective which could be minimized by mature continuous optimization techniques. The resulting algorithm, named GraphNOTEARS, outperforms baselines on simulated data across a wide range of settings that may encounter in real-world applications. We also apply the proposed approach on two dynamic graphs constructed from the real-world Yelp dataset, demonstrating our method could learn the connections between node features, which conforms with the domain knowledge.

图表神经网络（GNNS）最近在人工智能（AI）领域的普及，这是由于它们作为输入数据相对非结构化数据类型的独特能力。尽管GNN架构的一些元素在概念上类似于传统神经网络（以及神经网络变体）的操作中，但是其他元件代表了传统深度学习技术的偏离。本教程通过整理和呈现有关GNN最常见和性能变种的动机，概念，数学和应用的细节，将GNN的权力和新颖性暴露给AI从业者。重要的是，我们简明扼要地向实际示例提出了本教程，从而为GNN的主题提供了实用和可访问的教程。

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them.Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field.

We investigate the representation power of graph neural networks in the semisupervised node classification task under heterophily or low homophily, i.e., in networks where connected nodes may have different class labels and dissimilar features. Many popular GNNs fail to generalize to this setting, and are even outperformed by models that ignore the graph structure (e.g., multilayer perceptrons). Motivated by this limitation, we identify a set of key designs-ego-and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations-that boost learning from the graph structure under heterophily. We combine them into a graph neural network, H 2 GCN, which we use as the base method to empirically evaluate the effectiveness of the identified designs. Going beyond the traditional benchmarks with strong homophily, our empirical analysis shows that the identified designs increase the accuracy of GNNs by up to 40% and 27% over models without them on synthetic and real networks with heterophily, respectively, and yield competitive performance under homophily.

最近，在对图形结构数据上应用深度神经网络有很大的成功。然而，大多数工作侧重于节点或图形级监督学习，例如节点，链接或图形分类或节点级无监督学习（例如节点群集）。尽管其应用广泛，但图表级无监督的学习尚未受到很多关注。这可能主要归因于图形的高表示复杂性，可以由n表示！等效邻接矩阵，其中n是节点的数量。在这项工作中，我们通过提出用于图形结构数据的置换不变变化自动码器来解决此问题。我们所提出的模型间接学习以匹配输入和输出图的节点排序，而不施加特定节点排序或执行昂贵的图形匹配。我们展示了我们提出模型对各种图形重建和生成任务的有效性，并评估了下游图形水平分类和回归提取的表示的表现力。

图形神经网络（GNNS）通过考虑其内在的几何形状来扩展神经网络的成功到图形结构化数据。尽管根据图表学习基准的集合，已经对开发具有卓越性能的GNN模型进行了广泛的研究，但目前尚不清楚其探测给定模型的哪些方面。例如，他们在多大程度上测试模型利用图形结构与节点特征的能力？在这里，我们开发了一种原则性的方法来根据$ \ textit {敏感性配置文件} $进行基准测试数据集，该方法基于由于图形扰动的集合而导致的GNN性能变化了多少。我们的数据驱动分析提供了对GNN利用哪些基准测试数据特性的更深入的了解。因此，我们的分类法可以帮助选择和开发适当的图基准测试，并更好地评估未来的GNN方法。最后，我们在$ \ texttt {gtaxogym} $软件包中的方法和实现可扩展到多个图形预测任务类型和未来数据集。

我们从光谱的角度解决图形生成问题，首先生成图形laplacian光谱的主要部分，然后构建与这些特征值和特征向量相匹配的图。光谱调节允许直接建模全局和局部图结构，并有助于克服单发图生成器的表达性和模式崩溃问题。我们的新颖的甘（Spectre）称为Spectre，可以使用一声模型来产生比以前可能更大的图。Spectre的表现优于最先进的深度自动回归发电机在建模忠诚方面，同时还避免了昂贵的顺序产生和对节点排序的依赖。一个很好的例子，在相当大的合成和现实图形中，Specter的幽灵比最佳竞争对手的最佳竞争对手的改进是4到170倍，该竞争对手不合适，比自回旋发电机快23至30倍。