book

Graph Neural Networks in Action

by Namid Stillman, Keita Broadwater

February 2025

Intermediate to advanced

392 pages

12h 9m

English

Manning Publications

Read now

Unlock full access

1.1 Goals of this book1.1.1 Catching up on graph fundamentals1.2 Graph-based learning1.2.1 What are graphs?1.2.2 Different types of graphs1.2.3 Graph-based learning1.2.4 What is a GNN?1.2.5 Differences between tabular and graph data1.3 GNN applications: Case studies1.3.1 Recommendation engines1.3.2 Drug discovery and molecular science1.3.3 Mechanical reasoning1.4 When to use a GNN?1.4.1 Implicit relationships and interdependencies1.4.2 High dimensionality and sparsity1.4.3 Complex, nonlocal interactions1.5 Understanding how GNNs operate1.5.1 Mental model for training a GNN1.5.2 Unique mechanisms of a GNN model1.5.3 Message passing
2.1 Creating embeddings with Node2Vec2.1.1 Loading data, setting parameters, and creating embeddings2.1.2 Demystifying embeddings2.1.3 Transforming and visualizing the embeddings2.1.4 Beyond visualization: Applications and considerations of N2V embeddings2.2 Creating embeddings with a GNN2.2.1 Constructing the embeddings2.2.2 GNN vs. N2V embeddings2.3 Using node embeddings2.3.1 Data preprocessing2.3.2 Random forest classification2.3.3 Embeddings in an end-to-end model2.4 Under the Hood2.4.1 Representations and embeddings2.4.2 Transductive and inductive methods2.4.3 N2V: Random walks across graphs2.4.4 Message passing as deep learning
3.1 Predicting consumer product categories3.1.1 Loading and processing the data3.1.2 Creating our model classes3.1.3 Model training3.1.4 Model performance analysis3.1.5 Our first product bundle3.2 Aggregation methods3.2.1 Neighborhood aggregation3.2.2 Advanced aggregation tools3.2.3 Practical considerations in applying aggregation3.3 Further optimizations and refinements3.3.1 Dropout3.3.2 Model depth3.3.3 Improving the baseline model’s performance3.3.4 Revisiting the Marcelina product bundle3.4 Under the hood3.4.1 Convolution methods3.4.2 Message passing3.4.3 GCN aggregation function3.4.4 GCN in PyTorch Geometric3.4.5 Spectral vs. spatial convolution3.4.6 GraphSAGE aggregation function3.4.7 GraphSAGE in PyTorch Geometric3.5 Amazon Products dataset
4.1 Detecting spam and fraudulent reviews4.2 Exploring the review spam dataset4.2.1 Explaining the node features4.2.2 Exploratory data analysis4.2.3 Exploring the graph structure4.2.4 Exploring the node features4.3 Training baseline models4.3.1 Non-GNN baselines4.3.2 GCN baseline4.4 Training GAT models4.4.1 Neighborhood loader and GAT models4.4.2 Addressing class imbalance in model performance4.4.3 Deciding between GAT and XGBoost4.5 Under the hood4.5.1 Explaining attention and GAT models4.5.2 Over-smoothing4.5.3 Overview of key GAT equations
5.1 Generative models: Learning how to generate5.1.1 Generative and discriminative models5.1.2 Synthetic data5.2 Graph autoencoders for link prediction5.2.1 Review of the Amazon Products dataset from chapter 35.2.2 Defining a graph autoencoder5.2.3 Training a graph autoencoder to perform link prediction5.3 Variational graph autoencoders5.3.1 Building a variational graph autoencoder5.3.2 When to use a variational graph autoencoder5.4 Generating graphs using GNNs5.4.1 Molecular graphs5.4.2 Identifying new drug candidates5.4.3 VGAEs for generating graphs5.4.4 Generating molecules using a GNN5.5 Under the hood5.5.1 Understanding link prediction tasks5.5.2 The inner product decoder
6.1 Temporal models: Relations through time6.2 Problem definition: Pose estimation6.2.1 Setting up the problem6.2.2 Building models with memory6.3 Dynamic graph neural networks6.3.1 Graph attention network for dynamic graphs6.4 Neural relational inference6.4.1 Encoding pose data6.4.2 Decoding pose data using a GRU6.4.3 Training the NRI model6.5 Under the hood6.5.1 Recurrent neural networks6.5.2 Temporal adjacency matrices6.5.3 Combining autoencoders with RNNs6.5.4 Gumbel-Softmax
7.1 Examples in this chapter7.1.1 Amazon Products dataset7.1.2 GeoGrid7.2 Framing problems of scale7.2.1 Root causes7.2.2 Symptoms7.2.3 Crucial metrics7.3 Techniques for tackling problems of scale7.3.1 Seven techniques7.3.2 General Steps7.4 Choice of hardware configuration7.4.1 Types of hardware choices7.4.2 Choice of processor and memory size7.5 Choice of data representation7.6 Choice of GNN algorithm7.6.1 Time and space complexity7.7 Batching using a sampling method7.7.1 Two concepts: Mini-batching and sampling7.7.2 A glance at notable PyG samplers7.8 Parallel and distributed processing7.8.1 Using distributed data parallel7.8.2 Code example for DDP7.9 Training with remote storage7.9.1 Example7.10 Graph coarsening7.10.1 Example
8.1 Data preparation and project planning8.1.1 Project definition8.1.2 Project objectives and scope8.2 Designing graph models8.2.1 Get familiar with the domain and use case8.2.2 Constructing the graph dataset and schemas8.2.3 Creating instance models8.2.4 Testing and refactoring8.3 Data pipeline example8.3.1 Raw data8.3.2 The ETL step8.3.3 Data exploration and visualization8.3.4 Preprocessing and loading data into PyG8.4 Where to find graph data
A.1 Graph fundamentalsA.1.1 Graph propertiesA.1.2 Characteristics of nodes and edgesA.1.3 Categories of graphsA.2 Graph representationsA.2.1 Basic graph data structuresA.2.2 Relational databasesA.2.3 How graphs are exposedA.3 Graph systemsA.3.1 Graph databasesA.3.2 Graph compute engines (or graph frameworks)A.3.3 Visualization librariesA.3.4 GNN librariesA.4 Graph algorithmsA.4.1 Traversal and search algorithmsA.4.2 Shortest pathA.5 How to read GNN literatureA.5.1 Common graph notations
B.1 Installing PyTorch GeometricB.1.1 On Windows/LinuxB.1.2 On MacOSB.1.3 Compatibility issues

Content preview from Graph Neural Networks in Action

7 Learning and inference at scale

This chapter covers

Strategies for handling data overload in small systems
Recognizing graph neural network problems that require scaled resources
Seven robust techniques for mitigating problems arising from large data
Scaling graph neural networks and tackling scalability challenges with PyTorch Geometric

For most of our journey through graph neural networks (GNNs), we’ve explained key architectures and methods, but we’ve limited examples to problems of relatively small scale. Our reason for doing so was to allow you to access example code and data readily.

However, real-world problems in deep learning are not often so neatly packaged. One of the major challenges in real-world scenarios is training ...