Graph Neural Network (GNN)

An emerging area where Neural Networks may be able to make a large impact for society is in the area around graph prediction and classification problems. There are indeed a wide array of problems uncovered by Graph Neural Networks.

Graph (KG) vs GNN

GNNs operate on Graphs

Graph

Graph databases insert, update or retrieve nodes and edges given constraints or queries.

GNN

  • Graph Neural Networks (GNNs) are a class of Neural Networks designed to perform inference on Graph related data.

  • Produce excellent results in node and graph classification.

  • Can be directly applied to graphs

  • Provide an easy way to do node-level, edge-level, and graph-level prediction tasks.

  • Have been shown to achieve state-of-the-art results on both node and graph classification tasks.

  • Enabled a new field of study in graph representation learning.

GNN construction techniques

  • There are several different GNN variants proposed in papers

  • All follow a neighborhood aggregation scheme

  • Representation vector of a node are computed by recursively aggregating and transforming representation vectors of its neighboring nodes.

  • The representation of an entire graph can then be obtained through pooling

Chosing the best GNN

GNN models have unique strengths and are highly dependent on the use case.

Some examples are below

GCN

An early foundational model specially designed for semi-supervised learning on graphs, where the target is to predict the labels for nodes. It is the simplest GNN architecture for learning graph representations.

GAT

Convolution type models utilize a masked self-attentional mechanism as it selectively focuses on different parts of their neighborhoods. They are shown to be very effective at predictions involving networks of scientific citations or documents

ST-GAT

(Spatio-Temporal Graph ATtention) network is a model designed to predict future traffic speeds

CAL

(Causal Attention Learning) models are shown to be effective at predicting cause and effect relationships.

References

  1. Junho Song, Jiwon Son, Dong-hyuk Seo, Kyungsik Han, Namhyuk Kim, Sang-Wook. ST-GAT: A Spatio-Temporal Graph Attention Network for Accurate Traffic Speed Prediction

  2. Simi Job, Xiaohui Tao, Senior Member, Taotao Cai, Lin Li, Haoran Xie, Senior Member, Jianming Yong. Towards Causal Classification: A Comprehensive Study on Graph Neural Networks

  3. S. Ratna a, Sukhdeep Singh b, Anuj Sharma. An inclusive analysis for performance and efficiency of graph neural network models for node classification

  4. Petar Velickovi ˇ c´, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio GRAPH ATTENTION NETWORKS

  5. Thomas N. Kipf, Max Welling. SEMI-SUPERVISED CLASSIFICATION WITH GRAPH CONVOLUTIONAL NETWORKS