Course: Graph Machine Learning

The highest activity a human being can attain is learning for understanding, <br> because to understand is to be free. Baruch Spinoza

|<b>Lecturer</b> | | |:-:|:-:| | <img src="https://github.com/user-attachments/assets/781961f7-0ca6-4370-9b73-281db5995ec4" width=170pt > <br> <b>Zahra Taheri</b> | Graduate Course <br> <br> Data Science Center <br> <br> Shahid Beheshti University <br> <br> <b> Winter 2023</b> |

:bulb: Course Overview

<div align="justify">

Graph Machine Learning is a course that focuses on the application of machine learning algorithms on graph-structured data. Some of the key topics that are covered in the course include graph representation learning and graph neural networks, algorithms for the world wide web, reasoning over knowledge graphs, and social network analysis. The course is designed for graduate students with a background in machine learning and/or data science who want to expand their skills to work with graph data. The course may also be useful for students and professionals working in fields such as computer science, biology, chemistry, and physics that require the analysis of graph-structured data. The objective of the course is to provide students with a comprehensive understanding of graph machine learning and its various applications, challenges, and opportunities, as well as hands-on experience in implementing these algorithms.

</div>

:beginner: Prerequisites

• Familiarity with the basic probability theory, and the basic linear algebra
• Basic knowledge of machine learning and/or deep learning concepts
• Familiarity with the basics of Python programming language
• Familiarity with PyTorch is a plus

:books: Recommended Materials

Books

• Graph Representation Learning by William L. Hamilton
• Network Science by Albert-László Barabási
• Networks, Crowds, and Markets: Reasoning About a Highly Connected World by David Easley and Jon Kleinberg
• Analysis of Biological Networks, 2007.

Graph Machine Learning Tools

• DGL (Deep Graph Library)
• PyG (PyTorch Geometric)
• NetworkX

Courses

• CS224W: Machine Learning with Graphs by Jure Leskovec

Tips and Tools for Data Science

• Fundamental and Useful Tools and Tips for Data Science
• Essential Steps to Set Up Your PC for Graph Machine Learning with PyG

:closed_book: Other Materials

Books

• Deep Learning on Graphs by Yao Ma and Jiliang Tang
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković

:book: Contents

The contents and materials related to the course will be posted here.

1. Introduction to Graph Machine Learning

Required Reading:

• Slide: Introduction; Machine Learning for Graphs by Jure Leskovec

Suggested Reading:

• Papers: To fully understand the following papers, you should be familiar with graph neural networks.
  • Node-Level Graph ML Task:
    • Alphafold2: Highly accurate protein structure prediction with AlphaFold by Jumper, et.al., Nature 2021.
  • Edge-Level Graph ML Task:
    • Graph Convolutional Neural Networks for Web-Scale Recommender Systems by Ying et.al., KDD 2018.
    • Modeling Polypharmacy Side Effects with Graph Convolutional Networks by Zitnik et.al., Bioinformatics 2018.
  • Subgraph-Level Graph ML Task:
    • ETA Prediction with Graph Neural Networks in Google Maps by Derrow-Pinion et.al., CIKM 2021.
  • Graph-Level Graph ML Task:
    • A Deep Learning Approach to Antibiotic Discovery by Stokes et.al., Cell 2020.
    • Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation by You et.al., NeurIPS 2018.
    • Learning to simulate complex physics with graph networks by Sanchez-Gonzalez et al., ICML 2020.

2. Traditional Methods for Machine Learning on Graphs

Required Reading:

• Slide: Traditional Methods for ML on Graphs by Jure Leskovec

Suggested Reading:

• Blog:

  • Link Prediction in Large-Scale Networks by Cdiscount Data Science
  • Expressive power of graph neural networks and the Weisfeiler-Lehman test by Michael Bronstein

• Slide:

  • Introduction to Kernel Methods
  • Elements of Positive Definite Kernel and Reproducing Kernel Hilbert Space

• Paper:

  • Graph Kernels by Vishwanathan et al., JMLR 2010.
  • Efficient graphlet kernels for large graph comparison by Shervashidze et al., JMLR 2009.
  • Weisfeiler-Lehman Graph Kernels by Shervashidze et al., JMLR 2011.

• Question: How does the dot product determine similarity?

3. Node Embeddings

Required Reading:

• Slide: Node Embeddings by Jure Leskovec
• Example of node2vec: Detailed Example and Implementation by Zahra Taheri

Suggested Reading:

• Blog:
  • Complete guide to understanding Node2Vec algorithm by Tomaz Bratanic
  • Node2Vec Explained by Vatsal
  • node2vec: Embeddings for Graph Data by Elior Cohen
  • Node2vec explained graphically by Remy Lau
  • Word2Vec Tutorial - The Skip-Gram Model by Chris McCormick
  • How to Use Negative Sampling With Word2Vec Model? by Vijaysinh Lendave
  • Understanding Representation Learning With Autoencoder by Nilesh Barla
• Video:
  • Graph Embeddings (node2vec) explained - How nodes get mapped to vectors by Philipp Brunenberg
  • Node2Vec: Scalable Feature Learning for Networks | ML with Graphs (Research Paper Walkthrough) by TechViz-The Data Science Guy

4. Graph Neural Networks 1:

Required Reading:

• Slide: Graph Neural Networks 1: GNN Model by Jure Leskovec

Suggested Reading:

• Blog:
  • A Gentle Introduction to Graph Neural Networks
  • Understanding Convolutions on Graphs

5. Graph Neural Networks 2:

Required Reading:

• Slide: Graph Neural Networks 2: Design Space by Jure Leskovec

Suggested Reading:

• Blog:
  • Best Graph Neural Network architectures: GCN, GAT, MPNN and more

:pencil: Homework and assignment

More information about homeworks, assignments, and projects will be posted here.

• Assignment Set 1: Deadline 21 Feb 2023 (2 Esfand 1401) at 11:59pm.
• Assignment Set 2: Deadline 17 Mar 2023 (26 Esfand 1401) at 11:59pm.
• Assignment Set 3: Deadline 4 Apr 2023 (15 Farvardin 1402) at 11: