Starts: Mar. 19, 11am EST.
In this Working group, over ~ 10 weeks we’ll investigate various (domain-specific) evaluation metrics for link prediction tasks using GNN. Our weekly meeting time will be every Saturday at 11 am EST.
This engagement requires having basic knowledge in graphs (e.g. graph theory, graph algorithms or graph neural networks) and some experience with deep learning frameworks (e.g. PyTorch, TensorFlow). However, you don’t need to be an expert in either of them, passion in GNN is the most important criteria 🥳
Leads / Advisors
Percy (Boqi) Chen | Co-Lead
~ PhD Candidate at McGill University, ML Engineer at Aggregate Intellect
Peter Shih | Co-Lead
~ Senior Associate, One Analytics PwC Canada
Overview
What do you use to evaluate the link prediction model after training the GNN?
- Accuracy 🎯?
- Precision 🎯?
These are interesting metrics but they only check edges but not the graph structure... Imagine that you have a link prediction model that predicts parental relationships that & has a 90% accuracy. But this model also predicts that A is the parent of B, B is the parent of C and C is the parent of A 😲... This doesn’t look right 😰...
Join us in learning about various link prediction metrics to build confidence on any link prediction model you will ever train 👊!
Tentative Timeline
- March 19th introductory meet. We get to know each other & talk more about the project / scope.
- March 19th - Apr 9nd - Recap Period. We will go over project pre-requisites to help fill in gaps. Our main resource will be the Starter Recipe below. Members will be expected to go over in the assigned period.
- Apr 9nd - Members who can keep up will be invited to join a Core Team at the discretion of leads / advisors.
- Apr 9nd - June 6th - Core Team works towards building MVP. We target to create a medium post of the metrics we found and build a mini library for evaluating GNN models
Minimum Goal: Create a checklist for link prediction evaluation
Extended Goal: Create a mini library that implements various metrics
Starting Recipe!
We will create a recipe to get everyone started with link prediction with GNN. The Starting Recipe To Be Posted Soon!
Our Past Journey
Link Prediction in Knowledge graphs
We applied a SOTA link prediction algorithm: KBAT in a complex knowledge graph benchmark CoDeX
Prerequisite Learning for Curriculum Planning
We learn how to create a curriculum planning tool for any learning path with GNN
GNN Foundations
We went through the famous CS224W course to learn the fundamentals of GNN. (This course will also serve as an initial starting point for this working group!)
Why join?
Aggregate Intellect hosts one of the most diverse ML communities in the world. Over the course of the working group
- You’ll get an immersion into that community & walk out with some cool new friends.
- Work with enthusiastic teammates and present your work to others in the group
- Learn and experience the entire link prediction pipeline: from data acquisition to evaluation
- Post and contribute content on the cutting edge of link prediction metrics
The button below will take you to our coordination Slack channel. Just jump in, introduce yourself, and say you wanna join! We’ll try to help bring you up to speed.