Tutorial: Neural Algorithmic Reasoning

Learning on Graphs Conference · Beginner ·📐 ML Fundamentals ·3y ago

About this lesson

Organizers: Petar Velickovic, Andreea Deac, and Andrew Dudzik Abstract: Neural networks that are able to reliably execute algorithmic computation may hold transformative potential to both machine learning and theoretical computer science. On one hand, they could enable the kind of extrapolative generalisation scarcely seen with deep learning models. On another, they may allow for running classical algorithms on inputs previously considered inaccessible to them. Both of these promises are shepherded by the neural algorithmic reasoning blueprint, which has been recently proposed in a position paper by Petar Velickovic and Charles Blundell. On paper, this is a remarkably elegant pipeline for reasoning on natural inputs which carefully leverages the tried-and-tested power of deep neural networks as feature extractors. In practice, how far did we actually take it? In this tutorial, we aim to provide the foundations needed to answer three key questions of neural algorithmic reasoning: how to develop neural networks that execute algorithmic computation, how to deploy such neural networks in real-world problems, and how to deepen their theoretical links to classical algorithms. Our tutorial will be presented from the ground up, in a way that is accessible to anyone with a basic computer science background. Hands-on coding segments will also be provided, showing how attendees can directly develop their ideas in graph representation learning on relevant algorithmic reasoning datasets (such as CLRS), and then deploy them in downstream agents (e.g., in reinforcement learning). Website: https://logconference.org/schedule-tutorials/#neural-algorithmic-reasoning

Original Description

Organizers: Petar Velickovic, Andreea Deac, and Andrew Dudzik Abstract: Neural networks that are able to reliably execute algorithmic computation may hold transformative potential to both machine learning and theoretical computer science. On one hand, they could enable the kind of extrapolative generalisation scarcely seen with deep learning models. On another, they may allow for running classical algorithms on inputs previously considered inaccessible to them. Both of these promises are shepherded by the neural algorithmic reasoning blueprint, which has been recently proposed in a position paper by Petar Velickovic and Charles Blundell. On paper, this is a remarkably elegant pipeline for reasoning on natural inputs which carefully leverages the tried-and-tested power of deep neural networks as feature extractors. In practice, how far did we actually take it? In this tutorial, we aim to provide the foundations needed to answer three key questions of neural algorithmic reasoning: how to develop neural networks that execute algorithmic computation, how to deploy such neural networks in real-world problems, and how to deepen their theoretical links to classical algorithms. Our tutorial will be presented from the ground up, in a way that is accessible to anyone with a basic computer science background. Hands-on coding segments will also be provided, showing how attendees can directly develop their ideas in graph representation learning on relevant algorithmic reasoning datasets (such as CLRS), and then deploy them in downstream agents (e.g., in reinforcement learning). Website: https://logconference.org/schedule-tutorials/#neural-algorithmic-reasoning

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related AI Lessons

Stop Overfitting With Basically One Line of Code

Learn to prevent overfitting with a simple code tweak and understand the difference between Ridge and Lasso regression

Stop Overfitting With Basically One Line of Code

Learn to prevent overfitting in machine learning models with a simple code tweak and understand the difference between Ridge and Lasso regression

Medium · Machine Learning

Stop Overfitting With Basically One Line of Code

Prevent overfitting in models with a simple code tweak, understanding the difference between Ridge and Lasso regression

Medium · Data Science

Stop Overfitting With Basically One Line of Code

Learn to prevent overfitting in machine learning models with a simple code tweak, comparing Ridge and Lasso regression techniques

Medium · Python

Learn Deep Learning by Hand (Beginner's Guide - Part 1)