Oxford Course on Deep Learning for Natural Language Processing

 Deep Learning methods achieve state-of-the-art results on a suite of natural language processing problems

What makes this exciting is that single models are trained end-to-end, replacing a suite of specialized statistical models.

The University of Oxford in the UK teaches a course on Deep Learning for Natural Language Processing and much of the materials for this course are available online for free.

In this post, you will discover the Oxford course on Deep Learning for Natural Language Processing.

After reading this post, you will know:

• What the course entails and the prerequisites.
• A breakdown of the lectures and how to access the slides and videos.
• A breakdown of the course projects and where to access the materials.

Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

Oxford Course on Deep Learning for Natural Language Processing
Photo by Martijn van Sabben, some rights reserved.

Overview

This post is divided into 4 parts; they are:

1. Course Overview
2. Prerequisites
3. Lecture Breakdown
4. Projects

Course Overview

The course is titled “Deep Learning for Natural Language Processing” and is taught at the University of Oxford (UK). It was last taught in early 2017.

The great thing about this course is that it is run and taught by Deep Mind people. Notably, the lecturer is Phil Blunsom.

The focus of the course is on statistical methods for natural language processing, specifically neural networks that achieve state-of-the-art results on NLP problems.

From the course:

This will be an applied course focusing on recent advances in analysing and generating speech and text using recurrent neural networks. We will introduce the mathematical definitions of the relevant machine learning models and derive their associated optimisation algorithms.

Prerequisites

This course is designed for undergraduate and graduate students.

The course assumes some background in the topics:

• Probability.
• Linear Algebra.
• Continuous Mathematics.
• Basic Machine Learning.

If you are practitioner interested in deep learning for NLP, you may have different goals and requirements from the material.

For example, you may want to focus on the methods and applications rather than the foundational theory.

Lecture Breakdown

The course is comprised of 13 lectures, although the first and second lectures are both split into two parts.

The complete lecture breakdown is provided below.

The GitHub repository for the course provides links to slides, flash videos and reading for each lecture.

I would recommend watching the videos via this unofficial YouTube playlist.

Below is a course overview slide taken from the first lecture.

Deep Learning for Natural Language Processing at Oxford Lecture Breakdown

Note that there are many guest lecturers for the various topics covered and most are from Deep Mind.

• Lecture 1a – Introduction [Phil Blunsom]
• Lecture 1b – Deep Neural Networks [Wang Ling]
• Lecture 2a – Word Level Semantics [Ed Grefenstette]
• Lecture 2b – Overview of the Practicals [Chris Dyer]
• Lecture 3 – Language Modelling and RNNs Part 1 [Phil Blunsom]
• Lecture 4 – Language Modelling and RNNs Part 2 [Phil Blunsom]
• Lecture 5 – Text Classification [Karl Moritz Hermann]
• Lecture 6 – Deep NLP on Nvidia GPUs [Jeremy Appleyard]
• Lecture 7 – Conditional Language Models [Chris Dyer]
• Lecture 8 – Generating Language with Attention [Chris Dyer]
• Lecture 9 – Speech Recognition (ASR) [Andrew Senior]
• Lecture 10 – Text to Speech (TTS) [Andrew Senior]
• Lecture 11 – Question Answering [Karl Moritz Hermann]
• Lecture 12 – Memory [Ed Grefenstette]
• Lecture 13 – Linguistic Knowledge in Neural Networks

Have you watched any of these lectures? What did you think?
Let me know in the comments below.

Projects

The course includes 4 practical projects that you may wish to attempt to confirm your knowledge of the topic.

The projects are as follows, and each has their own GitHub project containing the description and relevant starting materials:

• Practical 1: word2vec
• Practical 2: text classification
• Practical 3: recurrent neural networks for text classification and language modelling
• Practical 4: open practical

Further Reading

This section provides more resources on the topic if you are looking go deeper.

• Deep Learning for Natural Language Processing: 2016-2017
• Course Lectures on GitHub (PDF)
• Course Lecture Videos on YouTube
• Discussion of Course on Hacker News

Summary

In this post, you discovered the Oxford course on Deep Learning for Natural Language Processing.

Specifically, you learned:

• What the course entails and the prerequisites.
• A breakdown of the lectures and how to access the slides and videos.
• A breakdown of the course projects and where to access the materials.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.