DeepNLPIntro2019

Hands on introduction to Deep learning for NLP. Review NLP fundamentals with Spacy, word representations with Gensim, deep learning architectures and applications with PyTorch, and introduce current state of the art models and methods for transfer learning.

Topic Outline

0. Introduction to Deep Learning and the Workshop

1. Introduction to NLP

• Deep Learning Concepts
• NLP, NLP vocabulary, why NLP is hard
• NLP core tasks and applications
• Instructions for running labs

Notebook – Introduction to NLP using Spacy

• Tokenization
• POS
• Named entity recognition
• Sentiment classification

2. Word Representations and Embeddings

• Embeddings and self-supervised pre-training
• word2vec, GLoVe, fasttext

Notebook – Word Representations

• Embeddings
• Note: You can skip the t-SNE cells at the bottom (very slow)

3. PyTorch for Deep NLP

• PyTorch intro
• Training networks
• NLP on PyTorch
• RNNs
• Sequence models
• Encoder-decoder
• Transfer learning

Notebook – Text Classification with BOW

Notebook – Text Classification with RNN

Notebook – Character language model with RNN optional

Notebook – seq2seq with Attention RNN

Notebook – Fine Tuning with GPT

Notebook – Embeddings with Context

Optional PyTorch Baiscs

Notebook – PyTorch Basics

• Tensors, tensor operations

Notebook – PyTorch Machine Learning

• Gradients, linear regression, logisitc regression, neural network

[Google Collab] (https://colab.research.google.com)