Learn-Natural-Language-Processing-Curriculum

This is the curriculum for "Learn Natural Language Processing" by Siraj Raval on Youtube

Course Objective

This is the Curriculum for this video on Learn Natural Language Processing by Siraj Raval on Youtube. After completing this course, start your own startup, do consulting work, or find a full-time job related to NLP. Remember to believe in your ability to learn. You can learn NLP , you will learn NLP, and if you stick to it, eventually you will master it.

Find a study buddy

Join the #NLP_curriculum channel in our Slack channel to find one http://wizards.herokuapp.com

Components each week

• Video Lectures
• Reading Assignments
• Project(s)

Course Length

• 8 weeks
• 2-3 Hours of Study per Day

Tools Used

• Python, PyTorch, NLTK

Prerequisites

• Learn Python https://www.edx.org/course/introduction-python-data-science-2
• Statistics http://web.mit.edu/~csvoss/Public/usabo/stats_handout.pdf
• Probability https://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf
• Calculus http://tutorial.math.lamar.edu/pdf/Calculus_Cheat_Sheet_All.pdf
• Linear Algebra https://www.souravsengupta.com/cds2016/lectures/Savov_Notes.pdf

Week 1 - Language Terminology + preprocessing techniques

Description:

• Overview of NLP (Pragmatics, Semantics, Syntax, Morphology)
• Text preprocessing (stemmings, lemmatization, tokenization, stopword removal)

Video Lectures

• https://web.stanford.edu/~jurafsky/slp3/ videos 1-2.5
• https://www.youtube.com/watch?v=hyT-BzLyVdU&list=PLDcmCgguL9rxTEz1Rsy6x5NhlBjI8z3Gz

Reading Assignments:

• Ch 1-2 of Speech and Language Processing 3rd ed, slides

Project:

• Look at 1-1 to 3-4 to learn NLTK https://github.com/hb20007/hands-on-nltk-tutorial
• Then use NLTK to perform stemming, lemmatiziation, tokenization, stopword removal on a dataset of your choice

Week 2 - Language Models & Lexicons (pre-deep learning)

Description:

• Lexicons
• Pre-deep learning Statistical Language model pre-deep learning ( HMM, Topic Modeling w LDA)

Video Lectures:

• https://courses.cs.washington.edu/courses/csep517/17sp/ lectures 2-6

Reading Assignments:

• 4,6,7,8,9,10 from the UWash course

Extra

• LDA blog post: https://medium.com/@lettier/how-does-lda-work-ill-explain-using-emoji-108abf40fa7d

Project

• https://github.com/TreB1eN/HiddenMarkovModel_Pytorch Build Hidden Markov Model for Weather Prediction in PyTorch

Week 3 - Word Embeddings (Word, sentence, and document)

Video lectures:

• http://web.stanford.edu/class/cs224n/index.html#schedule lectures 1-5

Reading Assignments

• Suggested readings from course

Project

• 3 Assignments Visualize and Implement Word2Vec, Create dependency parser all in PyTorch (they are assigments from the stanford course)

Week 4-5 - Deep Sequence Modeling

Description:

• Sequence to Sequence Models (translation, summarization, question answering)
• Attention based models
• Deep Semantic Similarity

Video Lectures

• https://www.coursera.org/learn/language-processing week 4

Reading Assignments

• Read this on Deep Semantic Similarity Models https://kishorepv.github.io/DSSM/
• Ch 10 Deep Learning Book on Sequence Modeling http://www.deeplearningbook.org/contents/rnn.html

Project

• 3 Assignments, create a translator and a summarizer. All seq2seq models. In pytorch.

Week 6 - Dialogue Systems

Description

• Speech Recognition
• Dialog Managers, NLU

Video Lectures

• https://www.coursera.org/learn/language-processing week 5

Reading Assignments

• Ch 24 of this book https://web.stanford.edu/~jurafsky/slp3/24.pdf

Project

• Create a dialogue system using Pytorch https://github.com/ywk991112/pytorch-chatbot and a task oriented dialogue system using DialogFlow to order food

Week 7 - Transfer Learning

Video Lectures

• My videos on BERT and GPT-2, how to build biomedical startup:
• https://www.youtube.com/watch?v=bDxFvr1gpSU
• https://www.youtube.com/watch?v=J9kbZ5I8gdM
• https://www.youtube.com/watch?v=0n95f-eqZdw
• Transfer learning with BERT/GPT-2/ELMO

Reading Assignments

• http://ruder.io/nlp-imagenet/
• https://lilianweng.github.io/lil-log/2019/01/31/generalized-language-models.html
• http://jalammar.github.io/illustrated-bert/

Project

• Play with this https://github.com/huggingface/pytorch-pretrained-BERT#examples pick 2 models, use it for one of 9 downstream tasks, compare their results.

Week 8 - Future NLP

Description

• Visual Semantics
• Deep Reinforcement Learning

Video Lectures

• CMU Video https://www.youtube.com/watch?v=isxzsAelQX0
• Module 5-6 of this https://www.edx.org/course/natural-language-processing-nlp-3

Reading assignments

• https://cs.stanford.edu/people/karpathy/cvpr2015.pdf
• Hilarious https://medium.com/@yoav.goldberg/an-adversarial-review-of-adversarial-generation-of-natural-language-409ac3378bd7

Project:

• Policy gradient text summarization https://github.com/yaserkl/RLSeq2Seq#policy-gradient-w-self-critic-learning-and-temporal-attention-and-intra-decoder-attention reimplement in pytorch