Course:
Deep Learning in NLP
Course channel: https://t.me/dlinnlp_discuss
Course classroom: http://github.com/deep-nlp-spring-2020/
Youtube channel: https://www.youtube.com/c/DeepPavlov
Now registration is closed. But you can find more information about course and useful links below.
Plan our classes
Nowadays, deep learning provides state-of-the-art techniques for many NLP problems. In this course, students gain a thorough introduction into the Neural Networks, RNN, Transformers. The requirements of this course are complete quizzers, practical assignments and a final project.
#
Lecturer
Description
Links
NLP#1:
Word vector representations
Aleksey Sorokin
  • Welcome to the Deep Learning in Natural Language Processing course – and to the magnificent world of artificial intelligence!
NLP#2:
Neural Networks. Backpropagation
Aleksey Sorokin
  • Word embeddings retraining to a specific task
  • Introduction to neural network
  • Backpropagation
  • Activation functions
NLP#3-4:
Neural Networks. Optimization and Normaization
Vasily Konovalov
  • Initialization
  • Batch Normalization
  • Babysitting the Learning Process
  • Optimization
  • Regularization
NLP#5:
Recurrent Neural Networks and Language Models
Aleksey Sorokin
  • RNNs
  • Backpropagation in RNNs
  • Vanishing / exploding gradients
  • LSTM, GR
Seminar:
The next generation of language models
Mikhail Burtsev
Mikhail Burtsev gives a talk about the problems of neural network architectures based on transformers (first of all, BERT and its variants) in relation to the task of language modeling, and offer research directions to overcome these problems.
NLP#6:
Deep contextualized word representations

Vasily Konovalov
  • Language models can build great word representations
  • ELMo architecture
  • ULMfit architecture
  • Fitting hacks and tricks
NLP#7:
Translation, Seq2Seq, Attention
Vasily Konovalov
  • Seq2seq architecture
  • Teacher forcing
  • Bottleneck problem
  • Attentio
NLP#8:
Contextual Word Embeddings
Vasily Konovalov
  • Transformer architectur
NLP#9:
Question Answering
Vasily Konovalov
  • BERT
  • Question answering over text
  • Attention flow
NLP#10:
Natural Language Generation
Vasily Konovalov
Supports

Keep in touch
Sign up to get the latest updates on our community and news.
We promise not to spam nor use your email for anything else.