
Date 
Topics 
Related Materials
and
Resources

Assignments 
1 
9/19/7 
Introduction to Computational
Linguistics / Natural Language Processing
Preliminaries:

PyTorch:

Install Python
3.x and PyTorch
모두를 위한 머신러닝/딥러닝(홍콩과기대 김성훈
교수)
 Linear
Regression의 개념:비디오, 강의 슬라이드
 Linear
Regression cost함수 최소화: 비디오, 강의 슬라이
 여러 개의
입력(feature)의 Linear Regression: 비디오, 강의 슬라이드
 Logistic
Regression classification: 강의 슬라이드Hypothesis 함수 소개:
비디오
cost 함수 소개: 비디오
How to
Implement Simple Linear Regression From
Scratch with Python
Logistic
Regression for Machine Learning

2 
9/89/14 
Introduction
to a Neural Network

PyTorch:

딥러닝 개념잡기

3 
9/159/21 
Introduction to a Neural Network

PyTorch:

HomeWork Assignment
1: Backpropagation Calculation in Pytorch (refer
to ETL)

4 
9/229/28 
Introduction to a Neural
Network
 Parameter Optimization
 Weight Decay
 Batch Normalization
 DropOut
Hyperparameter
Tuning Techniques in Deep Learning
An
Overview of Gradient Descent Optimization
Algorithm
New
State of the Art AI Optimizer: Rectified Adam
(RAdam)
New
Deep Learning Optimizer, Ranger: Synergistic
combination of RAdam + LookAhead for the best
of both

PyTorch:


5 
9/2910/5 
Introduction to a Neural
Network
 Parameter Optimization
 Weight Decay
 Batch Normalization
 DropOut
Hyperparameter
Tuning Techniques in Deep Learning
An
Overview of Gradient Descent Optimization
Algorithm
New
State of the Art AI Optimizer: Rectified Adam
(RAdam)
New
Deep Learning Optimizer, Ranger: Synergistic
combination of RAdam + LookAhead for the best
of both

PyTorch:


6 
10/610/12 
Convolutional Neural
Network
Understanding
Convolutional Neural Network for NLP

PyTorch:

HomeWork
Assignment 2: Convolutional Neural Networks
Paper 요약 (refer to ETL)

7 
10/1310/19 
Reccurent Neural Network
A Friendly Introduction to
Recurrent Neural Network
Long
ShortTerm Memory Neural Network and Gated
Recurrent Unit

PyTorch:


8 
10/2010/26 
EncoderDecoder
EncoderDecoder
Long ShortTerm Memory Networks
A
Gentle Introduction to LSTM Autoencoders
Stepbystep
Understanding LSTM Autoencoder layers

PyTorch:


9 
10/2711/2 
Attention Model
Neural
Machine Translation By Jointly Learning to Align
and Translate
Attention:
Illustrated Attention
Attention and Memory in Deep
Learning and NLP

PyTorch:
Translation with Sequence to
Sequence Network and Attention 

10 
11/311/9 
Transformer
Self Attention: Attention
is All you need
The
Illustrated Transformer
Seq2Seq
Pay Attention to Self Attention: Part I
Seq2seq
Pay Attention to Self Attention: Part 2

PyTorch:
Translation with Sequence to
Sequence Network and Attention
PyTorchTransformerrs
by Huggingface


11 
11/1011/16 
BERT
(Bidirectional Encoder Representations from
Transformers)

PyTorch:
The Annotated Transformer
BERT
Fine Tuning


12 
11/1711/23 
Embeddings (word embeddings)
Sebastian Ruder의 On word
Embeddings Part1, 2, 3, 4:
A
handson Intuitive Approach to Deep Learning
Methods for Text Data  Word2Vec, Glove, and
FastText
The
Current Best of Universal Word Embeddings and
Sentence Embeddings

PyTorch:
Word
Embeddings: Encoding Lexical Semantics
BERT Word Embedding


13 
12/112/7 
NLP Task 1: Sentiment
Analysis
PyTorch Sentiment Analysis
(IMDB)
NLP Task 2: Sentiment
Analysis for Korean
Naver Movie Review Sentiment Analysis

PyTorch:
A
Comprehensive Introduction to
Torchtext(Practical Torchtext part1)
Language
Modeling tutorial in Torchtext(Practical
Torchtext part2)


14 
12/812/14 
NLP Task
2: Chatbot
Chatbot
Tutorial
NLP Task 3: Generating and Classifying
Names with a CharacterLevel RNN



15 
12/1512/21 
Final Project
Presentation


