
Date 
Topics 
Related Materials
and
Resources

Assignments 
1 
3/153/21 
Introduction to Computational
Linguistics / Natural Language Processing
Preliminaries:

PyTorch:

Install Python
3.x and PyTorch
모두를 위한 머신러닝/딥러닝(홍콩과기대 김성훈
교수)
 Linear
Regression의 개념:비디오, 강의 슬라이드
 Linear
Regression cost함수 최소화: 비디오, 강의 슬라이
 여러 개의
입력(feature)의 Linear Regression: 비디오, 강의 슬라이드
 Logistic
Regression classification: 강의 슬라이드Hypothesis 함수 소개:
비디오
cost 함수 소개: 비디오
How to
Implement Simple Linear Regression From
Scratch with Python
Logistic
Regression for Machine Learning

2 
3/223/28 
Introduction
to a Neural Network

PyTorch:

딥러닝 개념잡기

3 
3/294/4 
Introduction to a Neural Network

PyTorch:

Hw1: Logistic
Regression: Diabetes Data

4 
4/54/11 
Introduction to a Neural
Network
 Parameter Optimization
 Weight Decay
 Batch Normalization
 DropOut
Hyperparameter
Tuning Techniques in Deep Learning
An
Overview of Gradient Descent Optimization
Algorithm

PyTorch:

Hw2: How To
Prepare Movie Review Data For Sentiment Analysis

5 
4/124/18 
Introduction to a Neural
Network
 Parameter Optimization
 Weight Decay
 Batch Normalization
 DropOut
Hyperparameter
Tuning Techniques in Deep Learning
An
Overview of Gradient Descent Optimization
Algorithm

PyTorch:

Hw3:
IMDB_TwoLayerNet

6 
4/194/25 
Convolutional Neural
Network
Understanding
Convolutional Neural Network for NLP

PyTorch:


7 
4/264/25 
Reccurent Neural Network
A Friendly Introduction to
Recurrent Neural Network
Long
ShortTerm Memory Neural Network and Gated
Recurrent Unit
MidTerm Test

PyTorch:

MidTermProject:
Korean News Data Torch FNN

8 
5/35/9 
NLP Task 1: Sentiment
Analysis
PyTorch Sentiment
Analysis (IMDB)
NLP Task 2: Sentiment Analysis for Korean
Naver Movie Review Sentiment Analysis

A Comprehensive
Introduction to Torchtext
Torchtext
Github


9

5/105/16 
EncoderDecoder
EncoderDecoder
Long ShortTerm Memory Networks
A
Gentle Introduction to LSTM Autoencoders
Stepbystep
Understanding LSTM Autoencoder layers

PyTorch:


10

5/175/23

Attention
Model
Neural
Machine Translation By Jointly Learning to Align
and Translate
Attention:
Illustrated Attention
Attention and Memory in
Deep Learning and NLP

PyTorch:
Translation with Sequence to
Sequence Network and Attention 

11 
5/245/30 
Transformer
Self Attention: Attention
is All you need
The
Illustrated Transformer
Seq2Seq
Pay Attention to Self Attention: Part I
Seq2seq
Pay Attention to Self Attention: Part 2

PyTorch:
Translation with Sequence to
Sequence Network and Attention
PyTorchTransformers
by Huggingface and Full
Documentation


12 
5/316/6

BERT
(Bidirectional Encoder Representations from
Transformers)
XLNet
What is XLNet and
why it outperforms BERT?
XLNet  a clever
language modeling solution
XLNET SOTA
pretraining method that outperforms
BERT
XLM
XLM  Enhancing
BERT for Crosslingual Language Model
FastBert
Introducing
FastBert  A Simple Deep Learning Library for
BERT Models
RoBERTa
RoBERTa:
A Robustly Optimized BERT Pretaining Approach
RoBERTa:
An optimized method for selfsupervised NLP
systems
Distilling
BERT  How to achieve BERT performance using
Logistic Regression
Meet
ALBERT: a new 'Lite BERT' from Google &
Toyota With State of the ART NLP performance
and 18x fewer parameters
BERT,
RoBERTa, DIstilBERT, XLNet  which one to use?

PyTorch:
The Annotated Transformer
BERT Fine Tuning
BERT FineTuning
Tutorial with PyTorch
Painless
FineTuning of BERT in Pytorch
The Latest
Breakthroughs and Developments
in Natural Language Processing


13 
6/66/13 
Embeddings (word embeddings)
Sebastian Ruder의 On word
Embeddings Part1, 2, 3, 4:
A
handson Intuitive Approach to Deep Learning
Methods for Text Data  Word2Vec, Glove, and
FastText
The
Current Best of Universal Word Embeddings and
Sentence Embeddings
Notebook

PyTorch:
Word
Embeddings: Encoding Lexical Semantics
BERT Word Embedding


14 
6/146/20 
Final Test


