Chapter 2: Deep learning

CNNs vs ViTs and transfer learning

You will be able to count image tokens and weights, and plan which pretrained weights to update. CNN means Convolutional Neural Network, a network built from convolution layers. ViT means Vision Transformer, a transformer that reads image patches.

20 minInteractive lessonChapter 2: Deep learningLesson 12 of 20

What you will learn

  • Compute a ViT's patch-embedding parameters and token count
  • Explain the inductive-bias trade-off between CNNs and ViTs
  • Plan a fine-tune: freeze, unfreeze and learning-rate groups

Key terms

patch
A patch is a square image piece.
token
A token is a list of numbers.
attention
Attention lets tokens exchange information.
CNN
CNN means Convolutional Neural Network, a network built from convolution layers.
ViT
ViT means Vision Transformer, a transformer that reads image patches.

Prerequisites

Take Detection metrics: IoU, NMS and mAP first. It comes right before this lesson in the course.

Where this lesson picks up: You scored detection boxes and removed duplicate boxes. Those detectors used small sliding windows; how can a transformer read the same image?

What comes next

What changes when the input already forms a sequence, such as words or time steps?

Next lesson: RNNs, LSTMs and why gradients vanish

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.