Chapter 2: Deep learning
CNNs vs ViTs and transfer learning
You will be able to count image tokens and weights, and plan which pretrained weights to update. CNN means Convolutional Neural Network, a network built from convolution layers. ViT means Vision Transformer, a transformer that reads image patches.
What you will learn
- Compute a ViT's patch-embedding parameters and token count
- Explain the inductive-bias trade-off between CNNs and ViTs
- Plan a fine-tune: freeze, unfreeze and learning-rate groups
Key terms
- patch
- A patch is a square image piece.
- token
- A token is a list of numbers.
- attention
- Attention lets tokens exchange information.
- CNN
- CNN means Convolutional Neural Network, a network built from convolution layers.
- ViT
- ViT means Vision Transformer, a transformer that reads image patches.
Prerequisites
Take Detection metrics: IoU, NMS and mAP first. It comes right before this lesson in the course.
Where this lesson picks up: You scored detection boxes and removed duplicate boxes. Those detectors used small sliding windows; how can a transformer read the same image?
What comes next
What changes when the input already forms a sequence, such as words or time steps?
Next lesson: RNNs, LSTMs and why gradients vanish
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.