Chapter 2: Deep learning
Forward pass: layers, activations and shapes
A neural network is a stack of weight tables with bends between them. Here you trace every table's shape through a network for photos of digits, count what it learns, and see why the bends matter. Interviewers ask exactly this, and shape mistakes are a common crash in real work.
What you will learn
- Write a two-layer MLP forward pass with correct shapes
- Explain why stacked linear layers without activations collapse to one
- Compare ReLU, GELU and sigmoid activations
Key terms
- batch
- a group of examples, one per row, handled together
- shape
- the rows and columns of a table, such as (32, 784)
- weight table
- the numbers a layer learns and multiplies its input by
- matrix product
- rows of one table times columns of another, written @
- transpose
- a table with its rows turned into columns, written .T
Prerequisites
Take Capstone: churn with survival analysis first. It comes right before this lesson in the course.
What comes next
The network gives 10 raw scores per photo: how do you turn them into chances that add up to 1?
Next lesson: Softmax, cross-entropy and the stable log-sum-exp
More lessons in this part
Take the lesson
Short cards, one idea at a time, with quick checks as you go.