Chapter 2: Deep learning

Forward pass: layers, activations and shapes

A neural network is a stack of weight tables with bends between them. Here you trace every table's shape through a network for photos of digits, count what it learns, and see why the bends matter. Interviewers ask exactly this, and shape mistakes are a common crash in real work.

20 minInteractive lessonChapter 2: Deep learningLesson 1 of 18

What you will learn

  • Write a two-layer MLP forward pass with correct shapes
  • Explain why stacked linear layers without activations collapse to one
  • Compare ReLU, GELU and sigmoid activations

Key terms

batch
a group of examples, one per row, handled together
shape
the rows and columns of a table, such as (32, 784)
weight table
the numbers a layer learns and multiplies its input by
matrix product
rows of one table times columns of another, written @
transpose
a table with its rows turned into columns, written .T

Prerequisites

Take Capstone: churn with survival analysis first. It comes right before this lesson in the course.

What comes next

The network gives 10 raw scores per photo: how do you turn them into chances that add up to 1?

Next lesson: Softmax, cross-entropy and the stable log-sum-exp

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.