Neural networks
Convolutional network
Imagine scanning a photo with the same small stencil to find edges wherever they appear. A convolutional network slides learned filters across a grid. Shared weights detect local patterns, and stacked layers combine them into larger features.
How it works
- Slide each filter across the input, taking weighted sums over local patches and input channels.
- Add a bias where used and apply a nonlinear activation.
- Stack convolutions to grow the receptive field, optionally using pooling or strides to reduce spatial size.
- Map the resulting features to task outputs and train the filters with backpropagation.
$$n_{\mathrm{out}}=\left\lfloor\frac{n+2p-d(k-1)-1}{s}\right\rfloor+1$$
The knobs that matter
- kernel size
- larger filters see a wider patch at each layer
- stride and padding
- control output size and treatment of boundaries
- channels
- more filters learn more feature maps at greater cost
- dilation
- spaces filter samples farther apart to expand the receptive field
What interviewers ask
- Why are CNNs a good fit for images compared with MLPs?
- How do you calculate the output shape of a convolution?
- How does stacking convolutions change the receptive field?
- How would you use a pretrained CNN as a feature extractor?
Common mistakes
- Forgetting dilation or padding leads to incorrect output dimensions.
- Weight sharing gives translation equivariance under suitable conditions, not automatic translation invariance.
- Flattening a large feature map into a dense layer can create an unexpectedly large parameter count.
When to use it
Use CNNs for data with meaningful local grid structure, including images and some signals. If distant relationships dominate, consider adding a mechanism that connects distant positions directly.
Implement it (9 problems)
- Simple Convolutional 2D LayerFreeMediumSolve
- 3D CNN Forward PassProHardSolve
Learn it
Interview questions about convolutional network
- MLP vs CNN vs RNNProEasySolve