Neural networks

Convolutional network

Imagine scanning a photo with the same small stencil to find edges wherever they appear. A convolutional network slides learned filters across a grid. Shared weights detect local patterns, and stacked layers combine them into larger features.

Input[1, 2, 4]Window 1[1, 2] · [1,−1]Window 2[2, 4] · [1,−1]Output[−1, −2]
The same length-2 filter [1, −1] is reused at both positions, with stride 1 and no bias.

How it works

  1. Slide each filter across the input, taking weighted sums over local patches and input channels.
  2. Add a bias where used and apply a nonlinear activation.
  3. Stack convolutions to grow the receptive field, optionally using pooling or strides to reduce spatial size.
  4. Map the resulting features to task outputs and train the filters with backpropagation.

$$n_{\mathrm{out}}=\left\lfloor\frac{n+2p-d(k-1)-1}{s}\right\rfloor+1$$

The knobs that matter

kernel size
larger filters see a wider patch at each layer
stride and padding
control output size and treatment of boundaries
channels
more filters learn more feature maps at greater cost
dilation
spaces filter samples farther apart to expand the receptive field

What interviewers ask

  • Why are CNNs a good fit for images compared with MLPs?
  • How do you calculate the output shape of a convolution?
  • How does stacking convolutions change the receptive field?
  • How would you use a pretrained CNN as a feature extractor?

Common mistakes

  • Forgetting dilation or padding leads to incorrect output dimensions.
  • Weight sharing gives translation equivariance under suitable conditions, not automatic translation invariance.
  • Flattening a large feature map into a dense layer can create an unexpectedly large parameter count.

When to use it

Use CNNs for data with meaningful local grid structure, including images and some signals. If distant relationships dominate, consider adding a mechanism that connects distant positions directly.

Implement it (9 problems)

Learn it

Interview questions about convolutional network