Backpropagation Gradients for a Dense Layer

MediumML Coding

Problem statement

Implement backpropagation for a single fully connected layer with sigmoid activation and squared-error cost. The previous layer has K neurons (activations a_prev) and the current layer has J neurons.

The forward computation is

  • z = W @ a_prev + b
  • a = sigmoid(z)
  • C = sum_j (a_j - y_j)^2

Given a_prev (shape K), W (shape J x K), b (shape J) and target y (shape J), implement dense_backward(a_prev, W, b, y) that computes:

  1. dC/dW (shape J x K)
  2. dC/db (shape J)
  3. dC/da_prev (shape K). Each a_prev[k] feeds all J neurons, so its gradient must aggregate contributions from every output neuron.

Return a dictionary with keys 'dW', 'db', 'da_prev' whose values are Python lists (nested for dW). Do not round inside the function.

Example

Input:

a_prev = [1.0, 2.0], W = [[0.1, 0.2], [0.3, 0.4]], b = [0.0, 0.0], y = [0.0, 1.0]

Output (rounded to 6 decimals):

{'dW': [[0.292561, 0.585121], [-0.093587, -0.187175]], 'db': [0.292561, -0.093587], 'da_prev': [0.00118, 0.021077]}

Reasoning: $z = Wa_{prev} + b = [0.5, 1.1]$ and $a = \sigma(z) \approx [0.6225, 0.7503]$. Then $\partial C/\partial z = 2(a - y)\,a(1 - a) \approx [0.2926, -0.0936]$, dW is its outer product with a_prev, db equals it, and da_prev $= W^\top \partial C/\partial z$ sums the contributions of both output neurons.

Show hint

What is the gradient of the cost with respect to each neuron's pre-activation, and which factors does the chain rule give you there?

Relevant roles: ML Engineer, Deep Learning Engineer, Research Scientist, Applied Scientist and Data Scientist.

Related free problems