Deep and Cross Network: Cross Layers v1 and v2

EasyML Coding

Problem statement

Explicit Feature Crosses

Implement the cross layers of DCN and DCN-v2 for a batch x0 of shape (n, d).

cross_layer_v1(x0, xl, w, b) with w and b of shape (d,):

$$x_{l+1} = x_0\,(x_l^T w) + b + x_l$$

where $x_l^T w$ is a scalar per example, so the first term is x0 scaled per row.

cross_layer_v2(x0, xl, W, b) with W of shape (d, d) and b of shape (d,):

$$x_{l+1} = x_0 \odot (W x_l + b) + x_l$$

with $\odot$ the elementwise product.

cross_network(x0, params, version)

  • params is a list of (w_or_W, b) tuples, one per layer. Apply the layers in order starting from x_l = x_0 and return the final x_L.
  • Also return, as the second element of a tuple, the number of parameters used per layer: 2*d for v1 and d*d + d for v2.

Example

Input:

x0 = [[1, 2]], w = [1, 0], b = [0, 0], one v1 layer

Output:

[[2, 4]]

Reasoning: x0^T w = 1, so the cross term is x0 itself; adding the residual doubles it.

Show hint

What shape does the per-example projection $x_l^T w$ have when you compute it for a whole batch at once?

Relevant roles: ML Engineer, Applied Scientist, Deep Learning Engineer and Research Scientist.

Related free problems