QLoRA: Quantized Low-Rank Adaptation Forward Pass

EasyML Coding

Problem statement

Implement the forward pass of QLoRA. The frozen pretrained weights are stored as 4-bit integers and dequantized to full precision during the forward pass, while the trainable LoRA matrices stay in full precision.

Inputs: x of shape (batch, in_features), integer quantized_W of shape (in_features, out_features) with a single scale and zero_point, B of shape (in_features, r) and A of shape (r, out_features), where $r$ is the number of rows of A.

  1. Dequantize: $W = \text{quantized\_W} \cdot \text{scale} + \text{zero\_point}$.
  2. Compute $\text{output} = x W + \frac{\alpha}{r} \, x B A$.

Return the output as a list of lists.

Example

Input:

x = [[1.0, 2.0]]
quantized_W = [[0, 10], [10, 0]]; scale = 0.1; zero_point = 0.0
B = [[0.5], [0.5]]; A = [[1.0, 1.0]]; alpha = 1.0

Output:

[[3.5, 2.5]]

Reasoning: dequantizing gives $W = [[0, 1], [1, 0]]$, so $xW = [2, 1]$. The adapter path is $xB = [1.5]$, $xBA = [1.5, 1.5]$, scaled by $\alpha / r = 1$. The sum is $[3.5, 2.5]$.

Show hint

How do you recover approximate real-valued weights from integers, a scale and a zero point?

Relevant roles: LLM Engineer, Gen AI Engineer, ML Engineer, AI Engineer and MLOps Engineer.

Related free problems