LoRA: The Adapted Forward Pass
Intermediate
The problem, from first principles
The adapted layer computes the frozen output plus the scaled low-rank path. Because that path is linear, the adapter can be merged into the weight after training, leaving inference cost unchanged.
From theory to code
Implement lora_forward(x, W, A, B, scale), the frozen layer plus the scaled LoRA path.
Constraints
- Matrix shapes follow the docstring.
Hints
Hint 1
Compute x W^T, compute x A^T then B^T for the adapter, scale it, and add.
Keywords
research-paperssystemsfine-tuningadapters
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.