Kernel Fusion: Fuse Two Elementwise Ops into One Pass, Measure the Win
The problem, from first principles
Every elementwise operation (relu, a scalar multiply, adding a bias) is, on real hardware, a separate "kernel" launched one at a time — each one reads its entire input array from main memory, computes, and writes its entire output back to main memory, only for the next op to immediately read that same data straight back in. Chaining n such ops means the data makes n unnecessary round trips through memory it never actually needed to leave.
From theory to code
Implement unfused_memory_traffic(n_elements, n_ops, bytes_per_element=4), the total bytes moved when each op in a chain runs as its own separate pass, and fused_memory_traffic(n_elements, bytes_per_element=4), the total bytes moved if all those ops were fused into a single kernel that never materializes the intermediates. fusion_speedup_estimate reports the ratio between the two.
Constraints
unfused_memory_traffic:n_opsseparate passes, each reading and writing a fulln_elements-sized array.fused_memory_traffic: exactly one read and one write of ann_elements-sized array, regardless ofn_ops.fusion_speedup_estimatereturnsunfused_memory_traffic / fused_memory_trafficfor the samen_elements/n_ops/bytes_per_element.
Hints
Open one at a time. Each gives away a little more than the last.
Hint 1
One pass over n_elements elements moves 2 * n_elements * bytes_per_element bytes total (one read direction, one write direction) — unfused_memory_traffic is just that number, times n_ops passes.
Hint 2
fused_memory_traffic never multiplies by n_ops at all — a fused kernel's memory traffic is a flat 2 * n_elements * bytes_per_element, exactly like a single pass, because it never writes an intermediate result to memory only to immediately read it back.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.