Neural Tangent Kernel: The ReLU Arc-Cosine Kernel
Advanced
The problem, from first principles
For a wide ReLU network, the NTK between two inputs reduces to an arc-cosine kernel: a formula that depends only on the two input norms and the angle between them. Similar inputs score high and dissimilar ones score low, without any training.
From theory to code
Implement arc_cosine_kernel(x, y), using the norms of x and y and the angle between them.
Constraints
- Return
0.0when either vector is zero.
Hints
Hint 1
Compute the angle theta from the normalized dot product, then apply |x||y| / (2 pi) * (sin theta + (pi - theta) cos theta).
Keywords
research-papersclassical-mlkernelsntk
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.