Neural Turing Machines: Content Addressing
Advanced
The problem, from first principles
Neural Turing Machines (Graves et al., 2014) read and write an external memory with differentiable attention. Content addressing finds memory rows similar to a key, using cosine similarity sharpened by a strength beta.
From theory to code
Implement content_weights(memory, key, beta), the softmax of beta times the cosine similarities.
Constraints
- Cosine similarity is scale-invariant in the key.
Hints
Hint 1
Compute each row's cosine similarity with the key, scale by beta, then take a stable softmax.
Keywords
research-paperssequence-modelsmemoryneural-computation
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.