word2vec: The Skip-gram Softmax
Intermediate
The problem, from first principles
Skip-gram models the probability that a context word appears near a center word. A full softmax over the vocabulary gives the probability, using the dot products between the center vector and every output vector.
From theory to code
Implement skipgram_probability(v_c, U, o), returning the softmax probability of the observed context word.
Constraints
- Subtract the maximum score before exponentiating.
Hints
Hint 1
Compute U @ v_c, apply a stable softmax, then read the entry at o.
Keywords
research-papersunsupervisedword-embeddingsword2vec
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.