Embedding Analogies

The problem, from first principles

A famous property of word embeddings is that relationships between words become directions in the space: the vector from "man" to "woman" is roughly the vector from "king" to "queen". That makes analogy questions "a is to b as c is to ?" solvable by arithmetic. Compute the target e_b - e_a + e_c and return the word whose vector is most similar by cosine to it. One rule is essential: the three question words must be excluded from the answer, otherwise the nearest word to the target is usually one of the inputs (typically c).

From theory to code

Implement solve_analogy.

Constraints

• E has shape (V, d) and vocab is a list of V words. solve_analogy(E, vocab, a, b, c) takes three words from vocab.
• Compute target = E[b] - E[a] + E[c]. Score every word by cosine similarity with target (treat a zero-norm vector as similarity 0).
• Exclude a, b and c from the candidates and return the word with the highest similarity (lowest index on ties).

Hints