Breakthrough · 2017

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

Introduces the transformer, which replaces recurrence with multi-head self-attention and positional encodings.