◑ applying · kind concept · level 3 · 30.17h
- Requiere: Neural Network
- Contradice: Recurrent Neural Network
A content-based weighted lookup over a set of representations — every position can attend to every other. Removing the sequential bottleneck of RNNs is the pivot of modern NLP.
In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In Natural Language Processing, importance is represented by “soft” weights assigned to each word in a sentence. More generally, attention encodes vectors called token Embeddings across a fixed-width sequence that can range from tens to millions of tokens in size.
Enlaces
- Requiere: Neural Network
- Contradice: Recurrent Neural Network