Natural language processing: Difference between revisions
| (One intermediate revision by the same user not shown) | |||
| Line 18: | Line 18: | ||
===Transformer=== | ===Transformer=== | ||
[https://arxiv.org/abs/1706.03762 Attention is all you need paper] | {{ main | Transformer (machine learning model)}} | ||
A neural network architecture by Google. | [https://arxiv.org/abs/1706.03762 Attention is all you need paper] | ||
It is currently the best at NLP tasks and has mostly replaced RNNs for these tasks. | |||
A neural network architecture by Google which uses encoder-decoder attention and self-attention. | |||
It is currently the best at NLP tasks and has mostly replaced RNNs for these tasks. | |||
However, it's computational complexity is quadratic in the number of input and output tokens due to attention. | |||
;Guides and explanations | ;Guides and explanations | ||