5,322
edits
(→Code) |
|||
Line 34: | Line 34: | ||
;Encoder-decoder attention | ;Encoder-decoder attention | ||
The decoder also uses encoder-decoder attention where the keys and values are | The decoder also uses encoder-decoder attention where the keys and values are from the output embedding of the encoder (i.e. the input sentence) but the queries are generated from the decoder input (i.e. the previously-generated output). | ||
===Encoder=== | ===Encoder=== |