The last vector is scored against every token in the vocabulary; softmax makes the scores probabilities.

Look at the last bar: all remaining tokens of the vocabulary together.

Predict after:

Probability of each next token

Real

Asking Llama 3.1 8B for its real top 20 candidates (one live call)...

12 of 12 live calls left this visit.

8 / 11