The token machine

1 of 10

A language model predicts the next token. Then it does it again.

Press the button and watch one piece of text join the end each time.

Text so far

Real

The dog barked because it was hungry.

What could come next

Candidates are not available for this model: it does not report its probabilities. See what such a list looks like.

Nothing picked yet.

Real model output: one call returns the whole continuation, each press shows its next token.
Edit text

Every press runs this loop once

  1. Read the text so far
  2. Score every possible next token
  3. Pick one
  4. Append it to the text

Then the whole machine runs again on the longer text.

Sources: 3Blue1Brown, But what is a GPTBrown et al. 2020, GPT-3 abstract

NextYour text becomes tokens