Examples of tokens (Credit: OpenAI) The product will take a bit of textual content (say, the opening sentence of the Wikipedia posting) and tries to predict the next token from the sequence. It then compares its output with the actual textual content from the instruction corpus and adjusts its param