A typo like 'strwberry' is unfamiliar. Why can a tokenizer still handle it?

Answer: Falls back to subwords

Checks a dictionary — Tokenization is not a normal dictionary lookup. The tokenizer has a learned vocabulary and rules for breaking text into available pieces. That is why it can process made-up strings without first deciding whether they are real words.

Falls back to subwords ✓ — Subword tokenization gives the model a graceful fallback. If the whole misspelled word is not a known chunk, it can be decomposed into smaller chunks or bytes that are in the vocabulary. The neat part is that a compression trick also becomes a way to survive typos and names.

Ignores the odd letters — The odd letters are not supposed to be ignored; they are represented by smaller available pieces. Subword tokenization was invented partly so rare or unknown words would not disappear outside a fixed word list. Dropping letters would turn a typo into a different word, which is not what tokenization is for.