Tokenizers: BPE, WordPiece, SentencePiece

Your LLM does not read English. It reads integers. The tokenizer decides whether those integers carry meaning or waste it.