DictionaryAI

tokenize

Short definition from Wiktionary

Verb

  1. To reduce to a token or set of tokens by lexical analysis.
  2. To substitute sensitive data with meaningless placeholders.
  3. To treat as a token minority.

We haven’t written up tokenize yet: Simple and Standard meanings, examples and a quiz.