Top Qs
Timeline
Chat
Perspective
Tokenization
Topics referred to by the same term From Wikipedia, the free encyclopedia
Remove ads
Tokenization may refer to:
- Tokenization (lexical analysis) in language processing
- Tokenization in large language models
- Tokenization in search engine indexing
- Tokenization (data security)
- Asset tokenization in finance
- Word segmentation
- A procedure during the Transformer architecture
See also
- Tokenism of minorities
This disambiguation page lists articles associated with the title Tokenization.
If an internal link led you here, you may wish to change the link to point directly to the intended article.
If an internal link led you here, you may wish to change the link to point directly to the intended article.
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads