Back to the Glossary

Token

The basic unit a model uses to process text — typically a word or a fragment of a word.

More in the same category

Fine-tuning Pre-training RLHF (Reinforcement Learning from Human Feedback)Dataset Gradient Descent Overfitting

Browse the full glossary