Training, tokens, and context windows explained

What Is Training?

Training is the process where AI learns from massive amounts of information.

Sources may include:

• Books
• Articles
• Websites
• Documentation
• Research papers

What Are Tokens?

AI does not read words exactly like humans do.

Example:

Artificial Intelligence is amazing.

May become: (tokens)

Artificial
Intelligence
is
amazing
.

Why Tokens Matter

Tokens influence:

• Cost
• Response length
• Processing speed
• Context limits

Rule of thumb:

1 token  +-  0.75 words

What Is a Context Window?

A context window is the amount of information AI can actively remember during a conversation.

Think of it as the model's short-term memory.