Simple explanations of AI concepts for everyone
The smallest unit of text a model processes. It can be a word, part of a word, or even a symbol.
In context: Text is split into small units the model can read. Each token is linked to a number, like its own ID. Notice how "agony" becomes two tokens: "ag" and "ony".
The process of breaking text into small units (tokens) a model can understand. Each token can be a word, subword, or character.
In context: When you send text to a model, this is the very first step before anything else happens.
The model turns each token into numbers that represent its meaning. Tokens with similar meanings have vectors (points positioned in space) that are close together.
In context: Turns tokens into points in space, grouped by meaning. "Cat" and "dog" end up close together; "car" and "truck" form another cluster.
The limit of how much text a model can consider at once. It reads and reasons only within this window, measured in tokens.
In context: The model can process a limited number of tokens at once. Modern models range from 4K to 200K+ tokens.
An internal map where the model organizes what it has learned. Each point represents a concept, and similar ideas group close together.
In context: Each dot is an embedding, placed near others with similar meaning. It's how the model organizes concepts to relate them efficiently.
A network of connected layers that learn from examples. Each layer refines the data, and together they learn patterns used to recognize images, understand language, or process sounds.
In context: Each layer transforms information a bit, finding patterns. By the end, the network can turn a question into the right answer.
Values the model learns during training that determine how strongly different parts of the network connect and respond. Together, they define how the model understands and generates information.
In context: More parameters typically mean a smarter, more flexible model—but not always. Quality of training matters too.
A system that has learned from data and can now use that knowledge to predict, generate, or understand new information.
In context: A neural network trained on tons of examples so it can predict or generate new things.
A type of neural network that looks at every word in a sequence at once. Unlike earlier models that read step by step, it learns how words relate across the whole text.
In context: Understands relationships between words across a whole sentence. The model sees that "sky" and "orange" are connected.
A mechanism inside Transformers that decides which words to focus on when processing a sentence. Each word looks at others and assigns more weight to the ones that matter most.
In context: Helps the model decide which words to focus on for meaning. When processing "The", it pays most attention to "sat" and "mat".
The first learning stage where a model trains on vast text data to learn patterns, context, and general knowledge.
In context: Like going to school before specializing. The model learns language fundamentals first.
Training a pre-trained model on new, specific data so it adapts to a particular task or tone. It keeps what it already knows but learns to apply it in a focused way.
In context: Teaches the model a new skill without forgetting what it already knows.
A training method where the model improves through feedback. It tries actions, receives rewards or penalties, and learns to make better decisions over time.
In context: Improves the model through trial, error, and feedback until it gets better results. RLHF (from human feedback) is commonly used.
Step-by-step reasoning the model writes to reach an answer. It helps break complex problems into smaller, more manageable steps.
In context: Shows how the model thinks through a problem before answering. Often improves accuracy on complex tasks.
The stage where a trained model uses what it has learned to generate a response. It predicts the next token step by step until the answer is complete.
In context: This is what happens behind the scenes when you use an AI product. Training teaches; inference applies.
Retrieval-Augmented Generation. A method that lets a model look up information before answering. It retrieves relevant data from external sources, then uses that context to write a better answer.
In context: When AI searches the web or your documents before answering, that's RAG in action.
An autonomous system that uses tools and feedback loops to accomplish tasks. It can plan, execute actions, observe results, and adjust its approach.
In context: Unlike a simple chatbot, an agent can take actions—like searching, writing code, or booking appointments.
A predefined sequence of steps where each stage uses the previous result to move the task forward toward a final outcome.
In context: Connects steps into a clear, predictable path. Unlike agents, workflows follow a fixed sequence.
Large Language Model. A very large neural network trained on vast text data to understand, predict, and generate human language.
In context: ChatGPT, Claude, and Llama are all LLMs. They power most AI chat applications today.
The text input you give to an AI model to tell it what you want. It's your question, instruction, or the context that guides the model's response.
In context: The better your prompt, the better the response. Prompt engineering is the art of crafting effective instructions.
When an AI model generates information that sounds confident but is actually false or made up. The model doesn't "know" it's wrong.
In context: Models predict likely text, not truth. Always verify important facts, especially for names, dates, and citations.
A setting that controls how random or creative the model's outputs are. Low temperature = more focused and predictable. High temperature = more varied and creative.
In context: Use low temperature for factual tasks, higher for creative writing or brainstorming.
Teaching a model what you want by showing it a few examples in your prompt. The model learns the pattern and applies it to new inputs.
In context: Instead of explaining rules, you show examples. The model figures out you want opposites.
Asking a model to do a task without giving any examples. The model relies entirely on its pre-trained knowledge to understand and complete the task.
In context: Modern LLMs are good at zero-shot tasks because they've seen similar patterns during training.
AI that can understand and work with multiple types of input: text, images, audio, video, or any combination of these.
In context: You can show an image and ask questions about it. The model "sees" and understands both text and visuals.
A technique used by image generators. It starts with random noise and gradually removes it step by step, guided by your text prompt, until a clear image emerges.
In context: DALL-E, Midjourney, and Stable Diffusion all use this technique to generate images from text.
A database designed to store and search embeddings (numerical representations of data). It finds similar items by measuring distance between vectors.
In context: Powers semantic search and RAG systems. When you search "happy dog photos", it finds images with similar meaning, not just matching words.
Search that understands meaning, not just keywords. It finds results that are conceptually similar to your query, even if they use different words.
In context: Searching for "how to fix a broken heart" returns emotional advice, not cardiology articles.
Graphics Processing Unit. Originally designed for rendering graphics, GPUs excel at the parallel math operations that AI models need, making them essential for training and running AI.
In context: AI training requires massive parallel computation. A single GPU can do thousands of calculations simultaneously.
Compressing a model by reducing the precision of its numbers. Instead of using 32-bit numbers, it might use 8-bit or 4-bit, making the model smaller and faster with minimal quality loss.
In context: Lets you run large models on smaller devices. A 70B model quantized to 4-bit can run on a laptop.