How AI Works: From Your Prompt to Intelligent Action

Technical Text is split by a tokenizer (e.g. BPE) into subword units, each mapped to an integer ID from a fixed vocabulary (~50k–200k entries). IDs are looked up in an embedding matrix, producing dense vectors the transformer operates on. The full request = system prompt + conversation history + user turn, concatenated into one token sequence bounded by the context window.

Technical There is no separate “NLU module” in an LLM — understanding is implicit. Self-attention lets every token attend to every other token, so “cheap” modifies “flights” and “Sydney” binds as destination. Semantically similar phrases land near each other in embedding space, which is why paraphrases (“low-cost fares to SYD”) resolve to the same meaning.

Technical A decoder-only transformer maps the token sequence to a probability distribution over the vocabulary for the next token. Sampling parameters (temperature, top_p) trade determinism for creativity. Generation is autoregressive: each emitted token is appended and the pass repeats. Weights are frozen at inference — the model learns nothing new while answering you.

Technical “Memory” is engineered, not innate. Short-term = the conversation replayed into the context window each turn. Long-term = facts persisted in a store (often a vector database), retrieved by similarity and injected into the prompt. External knowledge = API/tool calls at inference time. The model itself is stateless between requests.

Technical Docs are chunked and embedded into a vector database. At query time the question is embedded, nearest chunks retrieved (cosine similarity, often + keyword/BM25 hybrid, then reranking), and the top chunks are stuffed into the prompt with instructions to answer only from them and cite sources. Quality levers: chunking strategy, embedding model, k, reranker, and prompt grounding.

Technical Tool/function calling: tools are described to the model as schemas (name, parameters). The model emits a structured call like book_table(time:"19:00", size:2); the runtime executes it and feeds the result back as a new message, looping until done. Standards like MCP make tools plug-and-play across apps. Guardrails live in the runtime, not the model.

Technical An agent = an LLM in a loop with tools and state: while not done: observe → reason → call tool → read result. The chatbot is a single model call; the agent is an orchestrated runtime with termination conditions, error handling, retries, and permission scopes. Same model — different harness.

Technical Patterns: ReAct (interleaved reasoning + tool calls), plan-then-execute, and reflection/self-critique passes. State (goal, plan, tool results, scratchpad) is carried in context or external memory. Orchestration frameworks handle retries, timeouts, max-iteration caps, and structured termination (“goal satisfied” vs “blocked — ask human”).

Technical Architectures: orchestrator–worker (a lead agent decomposes and delegates), pipelines (sequential hand-offs), and debate/critique panels. Agents exchange structured messages; shared state lives in a common store. Hard problems: context hand-off fidelity, deadlock/loop detection, and end-to-end tracing across agents.

Technical The full stack: tokenizer → embeddings → transformer inference → context assembly (history + memory + RAG chunks) → tool-calling loop → agent orchestration. Each layer is independently swappable and testable — which is exactly how you should build, evaluate, and debug an AI system.