Skip to content

AI, From the Atoms Up

How a trained model becomes a useful product, an agent, and eventually a governed system.

29 minute read Researched with AI Revised 27 September 2026

AI:infrastructure, language models, training, inference, retrieval, agents, multi-agent systems, evaluation, security, governance, organisations

The most useful fact in AI engineering is also the easiest to lose: the model is not the system. A model is a learned computational artifact. It becomes usable when an inference engine runs it; situated when an application constructs context around it; informed when retrieval brings in external knowledge; capable of action when a runtime exposes controlled tools; agentic when the model can repeatedly choose what to do next; and operational only when the surrounding system can survive failures, enforce authority, produce evidence, involve humans, and remain governable over time.

The familiar “model → agent” ladder is therefore necessary but incomplete. It is better represented as a technical spine crossed by production rails. The spine runs from compute and data through training, model artifact, serving, context, tools, agent, orchestration, and product. The rails—identity, security, memory/state, evaluation, observability, reliability, human control, and governance—cross several or all stages. They do not appear only after the agent is built.

“Above the agent” is not another kind of smarter model. It is a widening control scope: a workflow or orchestrator coordinates tasks; durable execution preserves progress through time and failure; a platform governs shared models, tools, memory, identities, and budgets; a product embeds the capability in a human process; and an organization assigns ownership, risk, audit, incident response, and accountability. At each step, the engineering problem becomes less about generating the next token and more about controlling state, authority, coordination, and consequence.

The recommendation, in one line. Name the boundary before discussing “the AI”: model, model call, workflow, agent, multi-agent system, product, platform, or governed organizational capability.

The whole system in one picture

The stack has a spine and rails. The spine answers “what is assembled on top of what?” The rails answer “what must remain true across the assembly?”

  1. Human goal and operating context
  2. Product and business process
  3. Workflow, orchestration, multi-agent coordination
  4. Agent loop and runtime
  5. Tools, permissions, action environment
  6. Context, retrieval, memory, state
  7. Inference service and model API
  8. Model artifact
  9. Data, training, post-training, evaluation
  10. Compute, storage, network, power
  1. 1 Security and identity
  2. 2 Evaluation and observability
  3. 3 Reliability and cost
  4. 4 Human control and governance

The arrows do not mean every product owns every lower layer. A developer using a hosted model API consumes the lower layers as a service. The responsibility does not disappear; it is split across providers, integrators, deployers, operators, users, and affected parties.

The boundary glossary

Many AI arguments are vocabulary collisions. The following terms are related, but not interchangeable.

TermWhat it isWhat it is not
Algorithm / architectureThe structure of a computation or learning methodThe learned numerical state produced by training
Weights / parametersLearned numbers controlling the model’s computationA searchable database of exact training facts
CheckpointSaved training state at a particular momentNecessarily the complete deployable package
Model artifactArchitecture/configuration, weights, tokenizer, precision, templates, metadataA complete end-user product
InferenceRunning a trained model on input to produce outputTraining, retrieval, tool execution, or workflow control
Model service/APIManaged access to inference over a software interfaceThe model itself or raw infrastructure
ContextInformation visible to the model for the current callPersistent memory or guaranteed attention
External knowledgeDocuments, databases, APIs, websites, or other sources outside weightsAutomatically available to the model
RAGA pipeline that retrieves and places selected external knowledge into contextA synonym for vector database
MemoryStored state that can later be selected and returned to context or control flowThe context window itself
ToolA typed interface to observe or affect an environmentPermission to do anything the interface could theoretically do
Harness/runtimeCode that constructs calls, exposes tools, applies policy, executes actions, and manages stateLearned intelligence
WorkflowCode-defined control flow through model calls, tools, and stepsNecessarily an autonomous agent
AgentA system in which the model can repeatedly direct parts of its process and tool use toward a goalMerely a chatbot, a memory store, or one tool call
OrchestratorA coordinator that decomposes, dispatches, joins, budgets, and terminates workNecessarily a model; it can be deterministic code
Multi-agent systemMultiple agent processes with coordination and communicationAutomatically more capable, reliable, or correct
ProductUser experience, application logic, data, operations, policy, and business process containing AI capabilityThe model name shown in marketing
AI management systemOrganizational policies, roles, processes, measurement, and improvement for responsible AI useA runtime framework or agent library

A language instruction is not a security control. The model may decide what it wants to do; code outside the model must decide what is actually allowed and execute it within a defined boundary.

Below the model: the intelligence factory

Compute is the physical floor

A model exists as a deployable capability only because a substrate can store and execute it.Everything under this floor, from data centers down to chips and physics, is the subject of All Roads Lead to Rock. That substrate includes accelerators, CPUs, host memory, high-bandwidth accelerator memory, storage, network/interconnect, cluster scheduling, power, and cooling. At small scale it may be one laptop or one GPU. At large scale it becomes a distributed system with data, tensor, pipeline, and expert parallelism; checkpoint distribution; failure recovery; and capacity planning.

The lower layer matters twice:

  1. Training needs enough compute, communication, storage throughput, and power to optimize the model.
  2. Inference needs enough memory capacity, bandwidth, concurrency, and scheduling efficiency to meet latency and throughput targets.

ISO’s public AI overview frames the foundation as a triad of data, compute infrastructure, and algorithms. Removing any corner changes what can be trained and served.

Data is a governed system, not raw fuel

The data branch expands into:

  • acquisition and licensing;
  • consent, privacy, and rights;
  • provenance and lineage;
  • parsing and normalization;
  • filtering, quality scoring, and deduplication;
  • language, domain, and demographic coverage;
  • labeling, preference data, and feedback;
  • synthetic-data generation and validation;
  • train/validation/test partitioning and contamination control;
  • versioning, retention, deletion, and audit.

The ISO/IEC 5259 series treats data quality as its own discipline across measurement, management, process, and governance. This is why “the same architecture” can produce very different models: the training distribution and its treatment are part of the learned result.

Training is several pipelines, not one event

  1. Acquire and curate data
  2. Tokenize or encode
  3. Pretraining objective
  4. Base checkpoints
  5. Post-training and alignment
  6. Capability and safety evaluation
  7. Release candidate
  8. Model artifact
failure or gap

For a language model, pretraining commonly learns through a self-supervised prediction objective. Post-training may then include supervised demonstrations, preference optimization, reinforcement learning with human or verifiable feedback, safety training, tool-use training, distillation, or domain adaptation. The InstructGPT paper is a clear early example of supervised instruction tuning followed by preference-based reinforcement learning.

The scaling literature also corrects the folk story that model size is the only variable. Scaling Laws related loss to data, parameters, and compute; Chinchilla showed that balanced scaling of parameters and training tokens mattered under a fixed compute budget. Architecture, data, compute allocation, optimization, and post-training jointly shape capability.

What exactly is “the model”?

The Transformer established the attention-based architecture underlying most modern LLMs, but a runnable artifact still has several parts:

Artifact partJobTypical mismatch symptom
Architecture/configDefines layers, dimensions, attention, experts, position schemeWeights cannot load or produce nonsense
WeightsEncodes the learned numerical stateDifferent capability/behavior despite same architecture
Tokenizer/vocabularyConverts inputs and outputs between symbols and model IDsBroken text, wrong token counts, incompatible special tokens
Precision/quantizationChooses numerical representation and memory/performance tradeoffQuality loss, unsupported kernels, memory overflow
Chat/generation templateSerializes roles, messages, and special markersPoor instruction following or malformed turns
Model documentationRecords intended uses, limits, tests, provenanceDownstream misuse and untraceable assumptions

This boundary lets us say precisely what changes:

ChangeUpdates learned weights?Can materially change behavior?
System prompt or contextNoYes
Retrieval/index/rerankerNoYes
Tool set or permissionNoYes, including real-world consequences
Workflow or agent loopNoYes, especially multi-step reliability
LoRA/adapter/fine-tuneYes, through added or modified trained parametersYes
Continued pretrainingYesYes
New architecture or pretraining runYes, fundamentallyYes

LoRA is a useful example: it freezes the base model and trains small low-rank adapter parameters. The product can also change dramatically without any weight update at all.

Running the model: inference and serving

A model call has its own pipeline

ApplicationModel gatewaySchedulerInference engine
request, context, model, limits
authenticate, authorize, quota, policy
tokenized request
batch and route
prefill input and create KV state
loop: decode
stream generated token or chunk
usage, finish reason, trace metadata

The engine usually performs a highly parallel prefill over the input, then an autoregressive decode in which each generated token depends on prior state. The KV cache stores attention state so previous tokens need not be recomputed on every decode step. Queue time, tokenization, prefill, first-token latency, per-token latency, network time, and total completion time are different measurements.

Sampling is where probabilities become one output

The model produces scores over possible next tokens. A decoding policy turns that distribution into a selected token using techniques such as greedy choice, temperature scaling, top-k/top-p sampling, penalties, constrained decoding, or structured-output grammars. “The same model” can therefore produce different outputs under different generation settings.

Serving is a systems problem

Production serving adds:

  • request routing and model selection;
  • admission control and rate limits;
  • batching and continuous scheduling;
  • KV-cache allocation and reuse;
  • prefix/prompt caching;
  • streaming;
  • quantization and optimized kernels;
  • speculative decoding;
  • multi-GPU placement and autoscaling;
  • overload behavior, cancellation, and fallback;
  • usage accounting and cost allocation.

PagedAttention and vLLM showed how better KV-cache memory management and scheduling could raise throughput by 2–4x in the paper’s studied configurations. Speculative decoding demonstrated 2–3x acceleration in its experiments by letting a cheaper model propose tokens that the target model verifies. These are not changes to the user’s prompt or business logic; they are changes to how inference is executed.

MLPerf Inference separates Offline, Server, Interactive, SingleStream, and MultiStream scenarios. That taxonomy exposes why a single “tokens per second” number is insufficient.

MetricWhat it answers
Time to first token/chunkHow quickly does useful output begin?
Inter-token latencyHow smoothly does output continue?
End-to-end latencyHow long until the requested result is complete?
ThroughputHow much work completes per time unit?
Concurrency/tail latencyWhat happens when many requests arrive together?
Utilization/energy/costWhat resources were consumed per useful outcome?
Quality under optimizationDid quantization, routing, or caching alter acceptable behavior?

A hosted model API is not IaaS

NIST SP 800-145 formally defines SaaS, PaaS, and IaaS. “Model as a Service” is useful descriptive vocabulary, but not one of those canonical NIST models. A hosted inference API is clearly not IaaS: the consumer does not manage virtual machines, the operating system, accelerator drivers, weights, or serving engine. Depending on the offering it behaves more like a managed application capability or a platform building block.

Constructing the model’s situation

Context is what is visible now

The application, not the model alone, builds the working desk:

  1. governing instructions and policy
  2. application/developer instructions
  3. current user input
  4. selected conversation history or summary
  5. retrieved documents and provenance
  6. tool definitions
  7. tool results and observations
  8. retrieved memory
  9. plans or structured task state
  10. output format/schema

model-visible context for this call

A large context window is capacity, not persistent memory and not guaranteed use. Lost in the Middle found that some models used relevant information less reliably depending on where it appeared in long input. Context engineering is therefore a selection and ordering problem, not merely a token-maximization exercise.

Four kinds of “knowledge” must stay separate

KindWhere it livesHow it becomes available
Parametric model knowledgeDistributed through learned weightsActivated by the current input; not exact database lookup
Current contextRequest-visible token sequenceAssembled by the application now
External knowledgeDocuments, databases, search, APIsRetrieved or queried through software/tools
Persistent memoryStored records about prior state/experienceSelected by a memory policy and placed back into context/control flow

RAG is an information system

The original RAG paper combined parametric and external non-parametric memory. A production implementation expands into a full pipeline:

  1. Sources
  2. Parse, normalize, chunk
  3. Provenance, metadata, access policy
  4. Index and embed
  5. User task → Query understanding
    Retrieve candidates
  6. Filter and rerank
  7. Pack context
  8. Generate grounded answer
  9. Citation, evaluation, feedback
refresh or correction

Each stage can fail independently. The right document may not be ingested; a chunk may lose context; the index may be stale; retrieval may miss; authorization may filter incorrectly; reranking may select noise; the model may ignore good evidence; citations may not support the claim.

Memory is storage plus policy

The memory basket is larger than “vector store”:

Memory/state typeExamplePrimary risk
Working stateCurrent plan, completed steps, open questionsContext overflow or corrupted task state
Conversation stateMessages or summaries in one threadLossy summaries, role confusion
Episodic memory“The last deployment failed because…”Stale or false recollection
Semantic memoryProject facts, user preferencesConflict, leakage, overgeneralization
Procedural memorySkills, playbooks, operating rulesMalicious or outdated instruction
External knowledgeHandbook, codebase, databaseFreshness, authorization, provenance
Durable execution stateTask status, artifacts, idempotency keysDouble execution or unrecoverable partial work

MemGPT drew an operating-system analogy for managing memory tiers beyond the context window. Generative Agents used observation records, retrieval, reflection, and planning. Both reinforce the architectural point: useful memory needs decisions about what to write, consolidate, retrieve, expose, correct, expire, and delete.

Tools and the harness: intention becomes execution

Two actors are always present

ModelHarness/runtimePolicy and identityTool/environment
propose tool and arguments
validate schema and intent
authorize this actor, resource, action, context
allow, require approval, or deny
execute bounded operation
typed result, receipt, or error
observation safe for model context

The model generates a proposal. The harness owns the side effect. This split is the foundation of safe tool use.

Tools are more than read versus write

NIST’s agent-tool workshop taxonomy provides a stronger external anchor:

PurposeTool basket
Perceptiondatabases, monitoring, diagnostics, GUI, voice, internet search, physical sensors
Reasoning supportplanning, decomposition, pathfinding, scratchpads, calculators, simulations, resource management
Actionauthentication, computer use, code execution, software/API extensions, physical devices, human communication, agent interaction

The same tool must also be classified by access pattern, risk, reliability, modality, observability, autonomy, reversibility, and whether its environment is trusted. A browser reading public pages is different from a browser logged into payroll. A shell in an ephemeral sandbox is different from root access on a production host.

Permissions should express verbs and consequences

  1. discover
  2. read
  3. propose
  4. preview
  5. approve
  6. execute
  7. verify
  8. reverse

A mature system separates these permissions. “Can access calendar” is too coarse; reading availability, drafting an invitation, sending it, cancelling it, and changing attendee permissions have different consequences.

MCP standardizes capability access, not trust

The current MCP 2026-07-28 specification standardizes how hosts, clients, and servers expose resources, prompts, tools, elicitation, and optional extensions. The current core uses stateless, self-contained JSON-RPC requests; the Tasks extension supports long-running operations with durable handles and mid-flight input.

MCP belongs primarily between the application/agent runtime and capabilities. It does not decide whether a tool is appropriate for a user, does not eliminate prompt injection, and does not replace application authorization or business policy.

Where agency begins

Not every multi-step AI system is an agent

Anthropic’s production guidance draws a useful line:

  • Workflow: predefined code controls how models and tools are orchestrated.
  • Agent: the model dynamically directs parts of its own process and tool use.
Direct model call code path
Prompt chain / router workflow
Tool-enabled agent model-led
Durable multi-agent system distributed

The bars show increasing control complexity, not quality or maturity.

Common code-controlled workflows

  • Prompt chaining: one model step prepares input for the next.
  • Routing: code or a classifier chooses a specialized path.
  • Parallelization: independent model/tool calls run concurrently and are merged.
  • Orchestrator-workers: a coordinator decomposes work and dispatches workers.
  • Evaluator-optimizer: one step creates, another critiques, and the loop stops by a defined rule.

These patterns can be powerful, testable, and easier to govern than an open-ended agent.

Agent anatomy

ReAct popularized interleaving reasoning and action. A production-grade anatomy is broader:

  1. goal and success criteria
  2. model and instructions
  3. context construction
  4. perception/tools
  5. action tools
  6. working state
  7. plan or policy for choosing the next step
  8. budget: time, tokens, money, actions
  9. verification and error recovery
  10. stopping and escalation conditions
  11. optional persistent memory

agent runtime

The canonical loop is:

Goal

Observe Decide or plan Act through runtime Observe result Update state and verify Done, blocked, unsafe, or continue?
  • continue Observe
  • done Return outcome and evidence
  • needs intent or approval Escalate to human
  • unsafe or budget exhausted Stop safely

Without a budget and a stopping rule, an agent loop is an unbounded process. Without external verification, a fluent final message is not evidence that the goal was achieved.

Agency, autonomy, capability, and authority

DimensionQuestionExample
CapabilityHow well can it perform?Can it debug a distributed failure?
AgencyCan it choose and sequence actions toward a goal?Can it inspect, edit, test, and iterate?
AutonomyHow independently may it operate?Must it ask at every step or only at key gates?
AuthorityWhat resources and consequences may it reach?Read repo, merge code, deploy, spend money?
DurationHow long can work continue?One turn, hours, persistent background service?
ScopeHow broad is the delegated objective?Fix one test versus run engineering operations?

Levels of Autonomy for AI Agents describes user roles from operator through collaborator, consultant, approver, and observer. The key engineering insight is that autonomy can be calibrated independently of capability and environment. A highly capable model can remain read-only and approval-bound.

What sits above an agent

This branch often gets only one name, “production system.” It contains several distinct coordination scopes.

Workflow and orchestrator

An orchestrator decides how work is decomposed and recombined. It may be deterministic code, a model-directed supervisor, or a hybrid.

Responsibilities include:

  • task decomposition and dependency graph;
  • routing to models, tools, services, humans, or agents;
  • concurrency and priority;
  • budget allocation;
  • shared-state boundaries;
  • join/merge and conflict resolution;
  • retries, fallback, and escalation;
  • termination and acceptance criteria.

The orchestrator is “above” the agent only when it coordinates one or more agents. Inside a single agent, similar logic may be part of its runtime.

Durable execution

Reasoning is not durability. A long-running task must survive process restarts, network failures, timeouts, and human delays. Temporal’s workflow documentation is one concrete implementation of event history and replay; the general responsibilities are platform-independent:

  • persist state transitions and artifacts;
  • make mutation idempotent or deduplicated;
  • retry with bounded policy and deadlines;
  • wait for timers, external events, or approvals;
  • support cancellation and cleanup;
  • compensate for partially completed side effects;
  • resume after worker or machine failure;
  • version workflows without corrupting in-flight work.
User or eventDurable workflowAgentTool/serviceHuman approver
start task
bounded subgoal
propose and execute allowed work
transient failure
record state, backoff, retry
artifact and evidence
request consequential approval
approve later
idempotent mutation
completed outcome

Multi-agent coordination

Multiple agents can add specialization, independent perspectives, parallelism, or containment. They also add communication overhead, duplicated work, correlated failure, conflicting goals, false consensus, shared-memory hazards, and a larger security surface.

TopologyShapeUseful whenMain failure
Supervisor-workersOne delegates and mergesTasks decompose cleanlyBottleneck or biased supervisor
HierarchyDelegation treeLarge nested programsContext loss and accountability diffusion
Peer handoffAgents transfer ownershipWork crosses specialtiesDropped or looping handoffs
BlackboardShared workspaceAgents build on common artifactsPoisoned/stale shared state
PipelineFixed specialist stagesStable repeatable processError propagation and rigidity
Debate/committeeCompeting proposals and judgeIndependent critique mattersCorrelated beliefs and false consensus
Market/auctionCapability/cost-based allocationDynamic heterogeneous supplyGaming, opaque incentives, coordination cost
SwarmLocal rules, emergent coordinationSimulation or highly parallel explorationUnpredictability and weak global guarantees

Why Do Multi-Agent LLM Systems Fail? reported 14 failure modes grouped under specification/system design, inter-agent misalignment, and verification/termination. “Add more agents” is not a reliability strategy.

A2A and agent opacity

The current A2A 1.0 specification provides discovery through agent cards, message and artifact exchange, task lifecycle, streaming, asynchronous updates, cancellation, and standard authentication schemes. It lets independent agents collaborate without revealing their internal tools, memory, or implementation.

The boundaries are complementary:

InterfacePrimary relationship
MCPApplication/agent runtime to resources and tools
A2AIndependent agent system to agent system
Durable workflow/event busCoordination across time, services, and failures
API/RPCOrdinary typed service-to-service operation
UI/approval surfaceHuman to system decision and control

Agent platform and control plane

Once several products or teams share agents, the “above-agent” layer becomes a platform:

  1. model gateway and routing
  2. tool registry and secure execution gateway
  3. knowledge and memory services
  4. agent registry and versioning
  5. identity, delegation, and policy engine
  6. sandbox and secrets broker
  7. workflow/durable task service
  8. budget, quota, rate, and cost controls
  9. trace, evaluation, and audit plane
  10. deployment, rollout, rollback, and kill controls

This resembles a service platform more than a prompt library. Its most important outputs are consistent boundaries and evidence.

Product, process, and institution

The product layer decides what the user can ask, see, approve, interrupt, correct, and appeal. The business-process layer decides where records live, who owns exceptions, how separation of duties works, and what counts as completion. The organizational layerWhy organizations grow these organs at all, at every scale from cells to states, is the subject of Functional Anatomy of Systems. inventories systems, assigns risk owners, reviews suppliers, trains operators, monitors impacts, handles incidents, and decides when a capability should be changed or retired.

ISO/IEC 42001 calls this an AI management system. ISO/IEC 42005 adds lifecycle impact assessment. These are genuinely above any individual agent because they govern the organization’s relationship with an AI portfolio.

The production rails

Reliability rail

Agentic systems inherit ordinary distributed-systems failure modes and add probabilistic decision errors.

ControlWhy it exists
Deadlines and timeoutsPrevent dependencies or loops from consuming unbounded time
Bounded retries with backoff/jitterRecover transient failures without synchronized overload
Idempotency and deduplicationMake repeated mutation safe after ambiguous outcomes
Admission control and concurrency limitsKeep demand inside capacity
Backpressure and load sheddingPrevent queues and stale work from becoming a collapse loop
Circuit breakers and isolationContain failing dependencies
Cancellation and cleanupStop work whose user or workflow no longer needs it
Checkpoint/replayRecover progress after process or machine failure
CompensationReverse or offset partial multi-step side effects
Fallback/degradationPreserve a safe smaller capability during failure
Version pinning and lineageReproduce which model, prompt, tool, policy, and data acted
Canary, rollback, incident responseLimit and repair change-induced regressions

Google’s secure and reliable systems guidance emphasizes idempotency and understandability because distributed operations may be retried after an outcome becomes ambiguous. Agent tools that send, purchase, delete, deploy, or publish need the same discipline.

Evaluation rail

Model benchmarks are only the first layer of evidence.

Evaluation levelCentral questionExample evidence
ModelCan the model perform relevant primitives safely and efficiently?Capability, calibration, robustness, bias, toxicity, efficiency
Context/promptDoes the call follow instructions and use supplied evidence?Schema validity, grounding, long-context tests
RetrievalDid the right authorized evidence reach context?Recall, ranking, freshness, citation support
ToolWas the correct operation called safely?Selection, arguments, authorization, side-effect receipt
MemoryIs remembered state useful, isolated, correct, and deletable?Recall, conflict, poisoning, deletion tests
TrajectoryWas the process sound?Plan, step necessity, recovery, budget, stopping, escalation
Task/systemDid the real goal complete under realistic conditions?End-to-end success, latency, cost, failure recovery
HumanDid the system improve outcomes without miscalibrated trust or harm?Effectiveness, effort, satisfaction, accessibility, recourse
Operational/businessDoes deployment remain valuable and governable?Incident rate, containment, process quality, downstream impact

HELM made the case for multi-scenario, multi-metric model evaluation. AgentBench moved evaluation into interactive environments; SWE-bench required real repository edits against issue-level tasks. The August 2026 NIST TEVV-Athlon initial public draft explicitly targets customizable assessment across models and agentic systems. It is promising but still a draft, not a final standard.

Verification asks whether the build meets specified requirements. Validation asks whether the resulting system works for the intended use and context. Production monitoring asks whether those claims continue to hold after deployment.

Observability rail

The emerging OpenTelemetry GenAI conventions distinguish model operations, planning, tool execution, agent invocation, and workflow invocation. That hierarchy matches the system boundary:

  • workflow trace
    • agent invocation
      • plan
      • model call
      • tool execution
      • verification
    • human approval wait
    • final side effect and receipt

Useful telemetry includes trace/task IDs, model and component versions, token/cache usage, latency by stage, tool authorization and result, state transitions, retries/cancellations/approvals, termination reason, evaluation results, cost/budget, and artifact lineage. Prompt, completion, retrieved content, and tool arguments may contain secrets or personal data; full-content logging must be an explicit governed choice.

Security rail

The OWASP Top 10 for Agentic Applications 2026 provides a current failure anchor:

IDRiskPrimary boundary
ASI01Agent goal hijackInstructions, external content, intent integrity
ASI02Tool misuse and exploitationTool design, validation, policy, consequence
ASI03Identity and privilege abuseAuthentication, delegation, least privilege
ASI04Agentic supply-chain vulnerabilityModels, data, tools, plugins, protocols, dependencies
ASI05Unexpected code executionSandbox, interpreter, shell, generated code
ASI06Memory and context poisoningRetrieval, memory writes, provenance, isolation
ASI07Insecure inter-agent communicationAgent identity, message integrity, authorization
ASI08Cascading failuresMulti-step and multi-agent error propagation
ASI09Human-agent trust exploitationUX, explanation, approval quality, overtrust
ASI10Rogue agentsMisalignment, concealment, uncontrolled action

Security principles:

  • authenticate users, services, tools, and agents where relevant;
  • authorize consequential actions outside the model;
  • grant least privilege, least duration, and least agency;
  • keep credentials and secrets out of model-visible text;
  • sandbox code and computer use;
  • carry provenance and trust labels with external content;
  • validate tool schemas, arguments, results, and side-effect receipts;
  • separate read, propose, approve, and execute;
  • prefer reversible or compensatable operations;
  • inventory and verify models, adapters, data, tools, and dependencies;
  • preserve audit evidence and define containment, kill, rollback, and incident response.

NIST AI 600-1 expands the risk view beyond attacks to confabulation, privacy, bias/homogenization, information integrity, IP, environmental impact, human-AI configuration, and component/value-chain integration. NIST SP 800-218A applies secure-software-development practices to model producers, AI-system producers, and acquirers.

The human and organizational system

Start from the human activity, not the model technique

NIST AI 200-1 enumerates 16 domain-independent human-AI activities:

ClusterActivities
Create and transformContent creation, content synthesis
Judge and anticipateDecision making, detection, prediction, recommendation
Find and understandDiscovery, image analysis, information retrieval/search
Assist and adaptDigital assistance, personalization, performance improvement
Operate and automateMonitoring, process automation, robotic automation, vehicular automation

This prevents technique-first design. A recommendation system and a decision-making system may use the same model but need different authority, explanation, evaluation, and recourse.

Human oversight is a set of decision rights

“Human in the loop” is too vague. Oversight may happen:

  • Before work: scope, tools, data, budget, and policy selection.
  • During planning: preview, edit, or reject the plan.
  • Before mutation: require approval for consequential or irreversible action.
  • During execution: pause, cancel, redirect, or narrow scope.
  • At uncertainty: agent asks for missing intent or domain judgment.
  • After execution: verify outcome, audit evidence, correct, reverse, or appeal.
  • At system level: change permissions, model, policy, workflow, or deployment.

The right gate depends on consequence, reversibility, uncertainty, time pressure, affected parties, and the human’s actual ability to judge. A confirmation button on an unreadable plan does not create meaningful control.

Governance is continuous system ownership

The NIST AI RMF core wraps the lifecycle in four functions:

Govern roles, policy, risk tolerance Map context, purpose, people, impact Measure test, evaluate, monitor Manage prioritize, treat, respond

An organization operating AI needs at least:

  • a current system and dependency inventory;
  • named business, technical, data, security, and risk owners;
  • documented intended use, prohibited use, limits, and affected parties;
  • supplier and model/tool/data due diligence;
  • pre-deployment evaluation and approval evidence;
  • production monitoring, incident response, redress, and rollback;
  • change control for model, prompt, retrieval, tool, workflow, and policy updates;
  • operator training and user communication;
  • impact assessment and periodic review;
  • retirement, data deletion, and record-retention processes.

This is the final conceptual leap: an AI capability becomes an institution when other people depend on its decisions and actions. The institution needs ownership and repair mechanisms, not only better prompts.

What to do with the map

Teach the system as boundaries, not a buzzword ladder

Use one reconstruction sentence:

A company uses compute and data to train a model; serves the resulting artifact through an inference system; constructs context, knowledge, and memory around each call; exposes controlled tools through a runtime; lets a model direct repeated action when agency is needed; coordinates longer or multi-party work through durable workflows; and surrounds the whole capability with product design, evidence, security, operations, human control, and governance.

Ask seven questions whenever someone says “the AI”

  1. What human goal and activity is being served?
  2. Which behavior comes from the model, and which from code, context, data, or tools?
  3. Who controls the next step: human, deterministic workflow, or model?
  4. What may the system read, propose, approve, or change?
  5. What state must survive, and how is it recovered, corrected, or deleted?
  6. What evidence proves the components, trajectory, and full system work here?
  7. Who owns the outcome, monitors the system, and can stop or repair it?

Earn complexity one constraint at a time

  1. direct call
  2. add retrieval only when external knowledge is required
  3. add tools only when the system must observe or act
  4. add a workflow when the repeatable path is known
  5. add an agent when the path cannot be predetermined
  6. add multiple agents only when specialization or parallelism beats coordination cost
  7. add durable orchestration when work must survive time and failure
  8. add a platform and management system when capability becomes shared or consequential

Treat “above agent” as control, not more intelligence

Do not search for a mystical noun above “agent.” The practical progression is:

  1. agent
  2. orchestrated task graph
  3. durable multi-agent/service workflow
  4. shared agent platform and control plane
  5. human-facing product and business process
  6. governed organizational capability

Each level must add explicit state, authority, evidence, ownership, and repair.

Keep the map maximal and the build selective

The complete tree in the appendix is an atlas, not a backlog. Most applications should not build a multi-agent platform, durable memory hierarchy, marketplace, and governance office on day one. Keep dormant and niche branches visible, then activate only what the use case, consequence, and operating scale require.

The stopping rule. If a direct call or deterministic workflow can meet the goal with better predictability, do not add an agent merely to make the architecture sound advanced.

Appendix

The complete tree

Status labels: Live common and operational now; Emerging real but immature or uneven; Niche important outside the mainstream path; Dormant visible but not a default action branch.

Wing A — intelligence production

  • Live Compute substrate
    • accelerators, CPU, host memory, high-bandwidth memory;
    • local and distributed storage;
    • network/interconnect and cluster scheduling;
    • power, cooling, capacity, hardware supply chain.
  • Live Data system
    • acquisition, rights, privacy, provenance;
    • quality, filtering, deduplication, labeling;
    • synthetic and preference data;
    • partitioning, contamination, versioning, lineage.
  • Live Learning pipeline
    • objective, optimization, distributed training;
    • checkpointing, evaluation, post-training;
    • safety, distillation, adaptation, release.
  • Live Model artifact
    • architecture, weights, tokenizer, configuration;
    • precision/quantization, templates, documentation.
  • Niche here Other model families
    • diffusion, state-space, graph, probabilistic, symbolic;
    • classical ML, control, reinforcement learning.

Wing B — application assembly

  • Live Inference serving
    • routing, batching, prefill/decode, KV cache;
    • sampling, streaming, caching, quota, gateway.
  • Live Context engineering
    • instructions, history, retrieval, tool schemas/results;
    • state selection, ordering, compression, output contracts.
  • Live Knowledge and RAG
    • ingestion, index, retrieval, ranking;
    • permissions, grounding, citation, freshness.
  • Emerging Memory
    • working, episodic, semantic, procedural, durable execution;
    • write, consolidate, retrieve, correct, expire, delete.
  • Live Multimodal system
    • text, image, audio, video, sensor input and generated output.

Wing C — action and agency

  • Live Tool runtime
    • discovery, schema, validation, policy, execution, result, audit.
  • Live Deterministic workflow
    • chain, route, parallelize, evaluator, orchestrator-worker.
  • Live Single-agent loop
    • goal, observe, plan, act, update, verify, stop/escalate.
  • Emerging Durable long-running agent
    • persisted state, timers, recovery, cancellation, human approval.
  • Emerging Multi-agent system
    • supervisor, hierarchy, peers, blackboard, debate, market, swarm.
  • Dormant/high-risk Self-modifying or replicating agent
    • controlled self-improvement belongs under strict change management;
    • unconstrained self-replication is not a default production pattern.

Wing D — above-agent coordination

  • Live Orchestrator
    • decomposition, dispatch, dependency, budget, merge, termination.
  • Live Durable workflow engine
    • event history, retries, idempotency, compensation, async waits.
  • Emerging Capability and agent registry
    • discovery, ownership, health, policy, version, compatibility.
  • Emerging Interoperability
    • MCP, A2A, API/RPC, events, identity and delegation.
  • Emerging Agent platform/control plane
    • model/tool/knowledge gateways, policy, sandbox, budget, trace, deploy.
  • Niche/dormant Agent market/economy
    • machine identity, negotiation, payment, reputation, dispute, liability.

Wing E — product, people, organization

  • Live User experience and control
    • intent, progress, preview, explanation, approval;
    • cancel, correction, rollback, recourse.
  • Live Business process
    • case state, records, separation of duties, exceptions, accountability.
  • Live Operations
    • SLOs, observability, incident response, capacity, cost, rollout/rollback.
  • Incomplete Assurance
    • model and component tests, journey tests, red teaming, monitoring, audits.
  • Mandatory Security and privacy
    • identity, privilege, isolation, provenance, supply chain, containment.
  • Emerging/live Governance and management system
    • inventory, ownership, risk, impact, procurement, change, compliance.
  • Often omitted Societal and environmental impact
    • affected non-users, labor/process change, access, energy and externalities.

Niche shelf

  • Embodied agents and autonomous vehicles: perception, real-time control, physical safety, simulation, certification, and fail-safe behavior dominate.
  • Edge/on-device agents: intermittent connectivity, constrained power/memory, privacy, local models, and hardware diversity dominate.
  • Neuro-symbolic and formal systems: learned perception or generation is combined with rules, solvers, proof, planning, or verification.
  • Non-language multi-agent systems: robotics, games, operations research, economics, and control have older coordination traditions than LLM agents.
  • Human-agent organizations: incentives, labor design, authority, institutional memory, and power matter more than prompt syntax.
  • Privacy-preserving/federated/confidential AI: compute and data boundaries are redesigned around confidentiality and distributed ownership.
  • Agent economies: machine identity, delegated payment, contract, reputation, liability, and dispute resolution remain immature.
  • Controlled self-improvement: agents may propose changes to prompts, tools, memory policy, code, or adapters, but evaluation and approval must govern promotion.

Residual bucket

This research does not descend fully into semiconductor manufacturing, every learning paradigm, diffusion-first media systems, robotics safety standards, formal verification, every regional law, the philosophy of intelligence, or speculative AGI/ASI governance. These are real branches outside the chosen LLM-application depth. A map claiming no residue would be dishonest.

Method and scope

This research started from a model-to-agent study note, then widened it against external taxonomies so the surrounding system would not disappear behind the model. The primary anchors were ISO/IEC 23053 for generic ML-system components, ISO/IEC 5338 for lifecycle processes, the OECD classification framework and NIST AI RMF for applied-system dimensions and actors, NIST AI 200-1 for human-AI activities, NIST’s agent-tool taxonomy, and the OWASP Agentic Top 10 for 2026.

The map was then rotated by technical layer, lifecycle, source of control, failure mode, actor/counterparty, and deployment form. The result is deliberately scoped to modern LLM-centric application systems. Non-LLM AI, robotics, formal methods, semiconductor production, and jurisdiction-specific law remain visible on the niche and residual shelves but do not receive equal depth.

Scope discipline. This is a total map of the LLM-centric application system, not a claim to contain every branch of artificial intelligence as a scientific field.

Seed material. A model-to-agent study note, written 2026-08-28. The seed supplied the original distinctions and teaching metaphors. It was treated as a research seed, not as a source of external factual authority.

External anchors. ISO/IEC 22989:2022 terminology; ISO/IEC 23053:2022 ML-system framework; ISO/IEC 5338:2023 lifecycle processes; ISO/IEC 42001:2023 AI management systems; ISO/IEC 42005:2025 impact assessment; ISO/IEC 5259 data quality; OECD AI system classification; NIST AI RMF 1.0 lifecycle, actors, and Govern/Map/Measure/Manage; NIST AI 200-1 human-AI activities; NIST agent-tool taxonomy; NIST AI 600-1; NIST SP 800-218A; NIST TEVV-Athlon initial public draft.

Technical papers and primary documentation. Transformer; Scaling Laws; Chinchilla; InstructGPT; LoRA; RAG; Lost in the Middle; MemGPT; Generative Agents; Toolformer; ReAct; PagedAttention/vLLM; speculative decoding; HELM; AgentBench; SWE-bench; multi-agent survey and failure taxonomy; Anthropic agent architecture and trustworthy-agent guidance; MCP 2026-07-28; A2A 1.0; OpenTelemetry GenAI conventions; Temporal durable execution; MLPerf Inference; Google secure and reliable systems guidance; OWASP Agentic Top 10 2026.

Generators walked. (1) vertical stack, (2) lifecycle, (3) source of control, (4) failure surface, (5) actor/counterparty, and (6) deployment form. Their union created the spine-and-rails model and the four distinct scopes above a single agent.

Anchor diff result. The seed covered the conceptual core from model through memory. The largest misses were data and serving depth, workflow-versus-agent control, durable execution, multi-agent coordination and protocols, product/human activity, layered assurance, full agentic security, and organizational governance.

Caveats. ISO public pages summarize standards whose full normative text may require purchase. OpenTelemetry GenAI agent conventions remain under development. NIST TEVV-Athlon is an initial public draft as of August 2026. MCP and A2A are rapidly evolving protocols. Vendor architecture guidance is used as practitioner evidence, not as a universal standard. This research intentionally prioritizes LLM-centric application systems and labels other AI traditions as niche or residual rather than pretending equal coverage.

Audit line. Anchored against external lifecycle, system, tool, use, risk, telemetry, and interoperability taxonomies; six generators rotated; the supplied “agent” example expanded into workflow, durable execution, multi-agent topology, platform, product, and institution; niche shelf populated; residual declared. Coverage is standard-depth totality for the scoped LLM-centric system.

What the original draft had, and what was added
SurfaceSeed noteThis research
Model, weights, checkpointStrong foundationAdded artifact package and adaptation boundaries
Training and inferenceClear distinctionAdded data lifecycle, scaling, post-training basket, serving mechanics
API/serviceClear intuitionCorrected MaaS as useful, not canonical NIST service taxonomy
Context and RAGCorrect basicsAdded construction pipeline, long-context limits, full RAG lifecycle
Tools and permissionsStrong conceptual splitAdded NIST tool taxonomy, authority dimensions, MCP boundary
Agent loopCorrect coreAdded workflow distinction, anatomy, budgets, stopping, autonomy dimensions
MemoryCorrect separationAdded memory types and write/retrieve/correct/delete policy
Above the agentNamed only as productionAdded orchestrator, durability, multi-agent, A2A, platform, product, institution
Evaluation and observabilityIntroducedExpanded into component, trajectory, task, human, operational evidence
Reliability and securityPartialAdded distributed-systems controls and complete OWASP agentic categories
Human use and governanceThinAdded NIST 16 activities, oversight rights, RMF loop, ISO management/impact

The source was not wrong. It had the right skeleton. The expansion adds the connective tissue that makes the skeleton usable in production.