Skip to content

L1 · Application#

conceptstub

What it is#

The workload the cluster exists to run: training jobs, inference services, RAG pipelines, agents, and HPC codes. This is also where the AI back end meets the conventional web front end.

Why it matters#

Every other layer is sized from here. A latency target for a chatbot, a deadline for a training run, or a request rate for an API translates directly into GPU count, memory, network and scheduling requirements. Getting the application's requirements wrong means building the wrong platform.

Agents and tools (MCP)#

The Model Context Protocol (MCP) is an open protocol that connects a model or agent to tools and data: databases, APIs, file systems and other services. It lives at L1 because it describes what the application talks to, not how GPUs compute or communicate.

Part What it is Where it runs
Agent / MCP client The application loop that calls the model and decides which tool to use Application pods, web-style
MCP servers Small services that expose tools and data to the agent Ordinary pods, usually no GPU
Model endpoint The LLM the agent reasons with Inference service (L2 serving engines down to L9)

How agents change the load on the stack below:

  • One request becomes many model calls. Each step of an agent is a separate inference call, separated by tool waits, so latency adds up across the chain.
  • Prompts share long prefixes. System prompts, tool definitions and conversation history repeat across calls, which makes KV-cache reuse and prefix-aware routing valuable.
  • Traffic is bursty and long-lived. Sessions stay open for minutes, so autoscaling, rate limits and per-tenant quotas move to the control plane.
  • Security moves to the tool boundary. MCP servers act with real credentials, so authentication, authorization and audit of tool calls become part of the design.

See the Agentic AI workload.

Planned content

  • How applications express their infrastructure needs (requests, queues, SLOs)
  • Front end (web) vs back end (AI stack) responsibilities
  • Reference agent architecture: gateway, agent runtime, MCP servers, model endpoints
  • Link-out to the workload pages for each type