Multi-Agent Topologies

Notebook Summary#

Hand-copy this. Telegraphic by design — every line is expanded in Section 2.

Detailed Study Notes#

A support agent that worked in the demo and quietly rotted in production — and the architecture that fixes it.

Picture the agent from the last session. One language model, one long system prompt. It masks personal data, spots prompt-injection attempts, classifies the ticket, calls the CRM, formats a clean reply. In the demo it looked great. Then the knowledge base grew, three more tools got added, and one day it started issuing refunds when a customer merely asked about a bill. Nobody changed the refund logic. The prompt still said “do not issue refunds during classification.” It broke anyway — on maybe three requests in a hundred. That failure is the whole reason this lesson exists.

2.1 · The monolith and why it’s an anti-pattern#

A monolithic agent is a single LLM call, carrying one system prompt, made responsible for every job in the workflow. Teams reach it naturally: build a proof-of-concept with one model and a long prompt, then keep bolting on responsibilities. Need billing? Add billing instructions. Need inventory? Add inventory instructions. Need security? Add security rules. The prompt accretes duties until it becomes what practitioners call a god agent.

The core distinction: a monolithic Python module that has a bug is deterministically wrong — the bug fires every time, so you find it and fix it. A monolithic LLM agent is stochastically wrong — it fires the same bug with some probability on every call, and that probability climbs with scale, load, and time. You can pass every test and still ship a system that misbehaves 3% of the time in the wild.

2.2 · Context dilution — why long prompts stop working#

Context dilution is the failure where a model’s attention is spread so thinly across a long prompt that critical instructions receive almost no weight. LLMs are attention-based: they can attend to any piece of the context window, but that attention is a finite budget divided among everything present. A tight 500-token prompt gives the model sharp resolution on every rule. Stretch the same rules into a 10,000-token prompt and the instructions near the edges become, in practice, invisible.

[!] Treat the exact 1/k² law as an illustrative heuristic for the intuition, not a precise universal constant — real attention profiles depend on the model. The direction is what matters: longer prompt ⇒ edge instructions lose weight.

The real-world symptoms are exactly what you’d predict from attention collapse: an agent that issues refunds on a billing question (the refund tool sat near the classification rules, which drifted out of the attention window); an agent that calls a CRM tool for a plain knowledge-base query; an agent that spits raw JSON when asked a natural-language question. None of these are “the model is dumb” — they’re the geometry of a bloated prompt.

2.3 · Tool confusion grows quadratically#

Every tool you bind forces the model to solve an n-way classification (“which of these do I call?”) on every single turn. This isn’t linear. Similar tools create semantic interference in the model’s selection process, so the risk of a wrong pick scales with the number of pairs the model could confuse — and pairs grow quadratically.

Same total capability, radically different failure surface. Splitting the toolbox is the single cheapest way to buy reliability.

2.4 · The principles that justify splitting#

Separation of Concerns (SOC) is the plain idea that a system should be divided into distinct sections, each addressing one concern — one piece of information that affects behavior. It was articulated by Edsger W. Dijkstra around 1974. [!] The transcript guessed he was “from France” and was unsure — Dijkstra was Dutch. The principle and date stand. SOC doesn’t tell you how to separate; only that you must.

In ordinary software, violating SOC gives you code that’s hard to read and maintain. In LLM systems it gives you something worse: a system that is probabilistically incorrect. That’s the reason the principle takes on new urgency here.

Single Responsibility Principle (SRP) — from Robert C. Martin’s SOLID set — sharpens SOC into: a class (here, an agent) should have exactly one reason to change. So the triage agent exists to sanitize and classify; the tech-support agent exists to resolve issues; a billing agent exists to process billing. When new PII patterns appear, only triage changes. When a new CRM tool lands, only tech support changes. The others are untouched.

There’s also a hard cost argument. A 10,000-token system prompt is re-sent on every agentic turn — every tool call, every routing decision, every response. At 10,000 requests a day that’s on the order of 100 million tokens/day spent on prompt overhead alone. Split those instructions across a 500-token triage prompt and a ~1,000-token tech-support prompt and the average cost per request drops by roughly an order of magnitude.

2.5 · Three production failure modes#

These aren’t hypothetical — they’re documented patterns. Each is a role bleeding past its mandate because attention on the “stay in your lane” instruction has decayed.

2.6 · The three topologies#

A topology is the communication pattern between your agents — who is allowed to talk to whom. Choosing it is as consequential as choosing a data model for a database: pick wrong and the system is structurally incapable of meeting enterprise needs (determinism, auditability, cost bounds).

Swarm — peer-to-peer, no boss#

Any agent can invoke any other agent; behavior is emergent from local decisions, and the group converges by something like majority voting. Genuinely useful for creative brainstorming, exploratory research, or simulation — where non-prescriptive, organic exploration is the point.

Hierarchical — supervisor / worker#

A single supervisor holds exclusive routing authority. It dispatches work to specialized workers based on the current state, each worker executes only its function and returns results only to the supervisor — never to another worker. The supervisor integrates results, updates shared state, and makes the next routing decision. This is the gold standard for enterprise systems and the topology this course builds.

Pipeline — linear sequence#

The simplest topology: a fixed sequence where each agent’s output is the next agent’s input. A processes raw input into enriched state, B applies its transform, and so on — like an Airflow DAG or an assembly line. No head, but also no branching intelligence.

2.7 · The handoff — a formal state transition#

A handoff is not a function call or a message — it’s the transfer of custody of a structured state object from one agent to the next, governed by a schema contract both agents agree to. Get it wrong and you get the most common class of production failure in multi-agent systems, almost always because someone treated the handoff as an informal string instead of a typed protocol.

The rule that kills ambiguity: the receiving agent must never infer, assume, or recompute anything the sender should have handed it. If triage classified the ticket as technical, tech support receives ticket_type = "technical" as an explicit field — not as something it must re-derive from the conversation text. If PII was masked, it receives pii_detected = True and sanitized_input as explicit fields.

from typing import TypedDict, Literal

# The handoff contract. Both agents agree to these fields and types.
class SupportState(TypedDict):
    raw_input: str
    sanitized_input: str          # written by triage
    pii_detected: bool            # written by triage
    injection_detected: bool      # written by triage
    ticket_type: Literal["technical", "billing", "fraud", "general"]
    is_safe: bool                 # triage's final safety verdict
    # ...downstream fields owned by tech support live here too

2.8 · State ownership, scope, and reducers#

State ownership is the multi-agent version of variable scope. Just as a well-designed function reads and writes only its own locals and declared parameters, a well-designed agent touches only the state fields in its scope. Triage owns sanitized_input, pii_detected, injection_detected, ticket_type, is_safe. Tech support owns tool_results, final_response, summary. Neither writes the other’s fields.

A reducer is a function that decides how concurrent writes to a state field are merged. When agents run in parallel, a field with no reducer defaults to plain overwrite — last writer wins — which silently loses data depending on who happened to finish last. LangGraph ships add_messages as the built-in reducer for the messages field; for every other field you must define reducers explicitly, and you must do it before you test, not after you’re debugging a data-loss incident.

2.9 · Tool scoping & the zero-trust agent#

Tool scoping means binding an agent exactly the tools it needs and no others — the AI translation of the principle of least privilege from information security: a process gets exactly the permissions it needs, not one more. Applied to LLMs, this goes beyond access control; it directly changes correctness and predictability.

Engineers keep trying to substitute a strong prompt rule (“do not use the refund tool for classification”) for actual tool absence. It doesn’t hold. LLMs are probabilistic: they will violate any natural-language instruction with some non-zero probability, and across thousands of calls that violation becomes statistically guaranteed. The only reliable mitigation is architectural: if triage should never refund, don’t bind the refund tool to triage.

2.10 · The two agents in this build#

The prompts follow the same logic as the tools. Triage’s system prompt is surgical: classification instructions for the four ticket types, PII-handling rules, and nothing else — no CRM instructions, no tool talk. Tech support’s prompt is purely technical: how to use its tools, interpret KB results, format resolutions, handle tool failures — and zero security or PII content, because triage already handled that. Short prompts mean high attention on what matters.

2.11 · Subgraphs vs. unified graphs#

A subgraph is a fully compiled StateGraph embedded into a parent graph as a single node. LangGraph gives you two options: one unified graph where every node shares a single state definition, or subgraphs where each agent is its own compiled graph plugged into a master graph.

The unified graph rots into a spaghetti problem faster than people expect. Eight nodes with three transitions each is already 24 conditional edges; the number of paths grows roughly as n!. Five developers editing one graph file means constant merge conflicts and no single person who can reason about the whole thing — the exact organizational failure that pushed the industry from monoliths to microservices.

Subgraphs fix this by making ownership enforceable in code. To the parent, a subgraph is a black box: it receives a state object, returns a modified one, and its internal nodes/edges are entirely opaque. Team A owns triage, team B owns tech support, team C owns the master orchestration and the contracts between them.

from langgraph.graph import StateGraph, START, END

# --- triage subgraph ---
triage = StateGraph(SupportState)
triage.add_node("ingress", ingress_node)        # PII + injection scan
triage.add_node("classify", classify_node)      # LLM @ temp=0
triage.add_edge(START, "ingress")
triage.add_conditional_edges("ingress", route_after_ingress)  # blocked? -> END
triage.add_edge("classify", END)
triage_graph = triage.compile()                 # NON-NEGOTIABLE

# --- embed it in the master graph as ONE node ---
master = StateGraph(SupportState)
master.add_node("triage", triage_graph)         # subgraph = a node
master.add_node("tech_support", tech_graph)
master.add_conditional_edges("triage", route_after_triage)  # by ticket_type
master_app = master.compile(checkpointer=checkpointer)

2.12 · The assembled system#

Put together, it’s a supervisor/worker hierarchy. The master graph holds the full shared state and two subgraphs. Everything from the outside enters triage; only safe, classified requests reach a worker.

2.13 · Interview & practice Q&A#

2.14 · Key takeaways#

Microexercises#

Reading proves you recognize the idea. These prove you can use it. Attempt each before opening the walkthrough.

3.1 · Build it — a reducer that stops silent data loss#

# triage node returns:
{"findings": [{"pii_scan": "clean", "injection": "none"}]}

# tech-support node later returns:
{"findings": [{"customer": "C10002", "balance": 998, "status": "past_due"}]}

# EXPECTED final state["findings"]  ->  a list of length 2, triage entry first

from typing import Annotated, TypedDict, Any

def merge_findings(old: list | None, new: list | None) -> list:
    return (old or []) + (new or [])   # append, never replace

class SupportState(TypedDict):
    findings: Annotated[list[dict[str, Any]], merge_findings]

# after both writes:
# state["findings"] == [
#   {"pii_scan": "clean", "injection": "none"},          # triage
#   {"customer": "C10002", "balance": 998, "status": "past_due"},
# ]   -> length 2, triage first  (correct)

3.2 · Diagnose it — the agent that refunds when it shouldn’t#

SYSTEM_PROMPT = """You are a CLASSIFICATION agent.
Return one of: technical | billing | fraud | general.
Do NOT initiate refunds. Do NOT call the refund tool."""

tools = [classify_ticket, search_kb, issue_refund]   # copy-pasted list
triage_llm = llm.bind_tools(tools)                    # temp defaults

def triage_node(state: SupportState) -> dict:
    msg = triage_llm.invoke([SYSTEM_PROMPT, state["raw_input"]])
    return {"ticket_type": msg.content, "is_safe": True}

tools = [classify_ticket]                 # zero-trust: bind ONLY what triage needs
triage_llm = llm.bind_tools(tools)
# (bonus) make classification deterministic:
triage_llm = llm.bind_tools(tools).with_config({"temperature": 0})