Agentic AI · Live session notes

Building an AI agent from scratch — then seeing LangChain do the same thing

This session takes one idea all the way down: an “agent” is a language model wrapped in a loop that can call your functions and remember the conversation. We build that loop by hand in plain Python, watch the message memory grow step by step, then open LangChain and recognise the exact same message trail running underneath.

✦Why the instructor teaches it this way

Mayank deliberately builds the agent loop in raw Python before touching any framework. His argument, repeated all session: if you jump straight to LangChain, everything is hidden and you never understand what is actually happening. Once you’ve written the loop yourself, LangChain becomes “just this same thing, handled for you.”

Section 00

Session at a glance

The class moves through a series of numbered files, each adding one idea to the previous one:

StageWhat was addedThe takeaway
Recap (03–05)Model, chatbot, providers, structured output, first tool callsThe model talks only in strings/tokens; structure lets us extract usable values.
File 06 — the agentThe full agent loop: memory, tool schemas, calling tools, feeding results backThis is the heart of the session — a “mini Claude / mini ChatGPT”.
File 07 — mini projectThree tools (weather, calculator, real currency API) + a Streamlit UIThe same loop scales to multiple tools and a real API.
LangChain V1create_agent(model, tools, prompt) then invokeA framework wraps the loop; the printed message trail is identical to ours.
◆Core idea

Everything in this session is one sentence, spoken near the end: “Agent = brain + tool + memory” — an LLM (the brain), the tools it can request, and the memory of everything said so far. Each file just makes one of those three pieces more real.


Section 01

Model → Chatbot → Agent

The session opens with a five-minute recap of the ladder of capability, because the whole point of an agent only lands once you see what a bare model lacks.

▸Definition

Model — you ask a question, it returns an answer. Nothing is remembered.
Chatbot — a model that also keeps memory of the conversation.
Agent — a chatbot that additionally has tools, and a loop that decides when to use them. “Everything becomes very nice with respect to memory and with respect to tools.”

The instructor is careful about one thing here: a raw model, by itself, is not very useful. Asked directly — “without memory, without tools, is a model helpful?” — the intended answer by the end of class is no. The value comes from the harness you wrap around it.

✦Instructor insight

“LLM is just a brain.” If you show the tool-calling code to a ten-year-old and ask them to fetch the weather, they’ll walk off — the brain needs to be strong enough to make the decision. A weaker model, or a badly written schema, will make mistakes. Capability is a property of the brain you plug in, not the loop around it.


Section 02

Providers & the OpenAI-compatible trick

Before code, the class settles a practical question that tripped people up the previous day: where does the “brain” actually come from, and how do you pay for it?

ProviderTypeNote from class
OpenAIClosed source“OpenAI, not ChatGPT.” Requires paid credits in dollars.
AnthropicClosed sourcePaid; and its API is shaped differently (see below).
GroqAccess to open + closed models“GROQ, that is GROQ” — not Elon Musk’s Grok/xAI. Gives free calls; preferred for the course.
OpenRouterAggregatorHas paid proprietary models and some open ones; ~a handful of free calls.
◆Core idea — “OpenAI-compatible”

OpenAI came first and defined the client.chat.completions.create(...) shape. Rather than invent a new interface, Groq and OpenRouter said: keep your OpenAI code, just change the base_url and use our key. Everything else stays the same. That one swap is the entire integration.

◈The analogy he used

Apple taught users to close an app by swiping up. Rather than teach a brand-new gesture, other phone makers copied it — it’s already a learned user behaviour. Groq/OpenRouter do the same: they mould themselves to the interface developers already know, instead of forcing new habits.

▲Watch out — Anthropic is different

Anthropic is not a drop-in base-URL swap. There is no chat.completions endpoint; it has its own way to call the API. So the “just change the base URL” trick works for OpenAI-compatible providers (Groq, OpenRouter, and many others) — but not for Anthropic.

●Practical tip

For the course, use Groq or OpenRouter so you “spend the least money learning these things.” Get a key, drop it into your environment, and remove any leftover demo keys — several people broke their setup the day before by leaving a placeholder key in place.


Section 03

Structured output, and tools as plain functions

Two recap ideas feed directly into the agent loop: getting structured output out of the model, and understanding that a “tool” is nothing exotic.

▸Definition — structured output

You ask the model to answer in a fixed shape (e.g. fields city and wants_fahrenheit) and then parse that into a real Python object (the class used Pydantic). Why bother? Because “once I’ve got the answer, my work starts in code” — you need a clean value like "Delhi" to actually pass into a function.

✦Instructor insight — everything is a string

“Your AI is just communicating with the help of string. You pass it tokens, it gives you back tokens — for it, everything is tokens only.” Structured output is how you turn that fuzzy string back into something your code can rely on.

▸Definition — a tool

“A tool is just a normal Python function.” The model does not run it. The model only says which function to call and with what arguments; your Python then runs it. The function “doesn’t care if it’s getting the city via API, via AI, or via you — it just needs the right argument.”

◇Extra context

Mayank stretches this further later: tools, methods, functions, even an MCP server — “they’re all just API calls at the end of the day.” If you can connect your Gmail or calendar as functions, they become tools too. Nothing magical.


Section 04

How the model requests a tool

This is the pivot the whole loop depends on. When you send a message and a list of tool schemas, the model’s reply is one of exactly two shapes.

  • A plain reply. Ask “hi, how are you?” → tool_calls is None, and content holds the text answer. Nothing else to do.
  • A tool request. Ask “what’s the weather in Tokyo?” → content is None, and tool_calls contains a structured request: the function name and the arguments.
◆Core idea — structure beats fuzziness

The model does not reply with fuzzy prose like “hey, go call getWeather with Tokyo.” If it did, you’d have to parse free text to find the function name and argument. Instead OpenAI standardised a structured tool-call format that everyone adopted — so name and arguments are cleanly, reliably extractable.

user message + tool schemas LLM decides only content = text · tool_calls = None → return the answer content = None · tool_calls = [ … ] → run the tool(s), loop again
One model call, two possible shapes of reply. The loop’s only job is to tell them apart.
✦Instructor insight — why this “changes the game”

The FIFA-results thought experiment: imagine ChatGPT replied “go to Google, search FIFA results, and paste them back to me.” Useless — you’d never use it. A real agent searches the web itself and hands you the answer. Tool calling is what lets the model take that action instead of asking you to.


Section 05

The agent loop — the heart of the session

This is file 06, the “mini Claude.” The setup is deliberately boring so nothing distracts from the loop: a mock weather tool over a small dictionary. All code below is the exact source from the supplied notebook.

The tool and the two lookup tables

# A "tool" is just a normal Python function.
SAMPLE_WEATHER = {
    "tokyo": {"celsius": 22, "conditions": "partly cloudy"},
    "delhi": {"celsius": 34, "conditions": "clear skies"},
    "london": {"celsius": 15, "conditions": "light rain"},
}

def get_weather(city: str) -> str:
    data = SAMPLE_WEATHER.get(city.lower())
    if data is None:
        return f"No weather data for {city!r}."
    return f"{city.title()}: {data['celsius']}C, {data['conditions']}"
# What the MODEL sees — so it knows the tool exists,
# what args it needs, and how to format the call.
TOOL_SCHEMAS = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

# What YOUR CODE uses — maps the string name to the real function object.
TOOLS_BY_NAME = {
    "get_weather": get_weather,
    "get_currency": get_currency,
}

Two tables, two audiences. TOOL_SCHEMAS is for the model (a description). TOOLS_BY_NAME is for you (the actual callable). Section 06 explains why you need both.

●Practical tip — always write a description

Technically every field in the schema is optional. But if you omit the description, the model has to guess from the name alone and “it will hallucinate — then you’ll wonder why your AI isn’t performing nicely.” A good schema and description is what makes tool selection reliable. (In future, a @tool decorator + docstring will generate this for you — see §11.)

The loop itself

Strip away the verbose printing and the whole agent is remarkably small. Read it once top-to-bottom, then we walk each decision.

def run_agent(messages: list[dict], max_turns: int = 4) -> str:
    client, model = get_client_and_model()

    for turn_number in range(1, max_turns + 1):

        response = client.chat.completions.create(
            model=model,
            max_tokens=300,
            messages=messages,
            tools=TOOL_SCHEMAS,
        )
        message = response.choices[0].message

        # 1) No tool requested → this is the final answer.
        if not message.tool_calls:
            messages.append({"role": "assistant", "content": message.content})
            return message.content

        # 2) Tool(s) requested → first record the request in memory,
        #    preserving order. content here is None.
        messages.append({
            "role": "assistant",
            "content": message.content,
            "tool_calls": [
                {
                    "id": call.id,
                    "type": "function",
                    "function": {
                        "name": call.function.name,
                        "arguments": call.function.arguments,
                    },
                }
                for call in message.tool_calls
            ],
        })

        # 3) Now actually run each tool and append its result.
        for call in message.tool_calls:
            tool_name = call.function.name
            arguments = json.loads(call.function.arguments)
            tool_function = TOOLS_BY_NAME[tool_name]
            result = tool_function(**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result),
            })
        # loop back: the model can now read the tool result and answer

    return "Reached max_turns without a final answer."
◆Core idea — why it must be a loop

A single model call isn’t enough. When the model asks for a tool, you run it, but then the model still hasn’t seen the result. You append the result to memory and call the model again so it can read it and produce the final natural-language answer. “Why do we loop again? Because the model has the tool result only after this step.”

user message → memory call the LLM memory + schemas tool_calls? yes run tool(s), append results (record request first, in order) loop back so the model can read the result no return final answer max_turns guards the loop
The agent loop: call → is it a tool request? → run & feed back, or return. Bounded by max_turns.

Watching memory grow (the verbose walkthrough)

After the break, Mayank re-runs everything in a Colab notebook that prints every state, so you can literally watch the message list grow. This is the moment most of the class said things clicked.

You typeMemory afterWhat happened
“Hi, I am Mayank”2 messagesPlain reply. tool_calls is None → return content. Just like ChatGPT.
“Who am I?”4 messagesAnswered “you are Mayank” — only because prior turns are in memory. Proof of why memory matters.
“Weather in Tokyo?”5 → 6 → 7 → 8Model returns a tool call (content None). Request appended (6). Tool runs, result appended (7). Loop again → model reads result → final answer (8).
✦Instructor insight — memory is why “who am I?” works

He isolates the trick: if you send only the message “Who am I?”, the model cannot know. It answers correctly because you resend the whole conversation each time. “I hope you understand in depth why memory is required.” Memory here is nothing fancy — just a Python list you keep appending to.

▲Watch out — the model runs on the server, not in your Python

Repeated several times: “the model does NOT run Python code.” The Llama model lives on Groq’s servers. It only decides whether to answer or which tool to request. Your Python is what executes the tool. Confusing these two is the single most common misunderstanding in the room.


Section 06

tools_by_name: turning a string into a function

The model gives you back a tool name as a string, like "get_weather". A string is not callable. This tiny mapping is how you recover the real function — and it’s the piece the instructor spends a live tangent making concrete.

# AI says: call "get_weather". But a string can't be called.
tool_name = call.function.name        # "get_weather"  (a string)
tool_function = TOOLS_BY_NAME[tool_name]   # the real function object
arguments = json.loads(call.function.arguments)   # '{"city":"Tokyo"}' → dict
result = tool_function(**arguments)   # actually call it
◈The live demo

He defines add_number(a, b) and subtract_number(a, b), then a dict {"add": add_number, "subtract": subtract_number}. In Python a dict value can be a function. So tools_by_name["add"](5, 6) returns 11. The AI can only hand you the string "add" — you look up the function and call it yourself. With 100 tools you’d have 100 entries in that map.

▸Definition

tools_by_name is a mapping (dictionary) from the tool’s string name to its actual Python function. The model returns a name; this dict returns the callable behind it. It’s the bridge between “what the model said” and “what your code can execute.”


Section 07

tool_call_id and why order is sacred

Two subtle details generated the most Q&A in the session, and both are about bookkeeping the message history correctly.

▸Definition — tool_call_id

A unique id the AI attaches to each tool request (you don’t create it). When you send the result back, you echo the same id so the model can match “this result belongs to that request.” It is never the same across calls — even calling the same tool with the same argument twice yields a new id.

✦Instructor insight — why record all requests before running any

Notice the loop appends the assistant’s tool-call request to memory first (step 2), and only then runs the tools (step 3). Reason: the message history must stay strictly linear and ordered — “first the AI told me to call these tools, so I write that down; then I call them one by one and add the results.” If the code broke mid-way (a timeout, a network drop), you must not end up with results whose matching requests were never recorded. You can’t shuffle history.

▲Watch out — multiple tool calls in one turn

message.tool_calls is a list. Ask “weather in Tokyo and Delhi and Hyderabad” and the model may return three tool calls at once, each with its own id. Your inner for loop handles all of them. In this instructor’s design, running several tools counts as a single outer iteration — a point he hammers in the Q&A (see §13).

◇Extra context — sequential vs parallel

Whether those tool calls can run in parallel is your decision. If they’re independent, call them asynchronously. If one depends on another (“get the weather, then convert currency for the coldest city”), you must run them in sequence. The model tells you what to call; how you call it is your code.


Section 08

max_turns and the safety stop

max_turns (4 in file 06, 5 in the project) is the guard rail on the outer loop. It carries no clever logic — it exists so the agent can’t run forever.

◆Rule / principle

Anything that runs in a loop needs a stop condition. An agent can otherwise keep researching, keep retrying a failing tool, or hallucinate “call 100 tools” — burning tokens, time, and money, and making the user wait forever. Pick a number (4, 5, 10, 100) suited to your use case.

◈Example — the runaway retry

Imagine the weather API key is wrong. The tool returns an error; the model, told “always give the right answer,” tries again… and again. Without a cap it loops indefinitely. max_turns (much like the “retry 3 times” pattern from ordinary engineering) forces it to stop.

◇Extra context — real products do this too

Claude sometimes shows “the maximum time for this tool has expired — click Continue.” Same idea: it doesn’t want the agent running unbounded and racking up cost. The stop can be based on turns, on time, on tokens — an architectural decision that differs across frameworks. It’s optional, but leaving it out is “not a good design.”


Section 09

Mini-project: three tools + a Streamlit UI

File 07 keeps the identical loop but proves it scales: three tools instead of one, and a small web front-end. Mayank calls it “project zero — a mini ChatGPT / mini Claude.”

ToolKindNote
get_weatherMockSame dictionary tool, unchanged.
calculatorMockSimple arithmetic tool.
convert_currencyReal APICalls the live Frankfurter API — so results match real exchange rates (“pretty close”), unlike the mocked ones.

All three get a schema, all three go into TOOLS_BY_NAME, and max_turns is raised to 5. The run function is unchanged. He then wraps it in Streamlit (streamlit run) so you get a chat box in the browser — the same loop, now behind a UI.

◈Example — mock vs real

Ask “1 USD to INR” and the answer lands close to the true rate, because the currency tool hits a real API rather than a hard-coded dictionary. Ask two things in one message (“weather in Tokyo and the USD→INR rate”) and the model issues multiple tool calls in the same turn.

✦Instructor insight — you build this one yourself

This project isn’t just a demo to watch: “you have to understand this project and code this.” It’s the bridge between the hand-written loop and everything the framework section is about to hide from you.


Section 10

Why frameworks exist at all

Having just built the loop by hand, the natural question is: so why use LangChain? The answer is framed as a maintenance problem, not a magic one.

◆Core idea

Your hand-written agent works — until requirements pile on. Add AWS Bedrock, then Azure Foundry, then caching, then more tools, then guardrails, then multi-agent, then MCP… and you’re endlessly rewriting the same file. A framework absorbs that repetitive plumbing so you configure instead of re-code.

✦Instructor insight — the Lombok comparison

He compares it to Java: you could write every getter/setter by hand, but Lombok generates them. Frameworks are the same move for agents. “The more you try to standardise your own code, the more it drifts towards being a framework anyway.” And underneath, create_agent “is nothing but Python code only” — the same loop you just wrote.

●Practical tip — when NOT to reach for a framework

“If your work is very simple — just call some tools and an AI — I don’t go for frameworks, because your own system is fast and cheap.” Mayank says he sometimes deliberately writes his own agent (or none) in production precisely to keep control. Frameworks earn their keep when you need multi-agent, MCP, guardrails, and the like handled for you.

◇Extra context — how teams actually pick one

There’s no single “right” framework, and ~70% of concepts are identical across all of them. Real drivers: first-mover familiarity (LangChain/LangGraph got early adoption), matching your cloud (AWS teams lean on the AWS SDK), the level of control you need, and simply what your leadership already chose. “It’s just like programming languages.”


Section 11

LangChain V1 and the “harness”

The class installs LangChain with uv and writes the smallest possible agent. The shock is how little code it takes — and how the printed message trail is exactly what they built by hand.

from langchain.agents import create_agent

agent = create_agent(
    model="...",          # the brain; key read from the environment
    tools=[get_weather],   # pass functions directly
    system_prompt="...",
)

result = agent.invoke({"messages": [{"role": "user",
                                     "content": "weather in San Francisco?"}]})

Everything from §05 — the loop, the memory, feeding tool results back — happens inside invoke.

▸Definition — agent = model + harness

Straight from the LangChain docs he reads aloud: “Agent = model + harness.” The harness is “everything around the model loop — the prompts, the tools, and any middleware that shapes behaviour.” By giving your model tools, a way to run them, and memory, you harness its power. That’s literally what you built by hand.

The same message trail, revealed

He prints the raw messages LangChain produced, and walks them one by one:

MessageContents
HumanMessage“what’s the weather in San Francisco” (LangChain adds its own ids).
AIMessagecontent empty, finish_reason = tool_calls, a tool call with id, name, arguments.
ToolMessagethe tool’s result; tool_call_id matches the request exactly.
AIMessage (final)“It’s currently sunny in San Francisco.” Now it has content.
◆Core idea — the payoff of learning it the hard way

“Is it maintaining the memory? Yes. Is it running on a loop? Yes.” The loop ran twice — once to request the tool, once to answer with the result — exactly like the hand-written version. LangChain didn’t invent anything; it hid the plumbing. Because you built it, you can now read what’s happening behind the scenes. Tokens are visible too (prompt vs completion vs total).

●Practical tip — the @tool decorator

In LangChain you rarely hand-write a schema. Decorate a function with @tool and its signature + docstring become the schema automatically. You can pass the function straight to create_agent. That docstring is doing the same job as the description field you saw in §05.

Environment & project setup (uv)

The scaffolding matters for the project work that follows:

  • uv init to start the project, then uv add langchain langchain-openai langchain-anthropic (adding 3 packages pulls ~45 dependencies). uv sync / uv lock manage the rest; the virtual env doesn’t even need manual activation for uv, though you still source .venv/bin/activate on Mac to run scripts.
  • Create a .env with placeholder keys for every provider you might use (OpenAI, Groq, Anthropic, Gemini, OpenRouter).
  • Add a .gitignore that excludes .env and .venv — so secrets and the huge env never get pushed.
  • Ship a .env.example (a copy with blank values) so others know which keys to fill in, without leaking yours.
◇Extra context — caching & TTL

From the docs walkthrough: whether to cache a tool result depends entirely on the data. Never cache stock prices (they change every second). Weather is fine to cache for ~a day; currency maybe ~10 minutes. This is a time-to-live (TTL) decision, usually handled in something like Redis — “it depends a lot on the context.”

◇Extra context — memory, runtime context, monitoring

LangChain adds a checkpointer / InMemorySaver keyed by a thread config to persist memory across turns — but it’s still just saving messages somewhere (a list, a file, a DB — “Excel, pen and paper, whatever”). Tools can read runtime context (previous messages, agent memory). And for latency/cost/token visibility in production, you reach for observability tools like LangSmith — “especially once you have multiple agents together.”


Section 12

DeepAgents · LangChain · LangGraph — the abstraction ladder

LangChain ships several products. The instructor’s food analogy makes the trade-off between control and convenience memorable.

LangGraph lowest-level orchestration · “raw vegetables” · most control LangChain (create_agent) customisable harness · “home-cooked meal” · control the spice DeepAgents batteries included · “Swiggy” · order & go, little control ↑ more convenience ↓ more control
Same LangChain family, different altitudes. DeepAgents are built on LangChain agents; LangGraph sits beneath both.
◈The food analogy

DeepAgents = Swiggy: order finished food (auto context compression, virtual file system, sub-agent spawning) — but you can’t change how it’s cooked. LangChain = home-cooked meal: you choose the spice level and quality. LangGraph = the raw vegetables: the lowest level of orchestration, where you control every ingredient.

▸Definition — deterministic vs agentic flow

Deterministic: X → Y → Z, no runtime decision, no LLM needed — logging into Gmail with the right password, or 2 + 2 on a calculator. Pre-2022 software is deterministic. Agentic: the AI decides at runtime — maybe call a tool, maybe answer directly, maybe ask a follow-up. “Everything in AI is fuzzy.” LangGraph is favoured when you need to combine deterministic and agentic steps.

◆Rule / principle — the teaching path

Class order going forward: LangChain → LangGraph → DeepAgents. Learn the customisable middle first (because building real software needs control), then drop to the low level, then see the batteries-included top. Also: LangChain V1 (~1.3.x) is agent-first; everything before v1 is now “LangChain Classic” — most YouTube tutorials still show Classic, but the course teaches the latest.


Section 13

Production wisdom — from the Q&A

Nearly two hours of questions followed the teaching. These are the exchanges that carried real learning value, kept close to the concepts they clarify.

QDoes running multiple tools count as multiple iterations against max_turns?

In this instructor’s design: no. If the model asks for five tools in one turn, all five run inside a single outer iteration — like an inner loop that finishes before the next model call. So even with max_turns = 1, all five tools still get called. He’s explicit that this is his architectural choice: you could instead count each tool call as its own iteration. Frameworks may treat it differently.

QWhy does one simple question sometimes run the loop twice?

Because a tool was involved. Turn one: the model returns a tool request. You run the tool. Turn two: the model reads the result and writes the final sentence. Two model calls for one question is expected whenever a tool is used. (This also explains the latency — you’re making two round-trips to the provider.)

QWhat does “stateless LLM” actually mean?

The model maintains no state between calls. Send only “Who am I?” and it can’t answer. It only “remembers” because you resend the entire message history every single call. “You always have to send all the data.” Memory is a thing you maintain and replay, not something the model holds.

QWhat do the roles (system / user / assistant) do, and which gets dropped on overflow?

system sets behaviour and is given the most weight (“you are a polite assistant”). user is what the user said; assistant is what the AI replied. Every message goes in every call. When you exceed the context window, it’s the older user/assistant messages that get truncated — the system message is kept, because it’s needed to steer behaviour. He also shows you can hard-code a fake history to steer the model (“you made a calculation error”).

QCan I send our 600-million-row dataset to the AI?

Emphatically no — and this became the sharpest exchange of the session. The context window tops out around ~1M tokens; “AI will not even be able to take 100 million rows.” You run analytics/code over big data and send the results, not the raw data. “Whoever is doing that — please tell them to understand AI. It’s not an AI problem, it’s a system-design problem.”

QFor a coding assistant that needs our internal conventions, should I build a RAG?

RAG is “not the right answer” for coding context — it will confuse more than help. Instead, put your conventions in markdown context files (a CLAUDE.md / agent.md style file that says “this is how we normally code”). That’s a much better fit than retrieval for this use case.

QCould I just use a cheaper open-source model for tool-picking and summarising?

Yes — the two jobs here (choose a tool, summarise a result) don’t always need a top-tier model. Open-source models like Llama exist for exactly this and cut cost to just infrastructure. Why do companies still default to OpenAI/Anthropic? Partly unawareness (people confuse ChatGPT web with the API), partly quality/reliability for critical customer-facing flows where a smaller model may hallucinate. He expects the balance to shift toward open models as costs rise.

QCan’t I skip Claude Code / Devin and just use their agents?

No — those are pre-built agents in someone else’s ecosystem. You can’t deploy them for your company, you don’t control cost, memory, or how internal data is handled, and data security becomes a real problem. Building an agent is genuinely simple now (you just did it), so companies build their own to keep that control.

QFor architect / senior roles, what actually matters?

Roughly 80–85% classic software & system-design skill, 15% AI/agent knowledge. Companies want someone to make their systems “AI-ready”: reliable, cost-effective, well-monitored. “Pure AI development is nothing — it’s just creating software, but now with AI as well. If the cake is bad, the cherry is bad too.”


Section 14

Common mistakes & failure modes

Only the ones the session actually surfaced — no generic warnings.

Thinking the model runs your code

  • The model only decides; it runs on the provider’s servers. Your Python executes tools.

Leftover / demo API keys

  • Broke setups the previous day. Keep only the working key for your provider.

Omitting the tool description

  • The model guesses and hallucinates tool choice. Always describe your tools.

Shuffling the message order

  • Record tool-call requests before running them; history must stay linear or the model loses context.

No stop condition

  • A loop with no max_turns/time/token cap can run forever on a failing or research-y task.

Sending raw big data to the LLM

  • Never feed millions of rows. Compute analytics and send results. Respect the context window.

Trying to call the tool name string

  • The model returns a string; look up the real function via tools_by_name first.

Reaching for a framework too early

  • For simple “call a tool + AI” work, your own loop is faster and cheaper.

Section 15

Practice · generated from this session

These exercises are derived from what was taught — they aren’t transcript content. Work them against file 06/07.

1Recall · Trace the memory. You send “Weather in Delhi?” to run_agent. List every message appended, in order, and give content/tool_calls for each.

(1) user: “Weather in Delhi?”. (2) assistant: content None, tool_calls = one call to get_weather with {"city":"Delhi"} (id X). (3) tool: tool_call_id = X, content "Delhi: 34C, clear skies". (4) assistant: content = a natural-language sentence, no tool_calls → returned. Two loop iterations total.

2Apply · Add a tool. You want a get_currency(from_currency, to_currency) tool the model can actually pick. Which two places must you change, and what breaks if you edit only one?

Add it to TOOL_SCHEMAS (so the model knows it exists) and to TOOLS_BY_NAME (so your code can execute it). Schema only → the model requests it but TOOLS_BY_NAME[name] raises KeyError. Map only → the function is callable but the model never learns to request it. Note: in the source, get_currency is defined and in the map, but intentionally left out of the schema.

3Reason · Predict the failure. A colleague rewrites step 2/3 to run each tool and append its result immediately, skipping the “record all requests first” step. Give one concrete scenario where this corrupts the conversation.

If the model requested three tools and the code times out after the first result is appended, memory now contains a tool result whose matching assistant tool-request (with its ids and ordering) was never recorded — the history is non-linear and the model can’t relate results to requests on the next call. Recording all requests first keeps the trail consistent even if execution is interrupted.


Section 16

Active recall

Answer out loud before opening each one. These test understanding, not trivia.

?Explain, without notes, why an agent needs a loop rather than a single model call.

Because when the model requests a tool, running it isn’t the end — the model hasn’t seen the result. You append the result and call the model again so it can produce the final answer. And a task may need several tool rounds. The loop repeats “call → tool? → run & feed back” until the model replies with plain text.

?What are the two shapes a model reply can take when tools are attached, and how do you tell them apart?

Plain reply (content set, tool_calls = None) → return it. Tool request (content = None, tool_calls is a non-empty list) → run the tools. You branch on if not message.tool_calls.

?Why can’t you call the tool name the model gives you directly?

It’s a string, not a function. You look it up in tools_by_name to get the real callable, parse the JSON arguments, then call fn(**args).

?What is tool_call_id for, and who creates it?

The AI creates it. You echo it back on the tool-result message so the model can match each result to the request that asked for it — essential when multiple tools are called at once. It’s different on every call.

?Reconstruct “agent = model + harness.” What’s in the harness?

Everything around the model loop: the prompts, the tools and the way you run them, the memory, and any middleware that shapes behaviour. The harness is what turns a bare brain into something useful.

?Deterministic vs agentic — one example of each.

Deterministic: logging in with email + password (fixed X→Y→Z, no LLM). Agentic: “tell me the latest FIFA results” — the model decides at runtime to call a search tool, maybe more than once.


Section 17

Quick revision layer

The session in 60 seconds

An agent is LLM + tools + memory in a loop. You send the message history plus tool schemas; the model either replies in text or returns a structured tool call. If it’s a tool call, your Python looks the function up in tools_by_name, runs it, appends the result (echoing tool_call_id), and loops so the model can answer. max_turns stops runaways. The model never runs your code — it only decides. LangChain’s create_agent does all of this for you, and its message trail is identical to the one you built by hand.

Core ideas

  • Agent = brain + tools + memory, looped.
  • Model decides; your code executes.
  • Loop exists to feed tool results back.
  • Memory = resend the whole message list (stateless model).
  • Framework = your loop, plumbing hidden.

Instructor takeaways

  • Learn the basics; frameworks are the same underneath.
  • AI engineering ≈ 80% software development.
  • Always describe your tools or the model hallucinates.
  • Never send raw big data to an LLM — send results.
  • Real-world projects beat course certificates on a CV.

Code patterns worth keeping

# branch on the reply shape
if not message.tool_calls:
    return message.content        # final answer

# string name → real function, then call
fn   = TOOLS_BY_NAME[call.function.name]
args = json.loads(call.function.arguments)
result = fn(**args)

# echo the id back so the model can match request↔result
messages.append({"role": "tool",
                 "tool_call_id": call.id,
                 "content": str(result)})

# bound the loop
for turn in range(max_turns): ...

Key terms, one line each

  • Tool — a Python function the model can request.
  • Schema — the tool’s description the model reads.
  • tools_by_name — name string → function object.
  • tool_call_id — AI-issued id linking result to request.
  • max_turns — the loop’s safety stop.
  • Harness — prompts + tools + memory + middleware.

High-value warnings

  • Model runs on the server, not in your Python.
  • Record tool requests before running them.
  • No description → hallucinated tool choice.
  • No stop condition → runaway cost.
  • Big data → analytics/results, never raw rows.

Section 18

Glossary & resources
TermMeaning as used in this session
AgentAn LLM plus tools and memory, driven by a loop that decides when to act.
ToolA plain Python function the model can request by name.
Tool schemaThe JSON description (name, params, description) the model sees.
Tool callThe model’s structured request: function name + arguments.
tool_call_idUnique id the model assigns each request; echoed back with the result.
MemoryThe growing list of role/content messages, resent every call.
StatelessThe model keeps no memory itself; you replay history each time.
max_turnsCap on loop iterations to prevent runaway execution.
HarnessEverything wrapped around the model: prompts, tools, memory, middleware.
OpenAI-compatibleProviders you use by swapping only base_url + key (Groq, OpenRouter).
Deterministic flowFixed X→Y→Z logic, no model decision.
Agentic flowThe model decides actions at runtime; “fuzzy.”
Context windowMax tokens the model can hold (~up to ~1M); overflow drops old user/assistant messages.
TTL / cachingHow long a tool result stays valid before re-fetching.
Harness/observabilityMonitoring cost, latency, tokens (e.g. LangSmith).

Tools & libraries referenced

  • Groq & OpenRouter — OpenAI-compatible providers with free tiers; used throughout so learning stays cheap. (Groq ≠ Elon Musk’s Grok/xAI.)
  • Llama 3.3 70B — the open model used as the “brain” in the notebook.
  • Frankfurter API — the real currency endpoint behind the project’s convert_currency tool.
  • Streamlit — the quick web UI for the mini-project (streamlit run).
  • uv — project/dependency manager (uv init, uv add, uv sync, uv lock).
  • LangChain V1 — create_agent, @tool, init_chat_model, InMemorySaver; plus LangGraph, DeepAgents, LangSmith (monitoring).
  • Pydantic — used earlier for structured output; tiktoken — mentioned for counting tokens.
◇Reference note vs class content

The “agent = model + harness,” TTL/caching, @tool, and runtime-context points come from the LangChain V1 documentation the instructor read on screen — they support the session but weren’t built by hand in it. The agent loop, tools_by_name, tool_call_id/ordering, and max_turns are all from the live-coded files (06/07).

▲Transcript note — reconstructed items

The run_agent and tool/schema code is taken verbatim from the supplied notebook. The create_agent snippet in §11 is a faithful shape of what was shown, not an exact transcript capture (the screen code wasn’t fully dictated). Exact provider IDs, exchange rates, and some names spoken aloud were normalised where the recording was unclear. Next-session promises (LangChain in depth, monitoring, multi-agent, architecture) are noted but not taught here.