From a bare model to a working agent — in plain Python

Model vs Chatbot vs Agent

The session opens by re-drawing the picture from the previous class: an AI system is just a few boxes. There is a box for the agent, connected to a brain, a memory, and some tools. Every framework — n8n, LangChain, anything — is ultimately arranging those same boxes. The instructor's framing: once the basics are clear, frameworks are just different languages for the same idea.

The raw AI model: a stateless brain

A model on its own is “just a brain which is stateless.” You send a question, you get an answer, and it remembers nothing — not who you are, not the previous call, not the next one. Whatever you send, it replies based purely on its own capability.

The chatbot: a model wrapped in memory

A chatbot wraps a model with conversation history. Each call to ask() appends both the question and the answer into self.history, and the whole history is resent alongside the new question. The brain is still there and still stateless — the chatbot is the wrapper that simulates memory by carrying the full transcript every time.

The agent: memory + tools + the ability to act

An agent keeps the brain and memory, and adds tools — web search, a calculator, a currency lookup, a weather API, anything. Crucially (and this becomes the punchline of Section 8), an agent also runs in a loop: the model can ask for a tool, the tool result is fed back to the model, and the model continues until it can answer. That loop is the only real difference between “a model that suggests a tool” and “an agent.”

How messages really work

Every chat-style API call is built from a list of messages, and there are just three roles you need to hold in your head.

The instructor demonstrates this live in the Groq playground. Sending a single “hi, how are you?” produces three visible messages on the right: system, then user, then assistant. Add a second turn and now four messages are sent — the entire history travels every time.

The instructor illustrates the cost with a real example: opening a fresh chat in a hosted assistant and sending one token still registered roughly 19K tokens, because a long hidden system prompt (plus connectors) is prepended. He even pulls up a public “system prompt” dump to show how much standing instruction real assistants carry.

class Chatbot:
    def __init__(self, brain):
        self.brain = brain          # the stateless AI model
        self.history = []           # memory = a list of message dicts

    def ask(self, question):
        self.history.append({"role": "user", "content": question})
        answer = self.brain.get_answer(self.history)   # send FULL history
        self.history.append({"role": "assistant", "content": answer})
        return answer

In the very first file this brain call was a stub — a mock that echoes "stub prediction for {question}" instead of hitting a real API. The instructor's gloss: “stub means we are not actually calling any AI… it is just equal to a mock.” Real calls arrive in Section 4.

The cutoff wall

To motivate agents, the instructor asks the model a deliberately impossible question: “Who won the FIFA World Cup match yesterday?” The model can't answer. It replies that, as a text-based model, it has no real-time access to current events.

This sets up the arc for the rest of the class: to answer live questions the agent must be given tools (web search, calculator, get-weather, get-currency…), and the model must be able to say “call this tool.” The instructor is careful about a distinction that returns later — the model can decide which tool to call, but the actual calling is done by your code.

Calling a real model

Having built the model / chatbot / agent skeleton with a stubbed brain, the class now “removes the mocking” and calls a real provider. Two categories are introduced:

The OpenAI call

The key is loaded from the environment (kept in a .env file, never hard-coded), and the call mirrors the provider's own docs exactly: client.chat.completions.create(...) with a model, a max_tokens cap, and the messages array. The instructor's point: “this is exactly how OpenAI tells you to call it — I've just copied the documented approach.”

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def ask_openai(question):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        max_tokens=200,               # guardrail on OUTPUT length
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

Reading the response — where the answer actually lives

Printing the raw response reveals a rich object: an id for the call, the object type, a created timestamp, the model, a choices array, and a usage block. Your answer is buried at choices[0].message.content.

Tokens and the max_tokens guardrail

The usage block reports prompt_tokens (input) and completion_tokens (output). In the demo a short call showed ~24 prompt + ~30 completion tokens, and the reply respected the 200-token cap.

from openai import OpenAI      # Groq is OpenAI-compatible

client = OpenAI(
    api_key=os.environ.get("GROQ_API_KEY"),
    base_url="https://api.groq.com/openai/v1",   # ← the only real change
)

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    max_tokens=200,
    messages=[{"role": "user", "content": question}],
)
answer = response.choices[0].message.content

Structured output with Pydantic

The motivating example: ask for the weather and the model replies “Today the weather is 25 degrees Celsius.” Lovely for a human, painful for an application. What you actually want is something a program can consume:

# instead of: "Today the weather is 25 degree celsius"
{ "temperature": 25 }

# email example — instead of a vague sentence, a usable object:
{ "subject": "leave",
  "body": "hi, I am going to be outside for tomorrow" }

Why Pydantic instead of writing it in the prompt

You could type “reply in JSON with subject and body, subject must not be empty, default is Reply, more than two words…” by hand every time. Pydantic lets you declare all of that once as a class: field types, defaults, descriptions, validation — and reuse the same model throughout your code.

from pydantic import *

class EmailReply(BaseModel):
    Subject: str = 'Reply'     # default value
    Body: str

Reproduced exactly from the supplied Colab file. In class this was extended live with a Field(description=…) to show how each field can carry guidance (“name of the city to get the weather for”), which becomes richer context for the model.

The reply is always a string — even when it's JSON

A subtle but load-bearing point the instructor hammers: whatever the model returns, over an API it is always a string. When you ask for JSON, you get a JSON-shaped string — sometimes wrapped in markdown fences or stray text. So the real-world flow is: instruct → get raw string → trim the junk → parse JSON → cast into your Pydantic model.

import json

instruction = (
    "Read the user message and reply with ONLY a JSON object, "
    "no other text, in this exact shape: "
    '{"city": "<city name>", "wants_fahrenheit": true|false}. '
    "User message: " + question
)

raw = ask_openai(instruction)          # still a STRING
clean = raw.strip().removeprefix("```json").removesuffix("```").strip()
data  = json.loads(clean)              # → dict
parsed = WeatherQuestion(**data)       # → typed Pydantic object
print(parsed.city)                     # "Tokyo"

Giving the model a tool

A tool starts as an ordinary function. In class it's mocked with a small in-memory table so the mechanics are visible without a real API — but the instructor stresses this could be any real API (Google, a weather service, an internal microservice).

sample_weather = {
    "delhi":  {"celsius": 34, "conditions": "clear"},
    "tokyo":  {"celsius": 22, "conditions": "partly cloudy"},
    "london": {"celsius": 17, "conditions": "rain"},
}

def get_weather(city):
    data = sample_weather.get(city.lower())
    if data is None:
        return f"No weather data for {city}"
    return f"{data['celsius']}C, {data['conditions']}"

The tool schema — describing the tool to the AI

Passing the function object itself doesn't work (“object of type function is not JSON serializable” — shown live as a real error). Instead you describe the tool as a JSON schema: its type, name, description, parameters, and which parameters are required. This is the exact structure hosted assistants use internally — the instructor even opens a connector's “create event” tool to show it's the same shape.

get_weather_schema = {
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a given city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "city name, e.g. Tokyo",
                }
            },
            "required": ["city"],   # can't call get_weather without a city
        },
    },
}

Letting the model decide

Section 6 gave the model a tool. Now instead of you deciding which tool and which arguments, you send the question and the tool schema together, and let the model choose. Its reply may contain a tool_calls list instead of plain text.

Two behaviours are demonstrated back-to-back. A greeting (“hi, how are you?”) returns a normal message and tool_calls = None. A tool-worthy question returns a tool call. With two schemas registered (get_weather and a calculator), “what is 2 × 2?” correctly picks the calculator — even though the model plainly knows the answer, it defers to the registered tool. The instructor then confirms the mirror image live in Claude: asked for latest FIFA results, Claude visibly calls web-search, gets a result, feeds it back, and only then answers.

import json

response = client.chat.completions.create(
    model="gpt-4o-mini",
    max_tokens=200,
    messages=[{"role": "user", "content": question}],
    tools=[get_weather_schema, get_calculator_schema],   # give it the tools
)

msg = response.choices[0].message
if msg.tool_calls:                       # model WANTS a tool
    call = msg.tool_calls[0]
    name = call.function.name             # "get_weather"
    args = json.loads(call.function.arguments)   # {"city": "Tokyo"}
    result = TOOLS_BY_NAME[name](**args)  # YOU run it
else:
    result = msg.content                  # plain answer, no tool needed

The dispatch table used to actually call the chosen function comes straight from the supplied file — this is the pattern that turns a name string into a real call:

def add_number(a, b):
    return a + b

def subtract_number(a, b):
    return a - b

TOOLS_BY_NAME = {"add": add_number, "subtract": subtract_number}

functionToCall = TOOLS_BY_NAME["add"]   # look up by the name the model returned

The agent & the loop

In file 06 the agent gets a get_client_and_model() for its brain, a set of tool schemas, and — the new ingredient — a loop. On each pass the model either returns plain text (done → return it to the user) or asks for a tool (run it, append the result, loop again).

def run_agent(question, tools, tools_by_name):
    client, model = get_client_and_model()
    messages = [{"role": "user", "content": question}]

    for _ in range(4):                     # max_iterations = 4
        response = client.chat.completions.create(
            model=model, messages=messages,
            tools=tools, max_tokens=200,
        )
        msg = response.choices[0].message

        if not msg.tool_calls:            # plain answer → done
            messages.append({"role": "assistant", "content": msg.content})
            return msg.content

        for call in msg.tool_calls:        # else run each requested tool
            name   = call.function.name
            args   = json.loads(call.function.arguments)
            result = tools_by_name[name](**args)
            messages.append(tool_result_message(call, result))  # feed back
    # loop repeats with tool results in context until a final answer

Scaling to many tools

The instructor demonstrates the cost live: a hosted assistant with many connectors enabled ballooned a trivial “hi” from a few thousand tokens to ~30,000, purely from connected tool definitions. His analogy: giving someone 100 laptops and asking them to work — the sheer clutter degrades performance and cost. Weak models also start mis-selecting or skipping tools as the list grows.

Q&A that sharpened the edges

Instructor's worldview & production notes

Practical production themes he surfaced, all grounded in the session (nothing imported from future topics):

• Cost & tokens. The full transcript and every tool schema are billed on each call; watch usage, cap output with max_tokens, and trim history/tools deliberately.
• Context bloat. Too many tools degrade both cost and model accuracy — lazy-load schemas and use sub-agents.
• Model capability. Tool selection, multi-tool calls, and instruction-following all scale with model strength; small models hallucinate tool choices.
• Data privacy. Feeding proprietary tool outputs back to a hosted model sends data out; self-hosted / open-source models keep it in-house.
• Reliability. Tool errors and timeouts are handled with ordinary guards (try/except, sentinels) plus a prompt that tells the model how to respond to failure.
• Freshness. For anything after the cutoff, attach a web-search tool (Tavily, Google, or a provider's built-in) — not the model alone.

Glossary

Practice

{ "type":"function",
  "function":{
    "name":"get_capital",
    "description":"Return the capital city of a given country.",
    "parameters":{
      "type":"object",
      "properties":{"country":{"type":"string","description":"country name, e.g. Japan"}},
      "required":["country"] }}}

Active recall