The Everyday Python Toolkit

01The Hook#

You are reading someone else's Python and you hit this: a function that calls getattr(self, "do_" + command)(), decorated with @retry, that awaits an httpx client inside an async def, and takes *args, **kwargs. None of it is advanced computer science. It is a dozen small, sharp tools that experienced Python programmers reach for constantly, and that tutorials rarely explain together.

The gap is not knowing what each does in isolation, it is knowing why you'd choose it and where it bites you. Miss the mutable-default trap or the async blocking trap and your code works in the demo and fails in production. This guide stitches these tools into one mental model: Python lets you treat code, arguments, and types as data you can inspect, reshape, wrap, and defer. Everything below is a variation on that theme.

02Introspection: getattr, setattr, hasattr#

Ask an object about its attributes using a name written as a string, at runtime, instead of hard-coding the dot access.

Normally you write obj.color. But what if the attribute name only becomes known while the program runs, for example it came from a config file, a web request, or a user command? You cannot write obj. followed by a variable. That is what these three built-ins are for, the "spread-the-load box" of attribute access, they turn the name into an ordinary value you can compute.

# getattr(object, name[, default])
color = getattr(car, "color")              # same as car.color
color = getattr(car, "color", "black")     # default if missing, no crash

# the name is computed, that is the whole point
field = "do_" + command
getattr(self, field)()                    # call self.do_start(), self.do_stop()...

setattr(car, "color", "red")             # car.color = "red", name as string
if hasattr(car, "turbo"): ...            # True/False, does the attr exist?

03isinstance & honest type checks#

Ask "is this value one of these types (or a subtype)?" and get back True or False.

The "is-this-the-right-kind-of-thing check" (a runtime type test) answers whether a value belongs to a class or any of its subclasses. Pass a tuple of types to mean "any of these".

if isinstance(x, int): ...
if isinstance(x, (int, float)): ...     # any-of: int OR float

# why not type()?
class Admin(User): ...
a = Admin()
isinstance(a, User)     # True  — an Admin IS a User
type(a) == User        # False — exact match only, ignores inheritance

04lambda#

A throwaway, unnamed function written in one line, holding a single expression.

A "one-line unnamed function" (a lambda) is just a function you don't bother to name because you'll use it once, right where you define it. Its body is a single expression whose value is returned automatically, no return, no statements.

sorted(people, key=lambda p: p.age)          # sort by age
max(words, key=lambda w: len(w))            # longest word
add = lambda a, b: a + b                    # works, but a def is clearer

05The * and ** spread operators#

Pour the contents of a collection into another place: * spreads a sequence's items, ** spreads a dict's key/value pairs.

You have a box of things and want its contents, not the box, poured into a function call, a new list, or a new dict. That is unpacking, the "tip-the-box-out" operator. * works on any iterable (list, tuple, set, generator); ** works on mappings (dicts) and spreads them as keyword arguments.

pair = (3, 4)
point(*pair)                 # point(3, 4) — items become positional args

merged = [*a, *b, *c]        # flatten several iterables into one list
combined = {**base, **overrides}   # merge dicts; right side wins on clashes

first, *rest = [1,2,3,4]     # first=1, rest=[2,3,4] — capture the tail
*init, last = [1,2,3,4]      # init=[1,2,3], last=4

defaults = {"timeout": 5}
requests.get(url, **defaults)     # same as requests.get(url, timeout=5)

06The ... (Ellipsis)#

Three dots are a real, singleton object called Ellipsis, not special syntax, most often used as a "nothing here yet / anything goes" placeholder.

Newcomers assume ... is magic. It is simply a built-in constant, like None or True, that evaluates to the one Ellipsis object. Because it reads as "to be filled in," Python culture uses it as a tidy stand-in in a few specific places.

def fetch(url: str) -> bytes: ...     # stub body — "implement later", like pass

class Reader(Protocol):
    def read(self, n: int) -> bytes: ...   # interface declaration, no logic

Callable[..., int]        # typing: "any arguments" -> int
arr[..., 0]              # NumPy: "all remaining axes" then index 0

07Flexible signatures: *args, **kwargs, and the bare * / /#

Let a function accept an unknown number of extra positional arguments (*args) or extra keyword arguments (**kwargs), and control how callers are allowed to pass them.

Inside a definition, the stars do the reverse of §5: instead of spreading a collection out, they gather leftover arguments up. *args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict. This is what makes wrappers and decorators able to accept "whatever the real function takes".

def log_call(*args, **kwargs):
    print("positional:", args)     # a tuple
    print("keyword:", kwargs)      # a dict

log_call(1, 2, mode="fast")
# positional: (1, 2)   keyword: {'mode': 'fast'}

# control the calling convention:
def connect(host, /, port, *, timeout):
    ...
# host  : positional-only  (before /)
# port  : either way       (between / and *)
# timeout: keyword-only    (after *)  -> connect("db", 5432, timeout=3)

08The walrus operator := +#

Assign a value to a name and use that value in the same expression, in one step.

Normal assignment (x = 5) is a statement, it can't sit inside an if or a while condition. The walrus operator (named for its := "eyes and tusks" look, introduced in Python 3.8) is an expression that both assigns and yields the value, so you can capture a result exactly where you test it.

# before: compute twice, or an extra line
n = len(data)
if n > 10: print(n)

# after: capture inside the test
if (n := len(data)) > 10:
    print(n)

while (chunk := f.read(1024)):     # read until empty, no priming read
    process(chunk)

# star use: avoid calling g(x) twice in a comprehension
results = [y for x in xs if (y := g(x)) is not None]

09Comprehensions & generator expressions +#

Build a list, set, dict, or lazy stream in one expression by describing what you want, not looping and appending.

A comprehension is a compact "for-each that produces a collection". The brackets decide the type: [] list, {} set or dict, () a generator (a lazy, one-item-at-a-time stream that never builds the whole thing in memory).

squares  = [x*x for x in nums]                 # list
uniq     = {w.lower() for w in words}          # set
by_id    = {u.id: u for u in users}           # dict
stream   = (x*x for x in nums)                 # generator — lazy, no list built

total = sum(x*x for x in big_file)          # memory-cheap over huge input

grid = [(r, c) for r in rows for c in cols]  # nested = same order as nested loops

10Decorators +#

A function that takes another function and returns a new one with extra behavior wrapped around it, applied with the @ line above a definition.

A decorator is "wrapping paper for a function". @timing above def work(): is pure shorthand for work = timing(work). The decorator receives the original function, builds a replacement that does something before/after (log, time, cache, retry, check permissions), and hands that replacement back under the same name.

from functools import wraps

def timing(func):
    @wraps(func)                       # keep func's name/docstring — don't skip!
    def wrapper(*args, **kwargs):     # accept ANY signature (§7)
        start = time.perf_counter()
        result = func(*args, **kwargs)  # call the real function
        print(f"{func.__name__} took {time.perf_counter()-start:.3f}s")
        return result
    return wrapper

@timing
def work(): ...
# work = timing(work)

Decorators that take arguments#

To write @retry(times=3) you need one more layer: an outer function that takes the arguments and returns a real decorator. Three nested functions, argument layer → decorator layer → wrapper layer.

def retry(times):                 # 1) takes the arg
    def decorator(func):           # 2) the real decorator
        @wraps(func)
        def wrapper(*a, **k):      # 3) the replacement
            for _ in range(times):
                try: return func(*a, **k)
                except Exception: pass
        return wrapper
    return decorator

@retry(times=3)
def flaky(): ...

11Caching (lru_cache / cache) & partial #

@lru_cache remembers a function's results keyed by its arguments so repeat calls are instant; partial pre-fills some arguments to make a new, simpler function.

These both live in functools. Caching (memoization) is "don't recompute what you already computed", the decorator stores a mapping from arguments to result. partial is "freeze a couple of arguments now, supply the rest later".

from functools import lru_cache, cache, partial

@lru_cache(maxsize=128)          # keep 128 most-recent results
def fib(n):
    return n if n < 2 else fib(n-1) + fib(n-2)

@cache                          # Py3.9+, unbounded lru_cache(maxsize=None)
def settings(): ...

fib.cache_info()                # hits / misses / current size — inspect it!
fib.cache_clear()               # wipe the cache

# partial: freeze arguments
basetwo = partial(int, base=2)   # a new fn: parse binary strings
basetwo("1010")                  # 10

12dataclasses +#

A decorator that writes the boilerplate (__init__, __repr__, __eq__) for a "just holds data" class straight from its type annotations.

Most small classes exist only to bundle a few fields. Writing the constructor, the repr, and equality by hand is tedious and error-prone. @dataclass reads your annotated fields and generates all of it, you just declare the shape.

from dataclasses import dataclass, field

@dataclass
class Point:
    x: int
    y: int = 0                       # default value

p = Point(3, 4)
print(p)          # Point(x=3, y=4)   <- free __repr__
Point(3,4) == Point(3,4)   # True             <- free __eq__

@dataclass(frozen=True, slots=True)   # immutable + hashable + memory-lean
class Config:
    name: str
    tags: list = field(default_factory=list)   # mutable default, done right

13ABCs & Protocols +#

Two ways to define "what a type must be able to do." An ABC enforces it by inheritance; a Protocol enforces it by shape (which methods exist).

Sometimes you want to say "anything I accept here must have a .read() method." There are two philosophies. An Abstract Base Class (ABC) is nominal, a class only counts if it explicitly inherits from the ABC and implements its abstract methods. A Protocol is structural, a class counts if it merely has the right methods, no inheritance or declaration required. This is duck typing made checkable.

from abc import ABC, abstractmethod

class Shape(ABC):                 # nominal: subclasses MUST inherit
    @abstractmethod
    def area(self) -> float: ...

class Circle(Shape):              # must implement area() or can't instantiate
    def area(self): return 3.14 * self.r**2

from typing import Protocol, runtime_checkable

@runtime_checkable
class Readable(Protocol):        # structural: no inheritance needed
    def read(self, n: int) -> bytes: ...

def load(src: Readable): ...     # accepts ANY object that has .read()

14asyncio & httpx#

A way to run many I/O-bound tasks that spend most of their time waiting (network, disk) on a single thread, by letting each one hand control back to a scheduler whenever it would otherwise block.

Most of the time a program spends waiting on a web request, it is doing nothing, just waiting for bytes. asyncio exploits that: while one request waits, the event loop runs another. async def marks a function as a coroutine (a pausable function); await is the point where it says "I'm about to wait, let someone else run". httpx is a modern HTTP client that speaks this async language (and a sync one too).

import asyncio, httpx

async def fetch(client, url):
    r = await client.get(url)         # await = "pause me while this I/O runs"
    return r.status_code

async def main(urls):
    async with httpx.AsyncClient() as client:   # reuse ONE client (pools connections)
        tasks = [fetch(client, u) for u in urls]
        return await asyncio.gather(*tasks)     # run them concurrently, collect results

asyncio.run(main(urls))               # entry point: starts & stops the event loop

15tenacity: retries done right#

A library that automatically re-runs a function when it fails, with configurable limits, waits, and conditions, so you don't hand-roll retry loops.

Networks blip. A request that fails once often succeeds a moment later. Instead of writing for _ in range(3): try/except everywhere, tenacity gives you a @retry decorator (§10 in action) whose behavior you assemble from small policy pieces: how many attempts, how long to wait between them, and which errors are worth retrying.

from tenacity import (retry, stop_after_attempt,
                      wait_exponential, retry_if_exception_type)
import httpx

@retry(
    stop=stop_after_attempt(5),                        # give up after 5 tries
    wait=wait_exponential(multiplier=1, max=10),        # 1s,2s,4s,8s,10s...
    retry=retry_if_exception_type(httpx.TransportError),  # only transient errors
    reraise=True,                                    # raise the real error, not RetryError
)
async def get(client, url):
    r = await client.get(url)
    r.raise_for_status()
    return r.json()

16Interview & Practice Q&A#

The follow-ups this material naturally invites. Read each answer aloud in 2–4 sentences.

17Key Takeaways#