Design, write and review context managers so that every resource an LLM app,
RAG pipeline, agent or training loop acquires — files, DB and vector-store
connections, HTTP clients, GPU memory, locks, semaphores, temp directories,
global state like env vars or model.training — is released on every exit
path, exceptions are never swallowed by accident, and async code uses the
async protocol. Includes a static leak finder and a generator for typed,
documented managers and AI-engineering recipes.
Trigger Conditions
Activate this skill when the user:
- Asks to write a context manager, a
with-able class, or a@contextmanager/@asynccontextmanagerfunction - Has code that opens files,
sqlite3/psycopgconnections, sockets,httpx/aiohttp/requestsclients,ThreadPoolExecutor,subprocess.Popenor temp dirs withoutwith - Sees
ResourceWarning,Too many open files, "database is locked", deadlocks,generator didn't stop,generator didn't yield,'NoneType' object has no attributeon anasvariable, or__exit__() takes N positional arguments - Reports exceptions that "disappear" inside a
withblock - Needs setup/teardown for models (eval mode,
inference_mode, autocast, GPU cleanup), API clients, FastAPIlifespan, DB transactions, checkpoints, seeds, env vars, tracing, rate limits or agent sandboxes - Wants to manage a variable number of resources (
ExitStack) or cancel background tasks on exit
Workflow
1. INVENTORY -> list every resource/state change: what acquires it, who owns it, how it is released
2. SCAN -> python scripts/ctx_tool.py leaks <file|dir> (static; never runs code)
3. CHOOSE -> with-statement / @contextmanager / class / ExitStack / lifespan (table below)
4. WRITE -> setup before try, one yield inside try/finally, falsy __exit__, typed
(scripts/ctx_tool.py scaffold ... gives a verified skeleton)
5. EDGE -> partial setup failure, body raises, early return, cancellation, reuse, threads
6. TEST -> teardown on success + on exception + exception still propagates
7. RE-SCAN -> ctx_tool.py leaks --strict exits 0Choosing the Tool
| Situation | Use |
|---|---|
Resource already supports with (files, httpx.Client, pools, Popen, locks, TemporaryDirectory) | Plain with — do not wrap it |
Object has close() but no __exit__ | contextlib.closing(obj) / aclosing(obj) (3.10) |
| One-off setup/teardown around a block | @contextmanager generator |
Async setup/teardown (await in either half) | @asynccontextmanager or __aenter__/__aexit__ |
| Object with methods that owns resources (client wrapper, session, index) | Class with __enter__ returning self + close() |
| N resources decided at runtime, conditional managers | ExitStack / AsyncExitStack |
| Acquire several, keep them on success, release on partial failure | ExitStack + pop_all() |
| App-lifetime resources (models, clients, pools) in a web service | FastAPI lifespan |
| Optional manager | cm if flag else nullcontext() |
| Ignore one specific exception | contextlib.suppress(FileNotFoundError) |
| Reusable or re-entrant behaviour, or suppression logic | Class (with depth counter / RLock for re-entrancy) |
Protocol Rules (non-negotiable)
| Rule | Detail |
|---|---|
__enter__ returns the thing bound by as | Usually self; forgetting it binds None |
__exit__(self, exc_type, exc, tb) | Exactly four params (or *exc_info) |
__exit__ returns falsy | Truthy suppresses the exception — only for specific, handled types: return exc_type is not None and issubclass(exc_type, X) |
If __enter__ raises, __exit__ does not run | Undo partial setup inside __enter__ (or build it with ExitStack + pop_all()) |
__exit__ runs on return/break/continue and on BaseException | Including KeyboardInterrupt and asyncio.CancelledError |
An exception in __exit__ replaces the body's exception | Log secondary cleanup failures when exc is not None |
| Multiple items enter left-to-right, exit right-to-left | Parenthesized form is 3.10+ |
| Async protocol is separate | async def __aenter__/__aexit__; async with only |
@contextmanager Rules
from contextlib import contextmanager
from typing import Iterator
@contextmanager
def leased_gpu(pool: "DevicePool") -> Iterator[int]:
device = pool.take() # 1. setup BEFORE try (nothing to undo if it fails)
try:
yield device # 2. exactly one yield, inside try
finally:
pool.give_back(device) # 3. teardown in finally (runs on error too)| Do | Don't |
|---|---|
try/finally around yield whenever code follows it | Cleanup after a bare yield (skipped on error) |
except ...: rollback(); raise | except Exception: log(...) without raise (suppresses) |
else: for success-only work (commit) | Commit in finally (commits failed work) |
One yield per path (if/else branches OK) | yield in a loop or in an except after the main yield |
Call the function for each with | Reuse the returned object (single use) |
Before / After
Leaked resources in a RAG ingest job
# BEFORE
def ingest(paths, db_path):
conn = sqlite3.connect(db_path) # never closed
pool = ThreadPoolExecutor(8) # never shut down
for p in paths:
text = open(p).read() # closed only by GC
chunks = list(pool.map(embed, split(text)))
conn.executemany("INSERT INTO chunks VALUES (?, ?)", chunks)
conn.commit()# AFTER
from contextlib import closing
from pathlib import Path
def ingest(paths: list[Path], db_path: str) -> None:
with closing(sqlite3.connect(db_path)) as conn, ThreadPoolExecutor(8) as pool:
with conn: # sqlite3: transaction, not close
for p in paths:
text = p.read_text(encoding="utf-8")
chunks = list(pool.map(embed, split(text)))
conn.executemany("INSERT INTO chunks VALUES (?, ?)", chunks)exit that swallows everything
# BEFORE # AFTER
class EvalMode: class EvalMode:
def __enter__(self): def __enter__(self) -> "EvalMode":
self.m.eval() self.was_training = self.m.training
self.m.eval()
return self
def __exit__(self, *a): def __exit__(self, exc_type, exc, tb) -> None:
self.m.train() self.m.train(self.was_training)
return True # hides every bug # falsy: exceptions propagateLock without finally
# BEFORE # AFTER
lock.acquire() with lock:
cache[key] = compute() # raises -> cache[key] = compute()
lock.release() # deadlockAsync client per request → shared client in lifespan
# BEFORE: new pool + TLS handshake per call, never closed
async def complete(prompt):
client = httpx.AsyncClient()
r = await client.post(URL, json={"prompt": prompt})
return r.json()# AFTER
from contextlib import asynccontextmanager
from typing import AsyncIterator
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
app.state.http = httpx.AsyncClient(timeout=httpx.Timeout(60, connect=5))
try:
yield
finally:
await app.state.http.aclose()
app = FastAPI(lifespan=lifespan)AI-Engineering Patterns (summary)
| Need | Pattern |
|---|---|
| Inference without autograd | with torch.inference_mode(): (no_grad() if outputs feed training) |
| Mixed precision | with torch.autocast(device_type="cuda", dtype=torch.bfloat16):; backward() outside |
| Validation inside a training loop | Save model.training (per submodule), eval(), restore in finally |
| Free GPU memory between phases | del refs → gc.collect() → torch.cuda.empty_cache() in finally |
| Latency / token / cost accounting | Meter yielded from @contextmanager, duration in finally, perf_counter() |
| Tracing | OpenTelemetry start_as_current_span, or contextvars-based span; reset token in finally |
| Temporary API key / base URL | Save & restore os.environ (not thread-safe; prefer explicit client args) |
| Concurrency/rate limits | One shared asyncio.Semaphore; back off outside the slot |
| Model + client loading once | FastAPI lifespan; with TestClient(app) in tests |
| Per-request DB session | Dependency with yield (commit / rollback / close) |
| Checkpoints, indexes, manifests | Atomic write: temp in same dir → fsync → os.replace |
| Reproducible evals | Seed and restore random/NumPy/torch RNG state |
| Agent code execution | Temp dir sandbox + subprocess.run(cwd=..., timeout=...), never os.chdir |
| Background heartbeat for a block | create_task on enter; cancel() + await in finally |
Full code: references/ai-engineering-recipes.md.
Async Rules
httpx.AsyncClient,aiohttp.ClientSession,asyncio.Lock/Semaphore,TaskGroup,asyncio.timeout(3.11) needasync with.- Never swallow
asyncio.CancelledErrorin__aexit__or aroundyield. - Cleanup that must finish during cancellation:
await asyncio.shield(release()), kept short. - Close async generators that hold connections with
contextlib.aclosing(3.10). - No blocking I/O in
__aexit__; useawait asyncio.to_thread(fn). - Create shared semaphores/locks once;
async with asyncio.Semaphore(8):in a worker limits nothing.
Details: references/async-context-managers.md.
ExitStack Essentials
from contextlib import ExitStack
def open_shards(paths):
with ExitStack() as stack:
files = [stack.enter_context(open(p, encoding="utf-8")) for p in paths]
stack.callback(log.info, "closed %d shards", len(files))
return merge(files)
class Index:
def __init__(self, vec_path, meta_path):
with ExitStack() as stack: # releases on partial failure
self.vec = stack.enter_context(open(vec_path, "rb"))
self.meta = stack.enter_context(closing(sqlite3.connect(meta_path)))
self._stack = stack.pop_all() # success: keep them open
def close(self) -> None:
self._stack.close()Review Checklist
- Every acquire has a release on every path (success, exception, early return, cancellation)
- No
open(...).read(),json.load(open(...)), unclosedSession/Client/pool/Popen -
close()calls that are not infinallyreplaced bywith -
@contextmanager: setup beforetry, singleyieldintry, cleanup infinally, everyexceptre-raises -
__enter__returns a value;__exit__has 4 params and returns falsy unless suppression is intended and narrow - Partial setup in
__enter__undone on failure - Sync vs async protocol matches the call site
- Locks/semaphores created once and shared
- Global state (env, cwd, stdout, logging level,
model.training, RNG) restored infinally - Lazy iterators do not escape the
withthat owns their source - Tests cover teardown on exception and that the exception still propagates
Output Format
When writing or fixing context managers, answer with:
- Findings (review tasks): table of
line | rule | problem | fix, ordered by severity (pastectx_tool.py leaksoutput where useful). - Code: the complete manager or the rewritten function — typed, with a docstring stating what is acquired, what is released, whether exceptions are suppressed, and whether it is reusable/re-entrant/thread-safe.
- Usage: a short
with/async withexample in the user's domain. - Tests: at least "cleanup on success", "cleanup on exception + exception propagates", and (async) "cleanup on cancellation".
- Notes: version requirements (3.10 parenthesized
with/aclosing, 3.11chdir/TaskGroup), thread/async-safety caveats.
Scripts
scripts/ctx_tool.py — standard library only, Python 3.9+, never imports or runs the analysed code.
# Static leak and protocol-bug scan (files, directories, or - for stdin)
python scripts/ctx_tool.py leaks app/ingest.py
python scripts/ctx_tool.py leaks src/ --json
cat agent.py | python scripts/ctx_tool.py leaks - --strict # exit 1 on medium/high findings
python scripts/ctx_tool.py leaks src/ --strict --fail-on high # CI gate on high only
python scripts/ctx_tool.py leaks src/ --min-severity medium # hide low-severity notes
python scripts/ctx_tool.py rules # CTX001..CTX012
# Scaffold a typed, documented manager (verified with ast.parse; exit 2 if it does not parse)
python scripts/ctx_tool.py scaffold vector_tx --kind generator \
--setup "conn = pool.getconn()" --teardown "pool.putconn(conn)" --yields conn
python scripts/ctx_tool.py scaffold ModelSession --kind async-class \
--setup "self.client = httpx.AsyncClient()" --teardown "await self.client.aclose()" --import "import httpx"
python scripts/ctx_tool.py scaffold lease --kind async-generator \
--setup "conn = await pool.acquire()" --teardown "await pool.release(conn)" --yields conn
python scripts/ctx_tool.py scaffold --list-recipes
python scripts/ctx_tool.py scaffold --recipe gpu-inference # timer|temp-env|cwd|gpu-inference|
python scripts/ctx_tool.py scaffold eval_seed --recipe seed --json # atomic-write|semaphore|span|seed| Rule | Severity | Catches |
|---|---|---|
| CTX001 | high/medium/low | Resource opened outside with: never closed (high), closed outside finally / inline .read() / passed along (medium), module-level or handed to another owner (low) |
| CTX002 | high/low | .acquire() without try/finally release; acquire inside the try (low) |
| CTX003 | high | @contextmanager cleanup after yield not protected by finally |
| CTX004 | high | No yield, yield in a loop, or two yields on one path |
| CTX005 | high | __exit__/__aexit__ unconditionally returns a truthy value |
| CTX006 | low/medium | __enter__ returns None (medium when the file binds it with as) |
| CTX007 | medium | except around yield that does not re-raise |
| CTX008 | high | Wrong __exit__ arity |
| CTX009 | high | Sync/async protocol mix-up |
| CTX010 | high | Plain with on httpx.AsyncClient / aiohttp.ClientSession |
| CTX011 | high | with threading.Lock(): / asyncio.Semaphore() created in the with |
| CTX012 | medium | torch.no_grad(), suppress(...) etc. as a bare statement |
Exit codes: 0 = clean (or findings without --strict), 1 = --strict and a finding at or above
--fail-on (default medium), 2 = usage error, unreadable file, syntax error, or scaffold failed to parse.
Heuristics are intraprocedural: confirm "handed to another owner" (low) findings by reading the callee.
Additional Resources
| File | Content |
|---|---|
| protocol-reference.md | with semantics, __enter__/__exit__ return values, exceptions in __exit__, multi-item with, reusable vs re-entrant, typing, version notes |
| contextlib-reference.md | Every contextlib tool: contextmanager, asynccontextmanager, ExitStack, AsyncExitStack, closing, aclosing, suppress, redirect_*, nullcontext, chdir, ABCs, ContextDecorator |
| async-context-managers.md | Async protocol, cancellation, semaphores, background tasks, TaskGroup/timeout, httpx/aiohttp lifecycle, aclosing, dual sync/async managers |
| ai-engineering-recipes.md | torch no_grad/inference_mode/autocast, eval switch, GPU memory, meters, spans, env vars, rate limits, FastAPI lifespan, transactions, atomic checkpoints, seeds, sandboxes |
| recipes.md | General recipes: files, temp files, cwd, locks, pools, subprocesses, sockets, patching, logging, warnings, signals, timing, object pools |
| pitfalls-and-testing.md | 20 pitfalls with fixes and rule ids, generator pitfalls, lazy iterators, thread safety, pytest patterns, runtime leak detection |
python-beginner-concept-tutor
Teach Python concepts to beginners heading into AI engineering (data scientists, analysts, students, career switchers): plain-language definitions, analogies, minimal examples, line-by-line walkthroughs, common mistakes, check-your-understanding questions and mini exercises; Socratic hints before solutions; explaining tracebacks; mutable vs immutable, references and aliasing, is vs ==, integer division, floating point, slicing and off-by-one, indentation errors; a staged learning path from variables to calling an LLM API, embeddings and a tiny RAG. Ships a script that lists the concepts a snippet uses in learning order and traces execution step by step like Python Tutor.
python-comprehension-refactorer
Refactor Python loops into list, dict and set comprehensions and generator expressions, and unroll comprehensions that are too clever: filtering, mapping, nested-loop flattening, conditional expressions, walrus in comprehensions, dict inversion, grouping with defaultdict/Counter, lazy generators for sum/any/all/max, streaming large files, itertools (chain, islice, groupby, batched, pairwise, accumulate, product), scoping rules, side-effect anti-patterns and performance (when map/filter or numpy/pandas vectorization win). AI-engineering focus: building prompt lists, batching texts for embeddings, filtering retrieved chunks by score, id->embedding dicts, flattening tool-call results, lazy dataset streaming and token counting.
python-dataclass-converter
Convert boilerplate Python classes, dicts, tuples and JSON payloads into dataclasses, or into pydantic v2 BaseModel, TypedDict, NamedTuple, attrs or msgspec when that fits better. Covers every @dataclass option (frozen, slots, kw_only, order, unsafe_hash, match_args, weakref_slot), field() (default_factory, init=False, repr, compare, hash, metadata), __post_init__, InitVar, KW_ONLY, inheritance and default ordering, replace(), asdict deep-copy pitfalls, hashing/equality, serialization and validation. AI focus: LLM request/response and chat Message types, RAG chunks with embeddings (numpy eq gotcha), eval records, configs, typed OpenAI/Anthropic JSON.
python-exception-handling-designer
Design and review Python exception handling: custom exception hierarchies with retryable flags, what to catch where, EAFP vs LBYL, chaining with raise ... from, else/finally, logger.exception, add_note, ExceptionGroup and except* with asyncio.TaskGroup, warnings, asserts vs validation, FastAPI error translation, retries with backoff and jitter, timeouts, cancellation, pytest.raises, plus LLM-app failures: 429 rate limits with retry-after, 5xx overloaded, context-length exceeded, content filters, broken streams, malformed JSON repair loops, tool-call errors fed back to the model, partial embedding batch failures, CUDA OOM batch halving, vector DB outages, circuit breakers and model fallbacks.
python-idiomatic-code-reviewer
Review Python code for non-idiomatic, unpythonic or risky patterns and rewrite it into clean modern Python: PEP 8 / PEP 20, EAFP vs LBYL, iteration idioms (enumerate, zip, comprehensions, any/all), unpacking, f-strings, pathlib, truthiness, `is None`, mutable defaults, walrus, match statements, collections / itertools / functools, logging instead of print, with-blocks and resource handling. Includes AI-engineering checks: hardcoded API keys, time.sleep or requests inside async def, missing HTTP timeouts, unbounded retry loops, model clients built at import time, prompt string concatenation, sequential awaits, eval() on LLM output. Ships a 46-rule AST reviewer with severities and suggested rewrites.
python-type-hints-annotator
Add accurate, modern type hints to Python and fix mypy / pyright (pyrefly, ty) errors: builtin generics, X | None, TypeAlias and the 3.12 type statement, TypeVar and PEP 695 generics, Protocol, Callable / ParamSpec / Concatenate, TypedDict (Required, NotRequired, ReadOnly), Literal, Final, ClassVar, overload, Annotated, TypeGuard / TypeIs, Self, Never, TypeVarTuple, async types, Iterable vs Sequence vs Mapping, TYPE_CHECKING, stubs, py.typed, gradual strictness. AI-engineering focus: typed LLM message dicts and tool schemas, pydantic vs TypedDict, numpy NDArray, torch / jaxtyping shapes, streaming generators, async clients, callbacks, generic VectorStore protocols, typed config. Ships an AST audit: coverage gate, legacy-typing rewrites, inferred types.