cuesheet library released on PyPI for LLM test fixtures

cuesheet 0.1.0 landed on PyPI, promising developers a way to record and replay LLM API interactions without network calls. The package works by wrapping tests in a @cuesheet.cassette(...) decorator; the first run hits the real provider, stores the request‑response pair in a YAML file, and subsequent runs replay the exact bytes [1].

Built on top of httpx, cuesheet supports any Python SDK that uses the standard transport layer, including Anthropic, OpenAI, Gemini, Mistral, Cohere, Groq, DeepSeek, Together and LiteLLM. It captures both synchronous and asynchronous calls, and can record streamed responses as raw SSE chunks, replaying them at configurable speed. The YAML format is chosen for git‑friendly diffs, and the library automatically scrubs API keys, JWTs and common email patterns before writing files, reducing the risk of secret leakage [1].

Beyond the core decorator, cuesheet ships a pytest plugin that auto‑discovers cassettes, a CLI for listing, inspecting and statistics, and an optional local web UI built with FastAPI and HTMX. The UI watches the filesystem and updates in real time as tests record, offering a live view of cassette contents without requiring a daemon or persistence layer [1]. Installation is straightforward via pip install cuesheet for the core library, with extra dependencies for the web UI (pip install "cuesheet[web]") or all features (pip install "cuesheet[all]"). The project requires Python 3.10+ and is released under the MIT license [1].

The library’s design addresses three chronic pain points in LLM testing: slow and flaky CI calls, brittle hand‑rolled mocks, and the inability of existing HTTP fixture tools to handle LLM payloads or stream replay. By intercepting at the httpx transport layer, cuesheet remains provider‑agnostic, though the maintainers note that SDK‑specific quirks may still surface and encourage users to file issues with minimal reproductions [1].

If adopted widely, cuesheet could cut token costs and CI runtimes dramatically, especially for teams that rely on multiple LLM providers. Its replay‑only mode (mode="replay_only") is recommended for CI environments to guarantee no accidental live calls, turning a previously risky testing step into a deterministic, cost‑free process. The open question is how quickly the ecosystem will standardize on such fixtures and whether providers will adjust their SDKs to accommodate or counter this interception approach.