Tue, Dec 8, 2026 · 1:00 AM – 2:15 AM
Most teams inherit at least one agent they did not write: a vendor product, a contractor's prototype, or a colleague's weekend build that is now load-bearing. This workshop is about getting a defensible read on one of those without access to the prompts or the training data. You will start from a black-box agent, build a trace harness around it, and design an eval set from its observed failures rather than from its documentation. We cover sampling real traffic without leaking it, writing assertions that can actually fail, and estimating cost per resolved task. Bring a laptop with Python 3.11 and Docker. You will leave with a running harness, a scored baseline for the agent you brought, and a short list of the failures its vendor page does not mention.