Lean machine, continued: making the improvements repeatable

Make the gains repeatable

In Lean machine, we followed the measurements from a large always-on catalog to a small core with optional modules. The result was a much lighter starting point for an agent session.

That gave us the next question: how do we carry the same discipline through the work that happens after startup?

An agent opens its instructions. A harness loads context. A benchmark compares two approaches. A developer picks a playbook and installs its dependencies. Each is an opportunity to make the next step more focused, more predictable, and easier to verify.

We mapped that whole path, then worked through it in reviewed waves. The result spans the plugin runtime, the tools we use to measure it, and the build process for our paid prompt packs.

Start with the map

The inventory covered ten plugin roots: 84 skills, 30 agents, 14 commands, and 186 supporting resources. Alongside them, we cataloged 204 playbooks across five paid packs.

Those numbers gave us boundaries. We could follow an instruction from its source to its installed location, connect a playbook to the skills and agents it uses, and decide which checks belonged at each step.

That matters because these are connected products. A reference moved into an optional module needs to remain discoverable when that module is installed. A playbook that calls an agent needs the agent's provider in its setup instructions. A benchmark needs to find the same qualified skill that a user would invoke.

The catalog turned those relationships into something we could check repeatedly.

Put detail where the task needs it

The first post concentrated on descriptions: the small pieces of text a model sees before choosing a tool. This pass extended the same idea to the instructions it reads after making that choice.

Our prompt-engineer agent covers settings, hooks, slash commands, and skill authoring. Those are related jobs, but a request to write one hook rarely needs four complete manuals at once.

We kept the routing and working instructions in the agent body and moved the detailed material into four references, loaded when the task calls for them.

The manuals still ship. The agent now starts with a smaller working set and a clear route to the relevant detail. Reference resolution accounts for both the source repository and extracted plugin installs, so the structure remains useful after distribution.

This is a source-size measurement. It does not establish a matching reduction in runtime tokens or task latency. It gives us a concrete improvement to evaluate: less instruction text loaded up front, with the specialist material available on demand.

We also made the startup budget a continuing check. CI now includes agent, skill, and command descriptions in its estimates, with limits of 3,000 estimated tokens for core and 18,000 for the full suite. A new specialist has to fit the shared budget.

Give each harness the right work

Claude Code, Codex, Grok Build, and OpenCode share our source catalog. Their adapters translate that catalog into the conventions of each host.

The useful design question is what should stay consistent and what should be handled locally. An agent's intended tool scope should survive translation. The way a host represents that scope belongs in its adapter. Likewise, identifying the current host needs evidence about the running session; finding a CLI on the machine answers a separate question about which tools are available.

In OpenCode, the initial context snapshot now loads once per session. Concurrent requests share that load, and a failed load can be retried. Message-specific routing still runs for each message, and tool safeguards still run for each tool call.

That division removes repeated setup work while keeping the decisions that depend on the current request current. It is the same principle as the reference split: load stable information at the point where it becomes useful, then do fresh work where fresh input matters.

Measure the outcome you want to improve

Smaller instructions are useful when the resulting workflow still does its job. Our benchmark tooling now makes that comparison more explicit.

Baseline and skill-assisted runs start with the same fresh, tool-free conditions. Evaluation contracts identify the cases and assertions being tested. Results retain baseline successes, failures, and negative cases, so a useful comparison includes the requests where the skill should stay out of the way.

The reports also distinguish agent selection from actual workflow execution. Reaching the intended specialist is one result; completing the task correctly is another. Usage follows the same rule: an observed zero is zero, and unavailable telemetry stays unknown.

These details make the numbers more useful for decisions. A candidate that preserves quality while using less context can be worth adopting. A candidate with a higher score needs enough evidence to show the difference holds on fresh tasks. The runner helps us ask both questions without collapsing them into one headline.

Carry the process into the playbooks

For someone using a paid pack, the practical path starts with choosing a task and getting its prerequisites in place.

We reviewed setup information across all 204 playbooks and made the pack builder resolve both skill dependencies and named agent providers. Each playbook points to the plugins its own workflow needs. The full pack inventory remains available for someone who wants the broader collection.

We also reviewed the prose where specific API behavior, versioned contracts, and follow-up references determine the next action. The benefit is concrete: setup instructions and task instructions can be checked together, as one workflow.

The pack builder now validates content before replacing its output and generates a VALIDATION.md evidence report alongside the playbooks. It records the evidence available for each workflow and distinguishes historical records from structured live runs. Readers get a clearer basis for choosing what to run and what to verify in their own environment.

All 204 playbooks pass the content gate. We built and inspected six local archives: the five packs and the complete suite. Those checks cover source content and packaging; fresh execution of buyer workflows and delivery of updated customer archives remain separate release steps.

Work in waves, keep the review close

We used Orchestra to dispatch bounded tasks in three reviewed waves. Each worker received a specific area to own, the behavior to preserve, and the evidence needed to finish. That let us use smaller models for focused implementation while keeping integration and review centralized.

The briefs mattered as much as the dispatch. “Improve the benchmarks” is too broad. A task with named files, defined input and output behavior, and concrete acceptance checks gives a worker something it can finish and a reviewer something it can assess.

The integrated plugin changes passed twelve local verification gates, followed by installation checks in CI. All ten extracted plugins passed Claude's strict validator. The site changes passed 821 tests, TypeScript checks, and repository-wide lint.

The durable result is the process those checks now support: catalog the relationships, load detail when needed, preserve intent across adapters, compare complete results, and package the evidence with the work.

That is how the gains from the first Lean machine pass continue to compound. The next skill, adapter change, or playbook enters a system that already knows what to measure.

The plugin implementation and verification records are available in the public update. Explore the software development pack for workflows built around this approach.