- 1Tech Note
Retry counts told us nothing about the run (Part 1)
A run that called tools three times and one that failed quality checks three times both logged 'retry 3'. Splitting one loop into explicit states.
- 2Tech Note
We split the repos but not the dependencies (Part 2)
The engine had its own module, and running it still needed the product's config object, DB session, and canvas classes. Closing a real boundary.
- 3Tech Note
It installed, but we couldn't call it the same run (Part 3)
We packaged a workflow as a wheel and installed it elsewhere. Calling a tool exposed every assumption that only held inside the product.
- 4Tech Note
The model that wrote the answer was also grading it (Part 4)
When one model writes an answer and then judges it, generation method and pass criteria share a context. Splitting doing the work from accepting it.
- 5Tech Note
The submit tool fired and the run kept going (Part 5)
A terminal tool succeeded and execution continued; a record from a previous turn could end the current one early. Where a run actually ends.
- 6Tech Note
We thought we had held out a validation set (Part 6)
Building an automatic config search, we added promotion thresholds and a validation set. The component proposing candidates was reading that record.
- 7Tech Note
The 30x gap wasn't the model (Part 7)
Two models on the same workflow gave 2,488 and 84 seconds. Matching iteration ceilings and output limits turned 2,488 into about 100.
- 8Tech Note
Letting the nearest memory always win was wrong (Part 8)
A session request is more specific than a platform policy, so it seemed it should win. Left that way, a user could override policy.
- 9Tech Note
The tools were listed but couldn't be called (Part 9)
An allowlist meant to narrow search scope also dropped the tools the user had connected by hand. Name, schema, and permission at three moments.
- 10Tech Note
We burned every retry re-grading the same answer (Part 10)
Quality retries were exhausting the ceiling. The model was progressing through tool calls while the judge re-graded the previous failed answer.
Try XGEN free for 15 days — nothing to install, runs in your browserStart free trial
