Blog
Why I paused an AI project: the cache bill missing from the architecture
· agents, jev, prompt-caching, engineering, product
Jevis had a detailed design for frontier-model planning and cheaper execution. Prompt-cache economics exposed the hypothesis we still had to test.
Today I paused an AI project I had been preparing to build seriously.
It had a name, Jevis, a registered domain, and a second design specification running to more than forty pages. Work on the framework and website had started. Our discussion had even reached the point of deciding to build the whole design rather than keep debating feature priorities.
Then a question interrupted the momentum: what happens to the models' caches when we orchestrate them this way?
That question did not establish that multi-model collaboration is unworkable. It exposed something narrower: we had not demonstrated our central promise of delivering more reliable work within a limited budget.
This is a design-stage retrospective, not a benchmark report. We did not complete an end-to-end controlled experiment that could support a commercial conclusion. The numerical example below is hypothetical, not a measured Jevis saving or loss.
MVP and reality: conceptual cover showing an orchestration structure and its layered foundations
Why the idea was compelling
Frontier models are good at difficult judgments, but their cost and usage allowances are scarce. Many implementation tasks appear not to need the strongest model. Could we assign architecture to a frontier model and bounded execution to cheaper models, combining capability with efficiency?
Jev made that idea more concrete. TypeSafe describes it as a System One model that returns typed decisions: choices, scores and probabilities rather than free-form text. It can support routing and checks; it is not a generative model that writes code or summaries itself. TypeSafe's introduction
Our proposed division was straightforward: the frontier model defines correctness, ordinary models perform the repetitive implementation, and Jev helps transfer and evaluate the specification.
For an interface migration, a strong model might define the migration rules and invariants. Multiple workers would then edit separate files. Jev would supply signals on bound
Available in: en, zh-CN