Generation got cheap. Proof did not.

I can generate a working prototype of almost anything before lunch. So can your intern. So can a model with a browser.

McKinsey's 2026 survey put numbers on the hangover. Four in five people say AI makes them more productive at their desk. About a third of companies can point to any earnings impact. A sliver can call it significant. Individual speed went up. The P&L did not.

That is not a mystery if you have had to keep an AI feature alive. Generation was never the constraint. Proof was.

The agent will write. Someone has to check.

On my own stack, nothing important ships because a model said it was done. Build has to pass. Tests have to pass. The thing has to be exercisable the way a user would exercise it. If I cannot see the failure, I do not merge.

That sounds like ordinary engineering. It is, and most AI projects skip it because the demo never runs twice.

A demo has one happy path and a room that wants to believe. Production is retries that quietly double-book, an 800-millisecond pause that feels like a dropped call, a tool the agent was not supposed to hit, a sentence that was fluent and incomplete. The screenshot is marketing. The state-transition log is the product.

I care more about four receipts than I do about the prompt: the tool call that was denied, the partial failure, the human approval, the recovery after a restart. If those states do not exist in the system, you do not have an agent. You have a script that has not failed yet.

Coding agents fooled us

Software is unusually kind to this. Tests exist. Diffs exist. You can see whether the branch is still green. That is why coding agents scaled first, and why companies now skip software purchases because they can build the thing in-house.

Most other work is not like that. A tax return, a clinic note, a guest request at 11pm. The outcome takes hours or days, the data is messy, and "looks right" is not a verifier. If you only score the final answer, you cannot tell whether the path will hold on the next ten cases.

So I score the path. Did it call the live data or guess. Did it abstain when the seed was missing. Did it stay inside the allowlist. Completeness against what a human actually provided, not eloquence against a rubric the model can game.

What this does to the PM job

If your identity was "I wrote the spec and the team built it," cheap generation ate that. The job that remains is defining what done means for a probabilistic system, and refusing to ship without the proof.

Acceptance criteria for AI are not "when the user clicks X, Y happens." They are the distribution: unsure, timeout, low confidence, missing context, human in the loop. They are also the economics: blended cost per active user, fallback rate, p99 latency. I have written that list before. I am writing it again because teams still treat evals as an engineering nicety.

Evals are the product. They are how you know the thing you generated is still the thing you meant.

The companies stuck on the ROI road are not short on models. They are short on people who will withhold the merge until the proof exists. Generation will keep getting cheaper. That only raises the premium on the person who can tell finished from costume.