The baseline is the product
Every stalled AI programme I have seen shares one detail. Somebody can describe the problem in a sentence, and nobody can produce the number.
Ask what the current handling time is and you get a range. Ask where the range came from and you get an industry report. Ask what it is in this company, in this queue, last quarter and the room goes quiet. That silence is the project risk, and it shows up months later as an argument about whether the thing worked.
Write the query before the prompt
The discipline is simple and almost nobody does it. Before any model work starts, write the query that produces today's number, run it against production data, and get someone accountable to agree in writing that it is the number.
-- The whole baseline, agreed before sprint one.
select
date_trunc('week', closed_at) as week,
count(*) as volume,
percentile_cont(0.5) within group (
order by extract(epoch from closed_at - opened_at) / 3600
) as median_hours
from tickets
where queue = 'tier-1-access'
and closed_at >= now() - interval '90 days'
group by 1
order by 1;
That is not glamorous work. It is also the only thing standing between you and a launch where success is a matter of opinion.
What a real baseline needs
A number is not a baseline. A baseline is a number plus the conditions that make it reproducible:
- A query, not an estimate. Something a second person can run and get the same answer from.
- A window long enough to show variance. Two weeks of real data beats six months of benchmarks, and a single week usually hides a seasonal effect.
- A scope that matches the agent. If the agent touches one queue, do not baseline the whole department.
- A named owner. Someone whose job it is to agree the number is correct, before anyone has an incentive to prefer a different one.
- A written record. One page. Signed. Boring. Decisive six months later.
The three numbers that decide whether to build
Once the baseline exists, the decision stops being a matter of enthusiasm:
| Number | Where it comes from | Why it matters |
|---|---|---|
| Productivity value | Baseline × realistic improvement × loaded cost | The upside, stated honestly |
| Running cost | Model and infrastructure spend at target volume | Pilot volumes flatter this badly |
| Build cost | Engineering time to get to production and keep it there | Usually the part that gets forgotten |
Subtract the second and third from the first. If what is left is not comfortably positive, the answer is to rescope, not to build and hope.
An agent that cannot show it pays for itself is not ready to ship. That is not a budget rule. It is the difference between a capability and a cost.
The uncomfortable part
Sometimes you do the arithmetic and the project dies. A use case everyone was excited about turns out to save four minutes on a task performed twice a week, and no amount of engineering changes that.
Finding this out in week one is a good outcome. Finding it out after a launch, in front of a steering committee, is the expensive version of the same discovery. The baseline is what lets you fail early and cheaply, which is the only kind of failing worth designing for.