Measuring AI against commercial outcomes

AI tools are now being sold to SMEs at a pace that outstrips most businesses' ability to evaluate them. The demonstrations are impressive, the claims are broad, and the pressure — from boards, peers and suppliers — is to adopt something, quickly. What is usually missing is the discipline that would be applied to any other commercial investment: a clear statement of the outcome the tool is expected to move, and a way of measuring whether it did.

Activity is not an outcome

Most AI tools are justified, after the fact, by activity measures: emails drafted, calls summarised, content produced, hours notionally saved. These numbers are easy to generate and almost impossible to falsify — which is precisely why they should be treated with caution. A tool that doubles the volume of outbound messages has achieved nothing commercially unless those messages convert. A meeting summary saves time only if that time is redeployed to something productive.

The commercial questions are the same as for any investment: did revenue, margin or productive capacity change, and can the change be attributed with reasonable confidence?

Three outcome tests worth applying

Revenue: does it help win or keep business?

If a tool claims to improve selling — better targeting, faster responses, improved proposals — the evidence should appear in conversion rates, win rates or retention, measured against the period before adoption. If those measures were never baselined, the claim cannot be tested; that is a reason to establish the baseline, not to take the claim on trust.

Margin: does it reduce the cost of delivering the same outcome?

Genuine efficiency shows up as lower cost per quotation, per order processed, per enquiry handled — or as work absorbed without additional headcount. "Time saved" only becomes margin when it either reduces cost or is reinvested in revenue-generating work. Time saved and quietly reabsorbed into the working day is a benefit on paper alone.

Capacity: does it release skilled people for higher-value work?

For many SMEs this is the most realistic near-term gain: administrative work removed from estimators, engineers and salespeople, whose released hours have a clear alternative use. The test is whether that alternative use actually happens — which is a management decision, not a software feature.

The conditions that make measurement possible

• A baseline before adoption. Whatever the tool is meant to improve must be measured first. Without a starting position, every result is anecdote.
• A defined trial with an owner. A bounded period, a named owner, agreed success measures and a genuine willingness to stop if they are not met. Tools that survive only because cancelling feels awkward accumulate into a quiet monthly cost.
• Attribution honesty. Commercial results move for many reasons. Where a clean comparison is impossible, say so, and weigh the evidence accordingly — rather than crediting the newest tool with whatever improved.
• Total cost, not licence cost. Subscription fees are usually the smallest component. Integration, data preparation, checking and correcting output, and management attention are all real costs, and all routinely omitted from the comparison.

A discipline, not a verdict on AI

None of this is an argument against adopting AI — the productivity gains available to SMEs are real, and businesses that ignore them will compete against businesses that did not. It is an argument for adopting it the way disciplined businesses adopt anything: against a stated commercial outcome, from a measured baseline, with the willingness to keep what works and stop what does not. The tools will keep changing. The discipline is what compounds.