Are you letting the machines decide your metrics?

How a target stops matching the outcome when we’re not looking

Every metric is a proxy for something harder to measure directly: a sales target stands in for sustainable revenue; average response time stands in for service quality.

But once a target is set, a gap can open between the target number and the thing it was meant to represent, because the number can move without the underlying reality moving with it: a sales team can hit its target through discounting that erodes margin; a support team can hit its response time average by closing tickets before they’re resolved. Without checks to detect it, the gap sits unmanaged between what the target counts and what actually happened.

AI’s default is the path of least resistance

AI can widen this gap. Increasingly, the source material behind the metric, such as a report or a review, isn’t written by a person but by an algorithm. Then another AI program summarizes the material and scores it. The result: output produced by one AI that another AI is well suited to read favorably in an exchange between two machines without human oversight.

What good (target) practice looks like

Detecting and correcting for this phenomenon is a matter of periodically checking what the AI-mediated version of “good” has blindly created, and testing that against a small sample of the underlying work itself. Try the following:

• Spot-check outputs against source material, not against the score they generated.
• Ask what the system would produce if instructed to optimize for the outcome you actually want, rather than for how the output comes across.
• Treat a suspiciously smooth or steadily improving trend in AI-scored material as a reason to look closer, not as a reassurance that things are going well.