Companies Should Measure Completed Work, Not the Number of AI Licences

On July 17, OpenAI published a proposal for how companies should evaluate the return on investment in artificial intelligence. It criticised traditional software metrics such as purchased licences, active users and logins. Those figures indicate adoption, not economic results. The central metric should be completed work. A company should count how many customer requests the system helped resolve, how many contracts it reviewed, how many code changes it prepared and how many employee hours it saved. For every activity, it should also record quality and whether a person later corrected the output. This approach exposes a problem hidden by normal usage statistics. An employee may use a paid assistant every day only to polish wording or prepare low-value material. Another workflow may run only a few times a month but accelerate a contract or prevent an expensive mistake. OpenAI recommends measuring the cost of a successfully completed task. The calculation should include more than token or licence prices. Companies must add calls to other services, computing power, human review, corrections, reruns and integration costs. A cheaper model may cost more if it fails more often. Accuracy matters just as much. Companies should not track only average performance but also the frequency of severe errors. A legal assistant may review nine contracts correctly and miss a high-liability clause in the tenth. An average success rate of 90 percent says little about the actual business risk. “How do we get more value from our AI spending?” is the question OpenAI says chief financial officers are asking. Its second key formulation is: “Start with the work itself.” The supplier is shifting the purchasing logic away from subscriptions and toward measurable outcomes. Finance teams therefore need to connect technical statistics with operating data. Token consumption alone is not enough. It must be linked to a process identifier, task outcome, employee time and measurable economic benefit. Without that connection, companies cannot tell which application creates value and which one merely increases the bill. Performance measurement can nevertheless create false precision. In customer service, it is easy to count closed cases but harder to determine whether faster replies reduced satisfaction or caused repeat contacts. Companies therefore need a combination of speed, quality, cost and long-term impact. This approach will also affect supplier contracts. Customers may demand a stable price for a successful process, guaranteed accuracy or the ability to return to an older model version. Suppliers will seek to separate the price of their service from costs caused by poor corporate data or badly described procedures. OpenAI's own article promotes a model that supports higher usage where value increases. It is therefore also a commercial argument from the vendor. Even so, the advice is practical: AI spending should not be one broad budget item. It should be divided by specific processes, owners, expected benefits and acceptable risk.

Companies Should Measure Completed Work, Not the Number of AI Licences

The full article is available to users registered at Hard Skills