The Enterprise AI Measurement Problem

For years, organizations have measured software success using familiar metrics:

• Number of licenses purchased
• Active users
• Login frequency
• Feature adoption
• Subscription renewals

Those metrics made sense because traditional software acted primarily as a productivity tool.

Artificial intelligence is fundamentally different.

An AI system doesn’t create value because someone opens ChatGPT or activates Microsoft Copilot. It creates value only when meaningful work is completed accurately, efficiently, and consistently.

On 17 July 2026, OpenAI formally acknowledged this shift by introducing a new enterprise measurement framework called “Useful Intelligence per Dollar.” Rather than focusing on model costs or token consumption, OpenAI argues that organizations should evaluate AI according to the value of work completed relative to the total cost of producing that work. (OpenAI)

Although this is OpenAI’s commercial framework—not an independent industry standard—it reflects a broader evolution in enterprise AI strategy. Businesses are beginning to ask a more important question:

Is our AI creating measurable business value, or are we simply measuring AI activity?

That distinction may define the next generation of AI leaders.

Why Traditional AI Metrics Fail

Many organizations proudly report statistics such as:

• 10,000 AI licenses deployed
• 2 million prompts submitted
• 85% employee adoption
• Millions of API tokens consumed

While these numbers demonstrate usage, they reveal very little about actual business performance.

Imagine two organizations.

Company A

• 20 million AI prompts
• 12,000 users
• Low token costs

But employees still spend hours correcting outputs, rewriting documents, validating analyses, and repeating prompts.

Company B

• Half as many prompts
• More expensive models
• Higher cost per token

Yet AI completes customer requests correctly the first time, reduces manual work, accelerates decision-making, and improves customer satisfaction.

Which organization generated more value?

Clearly the second.

OpenAI’s framework argues that cost per token is becoming as misleading as measuring manufacturing performance by electricity consumption rather than products produced. (OpenAI)

OpenAI’s Four Questions Every Executive Should Ask

OpenAI proposes that enterprise AI should be evaluated using four fundamental questions.

1. Is AI Completing Meaningful Work?

The first metric shifts attention from AI usage to business outcomes.

Instead of asking:

“How many prompts were submitted?”

Organizations should ask:

• How many customer cases were resolved?
• How many contracts were reviewed?
• How many support tickets were closed?
• How many reports were generated?
• How many decisions were improved?

AI only creates value when work moves forward.

This represents a fundamental change from measuring activity to measuring productivity.

2. What Does a Successful Task Actually Cost?

Many organizations compare models purely on API pricing.

However, cheaper AI is not always less expensive.

The true cost includes:

• API costs
• Employee review time
• Corrections
• Failed attempts
• Multiple prompt iterations
• Manual intervention
• Quality assurance
• Opportunity cost

A more capable model may appear expensive while actually reducing total workflow costs because it succeeds on the first attempt.

This is why OpenAI argues organizations should evaluate cost per successful outcome, not simply cost per token. (OpenAI)

3. Can People Depend on AI?

Accuracy alone is no longer sufficient.

Enterprise AI must also be dependable.

OpenAI recommends understanding whether outputs are:

• Ready to use immediately
• Require minor corrections
• Require significant rewriting
• Need human escalation

Dependability determines trust.

Trust determines adoption.

Adoption determines business value.

This becomes even more important as organizations deploy autonomous AI agents capable of making decisions and executing workflows with minimal supervision.

4. Does AI Deliver More Value as It Scales?

Traditional software often becomes more expensive as organizations expand usage.

OpenAI suggests successful AI should become more valuable as adoption grows.

Organizations should observe whether expanding AI leads to:

• Lower cost per completed task
• Faster cycle times
• Improved decision quality
• Reduced operational overhead
• Greater productivity
• Better customer experiences

If costs increase faster than value, the AI strategy requires reassessment.

Why This Matters for Enterprise AI

OpenAI’s proposal signals an important shift.

The market is moving away from asking:

“Which AI model is best?”

toward asking:

“Which AI system creates the greatest business value?”

This changes executive conversations.

Boards are unlikely to approve multimillion-dollar AI investments based on prompt counts.

They will ask:

• What business problem did AI solve?
• What financial impact did it generate?
• How much risk did it reduce?
• How much productivity improved?
• How quickly was value realised?

These are governance questions as much as technology questions.

AI Governance Must Include Measurement

Many AI governance programmes concentrate on:

• Privacy
• Security
• Bias
• Transparency
• Regulatory compliance
• Human oversight

These remain essential.

However, governance also requires measuring whether AI is achieving its intended purpose.

Without measurement, organizations cannot answer fundamental questions:

• Which AI initiatives deserve further investment?
• Which workflows should remain human-led?
• Which AI systems require redesign?
• Which models deliver the greatest long-term value?

Measurement therefore becomes a governance capability rather than merely a finance function.

Beyond OpenAI: Building an AI Value & Dependability Scorecard

While OpenAI introduces four high-level questions, enterprises require operational metrics that can be tracked continuously.

A practical scorecard should include:

Together these provide executives with a far richer understanding of AI performance than adoption statistics alone.

Extending the Framework for Agentic AI

As organizations begin deploying autonomous AI agents, measurement must expand further.

Future enterprise scorecards should also assess:

• Decision accuracy
• Autonomous completion rate
• Escalation effectiveness
• Policy compliance
• Audit completeness
• Risk incidents
• Data quality
• Cross-agent collaboration
• Human intervention frequency
• Return on autonomous execution

These metrics help determine whether AI agents are operating safely, responsibly and effectively.

A New Enterprise Framework

Organizations often approach AI through three disconnected initiatives:

1. AI adoption
2. AI governance
3. AI measurement

In practice, these should form one continuous operating model:

Adoption → Governance → Measurement

Adoption

Identify valuable workflows where AI can improve productivity and decision-making.

Governance

Define authority, access controls, human oversight, compliance obligations and accountability.

Measurement

Evaluate business outcomes continuously and refine deployments using objective evidence.

This creates a feedback loop where AI systems become progressively more valuable, more dependable and more aligned with organizational goals.

The Future of Enterprise AI Will Be Measured in Outcomes

OpenAI’s “Useful Intelligence per Dollar” framework represents more than a new KPI.

It marks a transition in enterprise thinking.

Organizations are beginning to recognize that successful AI programmes are not defined by how many people use AI, how many prompts are submitted or how many tokens are consumed.

They are defined by the quality of work completed, the confidence people place in AI-generated outcomes and the measurable business value those outcomes create.

The enterprises that succeed over the next decade will not necessarily deploy the largest number of AI tools.

They will be the organizations that consistently answer four questions:

• Did AI complete meaningful work?
• What did that successful outcome actually cost?
• Can people depend on the result?
• Does AI create increasing value as it scales?

Those questions transform AI from an experimental technology into a measurable business capability—and that is where the next phase of enterprise AI begins.