AI puts more gray behind CIOs' yes-or-no decisions
CIOs still make yes-or-no decisions, but AI is making everything behind those choices more complicated -- from change and cost to architecture and productivity.
AI is making the gray areas behind CIO decisions harder to ignore, especially when the choice seems as simple as yes or no.
Approve the change or don't. Fund the modernization or don't. Call the AI use case productive or don't.
Up close, those answers depend on a much wider range of context, consequence and impact. A change can be trivial at one end and consequential at the other.
A workload can warrant an expensive frontier model in one context and a cheaper one in another. Existing architecture can be perfectly adequate until what the business asks of it changes. A productivity gain that looks real at one point can disappear further downstream.
Gray areas have always been part of enterprise technology decisions, but AI is making them harder to ignore and, in some cases, harder to see clearly.
AI change now spans a spectrum of cause, effect and impact. The possible enterprise responses span one too.
That puts a premium on visibility and measurement. CIOs need to understand when circumstances have changed enough -- in cost, risk, architecture, business importance or downstream effects -- to warrant a different response.
AI change itself is becoming a spectrum
AI systems can alter production behavior even when little or no conventional application code changes. Some of those changes are driven by AI agents themselves. Others come from people swapping models, altering workflows, changing tools or modifying routing, permissions or prompts.
When it comes to oversight, not every change deserves the same level of scrutiny or response.
A prompt adjustment affecting an internal summary is different from an equally small change affecting whether a customer receives a refund. The technical size of the change could be almost identical. The possible business consequence is not.
That leaves IT leaders with two related questions: Did something change in the first place? And did it change enough to matter differently?
Business consequences help determine the response. Even a seemingly minor change can become consequential when repeated, combined with other changes or allowed to persist over time. The enterprise response can be graduated, too.
Stacklet's Token Custodian announcement shows what that can look like in cost control. The early-preview product is designed around the idea that AI cost control does not have to mean simply allowing it or shutting it off. A workload can trigger a warning, require an exception, be routed to a cheaper model or be stopped. An exception can also be temporary.
A high-value customer-facing workload might justify a more expensive frontier model. A routine internal task might not. But blindly pushing work onto cheaper models can create its own problems when quality drops or the business outcome suffers.
So, the decision is more complicated than asking how much AI a particular piece of work is worth. Quality, risk, customer importance and business value all enter into it.
Architecture has its own gray areas
TechTarget reporting on AI scaling and legacy architecture found that 74% of organizations with implemented AI reported measurable top- or bottom-line impact, but only 13% had scaled their initiatives completely in line with the original business case. Legacy-system integration was among the barriers.
CIOs have faced this decision for years. APIs, wrappers and abstraction layers can extend the useful life of an older system without forcing an expensive modernization project.
But at what point do those architectural workarounds become enough of a contortion that the answer is: It is time to modernize?
The architecture did not suddenly become bad. What the enterprise was asking of it changed.
There is no single point.
Carnegie Mellon University's Ipek Ozkaya told TechTarget there is "no single threshold" at which an older system must be modernized rather than accessed through APIs, wrappers or abstraction.
A system that works perfectly well for occasional reads might become inadequate when AI requires high-volume transactions or fast write-back. The architecture did not suddenly become bad. What the enterprise was asking of it changed.
Latency, volume, security risk, maintenance burden, write-back requirements and business criticality can all change the answer.
There is no single digital threshold -- a zero or one -- that suddenly declares the architecture obsolete. Much of the decision lives in the analog space between obviously adequate and obviously inadequate.
Different ranges, similar decisions
AI does not create one universal threshold for CIOs to manage. Different decisions unfold across different ranges -- from how consequential a change is to whether architecture still holds up or a productivity gain survives downstream. The details vary, but the management problem is similar: CIOs need enough visibility and context to know when the enterprise should respond differently.
• Completion: task -> workflow -> business outcome
• Productivity: local gain -> downstream burden -> enterprise result
Each contains real decision points, some of them binary. The challenge is having enough visibility into the surrounding context to know when the enterprise should respond differently.
You ask AI to do something and it does it: task complete.
It contributes to the completion of a business process: workflow complete.
Something the business cares about at an organizational level is achieved: business-outcome completion.
All of them count. They are not interchangeable, but there can be several legitimate finish lines along the way. The CIO has to know which one matters for the decision being made.
Context matters and measuring model execution may make task completion enough. A claim about business ROI needs a much farther stop along the rail.
A local gain is not always an enterprise gain
David Linthicum takes the same problem downstream in a recent TechTarget CIO News column. AI can demonstrate a real productivity improvement at one point and still have that benefit disappear later.
A developer can produce code faster. A claims analyst can create more summaries. A salesperson can generate more outreach. Those gains can be real.
But AI measurement should not stop at the moment of execution.
Review time, rework, corrections, exceptions, escalations and queue time also matter. A productivity gain can be perfectly real at one point in the workflow and simultaneously disappear -- or even reverse -- farther downstream.
AI can make work faster at one measurement point while the end result is an enterprise that moves more slowly.
Eventually somebody still has to give a yes-or-no answer: Did this AI use case improve productivity or didn't it? That apparently simple answer rests on everything that happened before and after the AI did its part.
Flexibility matters
None of this means CIOs should constantly change standards to suit the moment.
But a certain level of flexibility is required because latency, volume, risk, cost, business importance and downstream burden can change enough that yesterday's reasonable response isn't today's.
CIOs can apply everything they already know about software, architecture, risk and operations. AI, however, is now embedded in standalone models, individual applications, large software suites, agents and workflows that connect systems in ways enterprises have not dealt with before.
That makes it harder to draw every line in advance.
IT can provide visibility into systems, costs, architecture, behavior and downstream effects. But IT cannot independently decide how much customer importance, business value, risk or outcome is enough to warrant a different response.
Those are enterprise decisions that CIOs must make with business leaders.
AI isn't eliminating yes-or-no decisions. It is making CIOs understand more of what sits behind them before they can responsibly choose yes or no.
That takes visibility, measurement, flexibility and business partnership -- not another fixed set of rules.
James Alan Miller is a veteran technology editor and writer and Lead Editor for CIO News at Informa TechTarget. He directs coverage of enterprise technology strategy, AI, software, data, infrastructure and the decisions shaping how CIOs manage increasingly complex IT environments.
