IT leaders face data infrastructure gaps as AI workloads grow
While AI is no doubt vital to IT strategy, Enterprise Strategy Group research shows that implementation challenges persist when it comes to data and storage infrastructure.
We provide market insights, research and advisory, and technical validations for tech buyers.
Published: 17 Sep 2025
IT leaders overall remain enthusiastic about the potential for AI technologies to drive strategic transformation at their organizations, but many are floundering on the rocks of implementation. This is in large part due to a range of data-related infrastructure challenges, as revealed by the key findings of a new study by Enterprise Strategy Group, part of Omdia.
As I noted recently, there's no successful AI strategy without a successful AI data strategy. And, as enterprises continue to ramp up their AI investments, they are increasingly turning their attention to the infrastructure that underpins their AI data environments, particularly the storage infrastructure.
Curious to understand these challenges in greater detail, we conducted a research study to delve into IT professionals' objectives, priorities and pain points. The survey, which focused on organizations with AI projects already in process, validated the range and extent of the challenges IT infrastructure decision-makers face as they begin to scale their efforts and embrace new AI approaches.
Enterprise AI strategies highlight need for infrastructure investments
First, the positive news. Overall, it's clear that AI is increasingly viewed as a strategic priority, with 84% of respondents agreeing that AI is critical to their organization's future. As this importance grows, there's also a growing realization that organizations will require infrastructure upgrades to support the often-substantial performance demands of running AI workloads at scale. Almost nine in 10 (89%) survey respondents said they are making significant infrastructure investments to support AI initiatives.
However, the research also highlighted that data- and storage-related issues are increasingly part of the challenge: 70% of organizations surveyed said storage challenges are a significant barrier to AI success, while 83% said they plan to upgrade their storage in the next two years to support AI initiatives.
What kinds of storage problems are organizations facing specifically? The research identified a range of issues, with the type and extent of the challenge varying depending on which AI lifecycle stage organizations are focused on. For example, at the data prep stage, the biggest issue is managing data and capacity. This is perhaps no surprise when you consider that 87% of respondents said they are finding that AI is already driving substantial data growth at their organizations.
Amid this quickly rising tide of data, organizations are struggling to identify, locate and prepare the data they require for their AI pipelines. This is a challenge often compounded by data fragmentation across multiple storage silos and in a range of on- and off-premises locations.
Indeed, while just over a third of respondents -- 34% -- said their on-premises data centers will be their primary infrastructure location for their organization's AI initiatives, a similar proportion -- 30% -- said hyperscale public cloud providers will be the primary location, with the remainder split between specialist GPU service providers (i.e., neoclouds), edge locations and colocation providers. With around 40% of respondents using both on-premises and cloud locations for their AI workloads, it's clear that many organizations will be looking for platforms that can effectively manage their AI environment across hybrid locations.
In the model training phase of AI, performance becomes the key storage challenge -- specifically, delivering enough throughput. GPUs are hungry for data, and running AI training processes such as checkpointing can put extraordinary pressure on the storage environment while also keeping GPUs saturated. Any delays here translate into lost GPU cycles, a costly waste of an extremely expensive resource.
Moving to the inference stage -- the "value stage" of AI where many are focusing most of their effort -- the emphasis is once again on storage performance, though this time with latency issues predominating. As organizations come to appreciate the power of advanced inferencing tools, such as retrieval-augmented generation, agentic AI and other reasoning-based approaches, they are asking increasingly sophisticated questions. Accordingly, context windows -- and token counts -- are exploding, and so are the I/O demands to the underlying storage.
With 79% of respondents noting that AI presents significant data security challenges to the organization, ensuring the security of the underlying storage environment has never been more important.
A further critical factor identified in the research is the importance of data privacy and security within AI overall. With 79% of respondents noting that AI presents significant data security challenges to the organization, ensuring the security of the underlying storage environment has never been more important. Indeed, IT and business leaders will simply not move forward with AI at scale if they do not have complete faith in their ability to secure key data and keep it private. Moreover, the research showed that almost half of organizations (44%) only have basic or even no measures to ensure privacy in AI storage systems.
As I recently noted, this is leading to a new wave of innovation and investment around data security and privacy within the storage and data infrastructure. While these innovations are generally applicable to the overall storage environment, there's a particular role for them to play within the AI context, which, if successfully executed, will provide IT leaders with the confidence and assurance that only the appropriate data is being used in their enterprise AI workloads.
Looking forward
While we should remind ourselves that we are still in the very earliest days of the AI era -- especially in the enterprise setting -- this new research underlines the criticality of the underlying data infrastructure to emerging AI efforts. It underscores that organizations should consider whether their current storage infrastructure strategy is fit for purpose in the AI era.
It also validates the growing ecosystem of storage and data infrastructure suppliers that are focused on delivering high levels of both performance and scale, increasingly in a way that has some manner of understanding -- or "intelligence" -- regarding the nature of the data being stored, either natively or through rich third-party integrations.
Indeed, as the power and potential of AI continue to evolve, it's clear that closer bonds will need to be forged between the storage infrastructure and the data management environment to build secure, effective and efficient AI data pipelines.
This presents somewhat of a challenge to the status quo at many organizations -- one that will necessitate, among other things, more collaboration among infrastructure-oriented IT professionals, data-oriented engineers and even data security, privacy and compliance teams. Still, the payoff for those who successfully navigate this transition could be meaningful.
Simon Robinson is principal analyst covering infrastructure at Enterprise Strategy Group, now part of Omdia.
Enterprise Strategy Group is part of Omdia. Its analysts have business relationships with technology vendors.
