Migrating AI to the cloud demands strategic planning, system assessments, component modernization, data pipeline optimization and phased deployment. Find out what you need to know.
Local AI deployments have practical limitations. The AI computing infrastructure and the corresponding energy demands can be costly to build, manage and maintain, often restricting the resilience and scalability that modern AI platforms depend on. Companies increasingly turn to the public cloud to host AI components, such as LLMs, and even entire AI platforms.
Yet despite the appeal of public clouds, transitioning an AI workload from local to cloud deployments can be challenging without a well-planned strategy. Direct lift-and-shift migrations rarely yield optimal outcomes and can diminish AI performance. The most effective AI migrations to public cloud require careful assessment, a clear cloud modernization plan, a well-structured data pipeline, cautious phased deployment, and continuous management and governance.
Basic AI cloud migration strategy
There's no one path to take a local AI deployment to a public cloud. Numerous approaches can streamline the transition and ensure objective evaluations of the resulting cloud performance. The key factors are comprehensive assessments, data considerations and workload optimizations that best use cloud resources and services.
The following steps can simplify the migration process.
Review the local AI system
Examine and map the AI system as it's deployed across the local infrastructure. Document the presence; purpose; requirements; dependencies, such as APIs and databases; and operational performance metrics of every model, agent and data or workload pipeline associated with the AI components systems involved in the cloud migration. Documentation ensures vital components aren't overlooked and each is optimally resourced and configured for migration.
Also, examine the data the models involved use. Data needs can restrict migration planning. For example, personally identifiable information and established intellectual property might require local storage and use, making cloud migration inappropriate for some industries. Conversely, AI components that use publicly available data might alleviate such restrictions.
Modernize components for cloud optimization
Simple lift-and-shift approaches work fine for traditional enterprise workloads. However, AI components and systems demand highly scalable high-performance infrastructures to be effective, making simple rehosting strategies inadequate for AI migrations.
Data can be a significant gating issue for AI cloud migrations.
The most effective AI can embrace cloud-native or cloud-first software development strategies, which typically require AI workloads modernized using one of three directions:
Refactor the component.Refactoring involves rewriting code to improve its design, mitigate technical debt by adding delayed features and capabilities, and enhance maintainability without changing the code's behavior.
Rearchitect the component. Rearchitecting updates the fundamental design of a workload, such as redesigning a monolithic application as a containerized microservices application. This approach might add new business requirements, support greater scalability or embrace cloud-native features, such as orchestration.
Replatform the component. In this context, replatforming involves replacing traditional components and services with cloud-native ones. For example, an AI migration might replace a traditional local database with a specialized cloud-native managed vector database to improve an AI system's retrieval-augmented generation performance.
Consider data migrations and pipelines
Data can be a significant gating issue for AI cloud migrations. Typical enterprise workloads migrated to the cloud could potentially access data from local data stores, but AI demands enormous volumes of data for training and instancing -- far too much data to make continuous network access practical. Consequently, AI data stores and data processing pipelines also must be migrated to the cloud.
Data can be transferred to cloud storage using high-volume applications, such as AWS DataSync and Microsoft Azure Data Box. Consider the costs involved with these services, as well as the corresponding storage costs once data is loaded into cloud storage.
Appropriate security features must be built into AI data storage, including zero trust, access control and encryption. Also consider AI data retention policies and implementation. AI data might have a shorter shelf life than traditional business data. For example, once an AI model is trained and validated, the vast data store used for training must be extensively updated and refined before the AI is retrained. This might require deleting existing cloud data stores and replacing them with fresh ones.
Finally, use mapping from the initial review to rearchitect and rebuild the data processing pipelines in the cloud. Local extract, transform and load and other data processing tools might need to be deployed to the cloud to recreate the complete pipeline or replatform some tools using cloud-native offerings.
Use a phased approach to migration
AI is quickly becoming a mission-critical business platform requiring significant investment. And it's too important to simply cut over and run AI from the cloud without testing and validating the deployment. This typically involves a period of learning and evaluation to compare local and cloud behaviors.
Cautious organizations avoid migrating the entire AI system from the start. Instead, they migrate individual models.
For example, AI systems aren't single applications but rather collections of models and supporting software components. Cautious organizations avoid migrating the entire AI system from the start. Instead, they migrate individual models, associated data and other required components in a proof-of-concept phase. The migrated model continues to work with the rest of the local AI system. It's tested to ensure the same inputs produce the same outputs as the local model. Validation is vital for explainability and AI compliance efforts. Once a model is successfully migrated, validated and operational, additional AI platform components can be migrated.
When the entire AI platform is migrated and each element is validated, the platform should be tested in its entirety. Production traffic is routed to the local and cloud AI platforms in parallel. Testing compares outputs, latency, throughput and error metrics. At this phase of migration, resources are optimized to achieve comparable AI results. The final cutover to the AI cloud platform is then planned and implemented.
Conduct ongoing management and governance
AI platforms demand careful management and governance to ensure adequate performance, resilience and cost control. Use metrics to monitor key AI performance attributes, and use KPIs to track how the AI cloud implementation affects the business. For example, look for declining throughput or rising per-instance costs.
Treat cloud optimizations as an ongoing exercise.
Treat cloud optimizations as an ongoing exercise. Cloud resources and services can become expensive, especially when relying on automated scaling. Avoid overprovisioning expensive resources and select those that see high use and offer high cost-effectiveness. Established cloud practices, such as FinOps, can help with cloud optimization strategies and cost-saving measures.
It's important to use strong AI governance practices, such as cloud budget limits and alerts, resource tagging to clearly identify cloud resources associated with the migrated AI platform, and strong identity and access management policies as a fundamental approach to migration.
5 key AI migration questions for CIOs
CIOs are responsible for the ultimate success or failure of an AI cloud migration. Before starting a project, consider costs, security, models, compliance and overall preparedness by addressing the following five questions:
How will AI cloud providers use proprietary data? Many AI platforms integrate third-party AI services, such as LLMs, that typically use submitted data to train and refine their AI offerings. The problem is the third-party service might retain sensitive business data or intellectual property and wind up being shared with other users -- even competitors. It's critical to ensure that any third-party AI services can't access or use confidential data through policies and contractual agreements.
Does AI migration change compliance or data sovereignty? Workload and data sovereignty regulations limit where applications are run and data is stored. A business operating a local AI platform might migrate AI components or systems in ways that run afoul of these regulations and result in noncompliance. AI migrations to the cloud must support existing regulations and maintain compliance with current and emerging regulations.
Are existing data storage and pipelines adequate for the cloud? Data storage and pipelines aren't uniform or ubiquitous. Many companies running local AI systems grapple with fragmented, siloed or poor-quality data storage and pipeline operation. For example, some businesses require manual intervention to transform or load data from disassociated data stores. Poor pipelines don't translate well to the cloud. An AI migration must consider data storage and pipeline efficiency and translate an efficient data scheme to the cloud where it can be readily automated and orchestrated.
What's the TCO for cloud AI infrastructure? Cloud computing creates recurring on-demand costs for the enterprise. Traditional applications present reasonably predictable costs. By comparison, AI workloads have operational spikes during training and inferencing. Use can be unpredictable, so the corresponding compute costs are also unpredictable and can result in cost overruns. Careful cost forecasting is vital to predict AI cloud costs. An organization's FinOps center of excellence might help identify and predict AI cloud costs.
Does the business have the skilled team necessary to run AI in the cloud? Team proficiency in running and governing a local AI system doesn't always translate to monitoring and supporting AI components or systems operating in the cloud. Public clouds exhibit distinct monitoring and operational behaviors. AI management teams must be able to deploy, monitor and maintain an AI platform in the cloud to validate ongoing performance, ensure proper AI governance and manage disruptions, including cloud provider outages. Additional team members might be needed to meet AI cloud deployment challenges.
Stephen J. Bigelow, senior technology editor at Informa TechTarget, has more than 30 years of technical writing experience in the PC and technology industry.
