The great workload reshuffle: Choices for AI and analytics

The modern architecture spectrum

Modern infrastructure options focus on three primary designs: public cloud, on-premises and hybrid cloud. Each includes its own benefits and trade-offs for specific workloads.

Public cloud:

• Strengths: elasticity, rapid provisioning and global reach.
• Trade-offs: cloud egress costs, long-term run-rate economics and vendor dependence.
• Best-fit workloads: bursty compute, experimentation and distributed access.

On-premises:

• Strengths: predictable costs, performance control and data sovereignty/security.
• Trade-offs: Capex, scaling constraints and lifecycle management.
• Best-fit workloads: steady-state compute, sensitive data and high-throughput processing.

Hybrid cloud:

• Strengths: workload portability, phased modernization, support for data gravity and latency constraints.
• Tradeoffs: operational complexity, data integration overhead, IT skill requirements and governance demands.
• Best-fit workloads: latency-sensitive pipelines, AI training, regulated or residency-bound data, modernized legacy systems and data-intensive applications.

The above strengths firmly establish hybrid as the default approach, along with its flexible deployment options and risk distribution benefits.

Modeling the real cost of each environment

Measuring and evaluating the costs for each environment means understanding the workload requirements and the infrastructure capabilities. Establish specific measures to provide actionable data using the following four categories.

Performance economics: Measure throughput and latency modeling using the following metrics:

• Translate SLAs into infrastructure requirements (GPU hours, IOPS, network bandwidth, etc.).
• Quantify the cost of delay (missed revenue, degraded UX) vs. the cost of capacity.
• Identify when proximity to data or users materially reduces compute spend.

Data movement economics: Track data management costs using these measures:

• Measure total data lifecycle cost: Ingest > process > store > transfer > destroy
• Compare compute-to-data vs. data-to-compute movement strategies, especially when considering distributed or edge micro-data centers.
• Model recurring transfer and replication costs over time.

Workload behavior profiles: Recognize workload behavior, labeling it in ways useful for tracking, including:

• Classify demand patterns: Steady-state, bursty, seasonal, experimental.
• Separate training vs. inference, batch vs. real time, transactional vs. analytical.
• Prioritize placement where utilization and scaling patterns maximize efficiency.

Calculating AI investment ROI: Explore real AI-related costs, such as:

• Evaluate cost per model run and per insight delivered.
• Factor utilization rates, idle capacity and refresh cycles.
• Include operational overhead, staffing and time-to-deploy impacts.

Risk, compliance and strategic control

Cost is not the only factor driving environment and workload optimization. Governance and resilience are crucial for ensuring compliance, control and visibility.

Compliance and data residency

Increasing regulation around data sovereignty, residency and security continues to drive architecture decisions. The distributed nature of the cloud was once considered a benefit, but it now subjects data to privacy requirements and regulatory compliance standards that organizations must satisfy.

Establish a firm understanding of where compliance and data residency requirements mandate local control, as these factors may force the organization to retain on-premises data management. Failing to meet these requirements subjects the organization to regulatory and legal action, with the potential for reputational damage.

Vendor lock-in risk

Vendor lock-in presents its own set of challenges, some economic and others technical. In either case, they can impact negotiation leverage and business agility. One way to mitigate potential vendor lock-in is workload portability, which enables organizations to redeploy the same workload in another environment without major code changes, refactoring or re-architecting.

Mitigating this risk means establishing flexibility and avoiding tight couplings to a single vendor's services, APIs or infrastructure.

Operational risk posture

Infrastructure decisions impact business continuity, especially in regions facing geopolitical instability or high risk of natural disasters. Organizations must also manage the complexity of security controls across disparate environments. Hybrid deployments may provide a risk-balancing mechanism.

These factors link the organization's risk posture to board-level accountability, ensuring governance keeps risks within acceptable strategic levels.

Decision framework: Matching workloads to environments

The following executive blueprint establishes a workload placement model and provides mapping examples. It includes a sample quick-decision matrix and governance structure.

Four-dimensional placement model

Begin by evaluating workloads using the following criteria:

1. Performance sensitivity (latency and throughput).
2. Data characteristics (volume, movement frequency and sensitivity).
3. Economic profiles (utilization stability and scaling pattern).
4. Risk and compliance exposure.

Workload evaluations should result in recommendations for the most suitable environment. Here are three likely examples:

• AI training infrastructure: Hybrid or on-premises for sustained GPU utilization.
• Streamlining analytics architecture: Hybrid (or edge) for latency-sensitive pipelines.
• Relational databases: Environment determined by transaction predictability and data residency rules.

Fast-path mapping of common workloads

The following generic recommendations may be useful to organizations just getting started.

Rework the workload evaluations for specific situations.

When faced with decisions that don't leave time for detailed scoring, assume the following:

• If data rarely moves but compute demand is constant > favor on-premises.
• If demand is unpredictable and experimentation matters > favor cloud.
• If performance is critical but scale varies > favor hybrid.
• If compliance drives architecture > start on-premises, and then extend outward.

Governance for ongoing optimization

Approach workload environment placement as a portfolio optimization problem and plan for long-term maintenance. Include continuous assessment of essential technical and financial KPIs.

• Rescore the top 10 workloads quarterly.
• Track cost per workload vs. business value delivered.
• Flag workloads where the environment score changes by 20% or more.
• Tie placement reviews to budget planning cycles.

Workload placement is not a one-time decision or an item to be crossed off a to-do list. It is an ongoing process of measurement and optimization.

Strategic takeaways for technology leaders

Framing infrastructure optimization as a workload-to-environment alignment activity reframes its outcomes to include:

• Margin protection.
• Time-to-insight.
• Innovation velocity.
• Risk exposure.

No universal best-environment exists for every workflow -- only best-fit placement. Risk posture and workload portability drive long-term flexibility, so recognize that cost modeling must account for performance and data-movement overhead. Organizations that continuously optimize where work runs -- not just how it runs -- will capture the next wave of AI and analytics with greater control and resilience.

Damon Garn owns Cogspinner Coaction and provides freelance IT writing and editing services. He has written multiple CompTIA study guides, including the Linux+, Cloud Essentials+ and Server+ guides and contributes extensively to Informa TechTarget, The New Stack and CompTIA Blogs.