Black Friday poses ultimate IT stress test for CIOs

Infrastructure readiness test

At the most basic level, Black Friday brings more traffic than other time periods, putting more pressure on infrastructure.

However, preparing infrastructure for Black Friday requires more than just adding servers and bandwidth capacity. Successful organizations should analyze past performance metrics to identify specific bottlenecks before they recur.

Finding the bottlenecks

The real infrastructure bottlenecks aren't where most IT teams expect them, according to Iliya Rybchin, principal at BDO USA.

"Yes, there are obvious things like server capacity, database performance and CDN capability, but those are table stakes," Rybchin said. "Any competent IT team handles those, the real bottlenecks emerge at the integration points."

Rybchin noted that when traffic spikes 10-20x normal, the handoffs between an organization's e-commerce platform, payment gateway, fraud detection systems and order management create cascading slowdowns.

Rafael Mercado, vice president and U.S. consumer and travel market leader at Kyndryl, identified four major infrastructure bottleneck areas that emerge during peak events:

• Scaling applications and platforms to meet demand. Outdated legacy systems and rigid e-commerce platforms tend to buckle under Black Friday pressure. CIOs can combat this with hybrid cloud strategies during traffic spikes to tap extra computer on demand, breakdown monolithic systems with microservices and APIs for faster and more flexible scaling and simulate peak traffic through synthetic load testing to find and fix weak spots before they happen.
• Network strains and lagging user experiences. When traffic surges across regions, slow networks and limited edge capacity lead to delays in page loads, checkout and personalized recommendations. Leading organizations are investing in edge computing to process data closer to users and optimizing content delivery networks for faster performance.
• Data bottlenecks across systems. Syncing multiple data sets across point-of-sale, ERP and logistics platforms can lag, causing errors and delays. Mercado noted that many retailers are adopting event-driven architectures that respond instantly to changes across systems rather than relying on batch processing.
• Slow incident response times. Fragmented monitoring tools and siloed teams slow down issue resolution when seconds count. Companies are deploying fully integrated observability platforms that provide end-to-end visibility.

The importance of load balancing

Yad Senapathy, founder and CEO of the Project Management Training Institute said that the biggest infrastructure issues are slow payment systems, overloaded databases and content networks that cannot keep up with the traffic.

Load balancing strategies separate successful retailers from those who struggle, he said.

"The companies that handle it best use flexible cloud setups that move traffic to backup data centers when needed," Senapathy said. "They process transaction data right away instead of waiting for batch updates which keeps memory use stable."

Observability and AIOps

Observability and AIOps have become essential for real-time infrastructure monitoring. Senapathy recommended that CIOs keep tracing tools running across every service so they can see where things slow down.

AIOps tools can provide significant value to organizations, he said. In some cases organizations that use them have cut false alerts by more than half, giving engineers more time to work on actual issues.

Cybersecurity and fraud prevention

The increased traffic that comes with Black Friday also brings with it potentially more risk as transaction volumes spike.

Sophisticated attackers can potentially use the massive surge in legitimate traffic as cover for malicious activities. Automated and sophisticated fraud dominates the cybersecurity landscape, from bot-driven inventory hoarding to loyalty point theft and return fraud, Rybchin said.

The primary threats break down into three categories:

• Distributed Denial-of-Service (DdoS) attacks. Some attackers extort retailers by threatening site takedowns during peak hours. Others use DDoS as cover for data theft or payment system compromises. The challenge is distinguishing malicious traffic from legitimate customer activity.
• Phishing campaigns. Email attacks spike in the weeks before Black Friday, targeting both customers (credential theft) and employees (system access). Fraudsters compromise accounts months in advance, then wait until Black Friday to make high-value purchases that blend in with legitimate shopping.
• Automated fraud. Bot networks execute account takeovers, payment fraud and inventory manipulation at scale. "Attackers are now acting earlier, automating faster and blurring the line between bots and legitimate human traffic, often powered by GenAI," Mercado said.

AI and automation

The emergence of AI and advanced automation tools has changed how CIOs can deal with Black Friday challenges.

"AI is moving from experimentation to execution," Mercado said. "Retailers are using it to detect fraud in milliseconds, predict demand shifts, personalize offers dynamically and even pre-empt infrastructure strain."

The primary use cases for AI and automation include:

• Customer support. Customer support triage benefits from AI-driven automation that routes issues based on urgency and complexity. During Black Friday, automated systems handle routine inquiries while flagging complex issues for human intervention, keeping response times low.
• Incident response. Senapathy noted that AI has made a real difference in predicting and preventing problems before they grow. It helps forecast how much load the servers will handle every hour so that teams can scale ahead of time.
• Anomaly detection. Organizations can use AI-driven behavioral analytics to flag deviations from known user journeys, enabling rapid detection of both technical issues and fraudulent activity. Agentic AI autonomously monitor store-level patterns and trigger automated remediation, Mercado said.

Real-time analytics

Real-time analytics are a foundational element of IT operations providing CIOs and IT leaders with clear insight into what is occurring in an environment.

The analytics are commonly presented on dashboards which helps to enable quick status updates as events occur. These dashboards provide visibility across technical performance and business metrics simultaneously, allowing teams to understand not just what's happening, but why it matters.

Leading organizations deploy unified dashboards that track critical technical key performance indicators (KPIs) including:

• Server response times.
• Database query performance.
• Payment processing latency.
• API call volumes.

These same dashboards display business metrics including:

• Conversion rates.
• Average order values.
• Cart abandonment rates.
• Promotional performance.

The analytics infrastructure supporting these capabilities requires careful architecture. Event streaming platforms capture customer interactions, system performance metrics and business transactions as they occur. Stream processing engines analyze this data in flight, identifying patterns and triggering automated responses or alerts. Data lakes and warehouses provide the historical context needed to interpret current patterns and predict future trends.

Compliance and governance are essential elements of the real-time analytics infrastructure. Retailers must ensure that real-time personalization and behavioral tracking comply with General Data Protection Regulation (GDPR), California Consumer Privacy Act (CCPA) and other privacy frameworks. Leading organizations build these controls into their analytics architecture, ensuring rapid decision-making doesn't create compliance exposure.

Collaboration across the enterprise

Aside from the various technical challenges that Black Friday brings to CIOs, it also brings people challenges that require collaboration across the enterprise.

According to Nora Jones, founder of Jeli.io, which was acquired by PagerDuty, most Black Friday “failures” are human, not technical.

"Years ago, I experienced a significant outage during a high-traffic campaign simply because the marketing and engineering teams weren’t aligned," Jones said. "Marketing launched a massive promotion without warning us, and the sudden spike crashed our systems within minutes."

Since then, Jones has advocated for a “pre-mortem” approach to planning. Rather than only reviewing what went wrong after the fact, teams should meet ahead of time to predict possible points of failure and design responses together.

"The proactive coordination between marketing and engineering, ensuring both sides know what’s coming and how to respond, is what truly separates the retailers that thrive on Black Friday from those that scramble," she said.

The CIO's role during Black Friday extends beyond IT operations to bridge technology, e-commerce, marketing and supply chain functions.

The best CIOs are not only technologists, but are also orchestrators, Mercado said, as they align IT, marketing and operations around shared priorities and create real-time feedback loops that enable teams to pivot instantly.

"They also communicate clearly what’s mission-critical, what can wait and who owns each decision to reduce confusion and accelerate response times during peak events," he said.

Key attributes of successful CIOs for Black Friday include:

• Trust. Building trust and alignment across departments they don’t directly control.
• Influence.The ability to influence teams across departments through shared goals, transparency and consistent communication across departments.
• Empowerment. Instead of centralizing every decision, they empower frontline teams with clear frameworks and real-time data access to allow faster response times without the need for top-down approvals.

"The CIOs who struggle often work in silos or treat peak events like Black Friday as one-off firefights rather than as part of an ongoing cycle of readiness balancing modernization, collaboration and the human agility needed to move at the speed of the customer," Mercado said.

Sean Michael Kerner is an IT consultant, technology enthusiast and tinkerer. He has pulled Token Ring, configured NetWare and been known to compile his own Linux kernel. He consults with industry and media organizations on technology issues.