Tomasz Zajda - Fotolia

How IT handles edge storage challenges from COVID-19

The COVID-19 'wake-up call' has stressed IT and storage teams, but those already using public clouds could adapt to new edge storage requirements posed by remote work.

This year has been a giant disaster recovery test for IT organizations, with real consequences.

A major goal of a DR plan is to ensure that a business can run if its facilities become unavailable for any reason. When the COVID-19 pandemic struck the U.S. hard in March, most businesses realized they could no longer use their offices, data centers and other physical sites. Those facility shutdowns forced millions of people to work at home and changed the way IT supported them.

For the most part, IT teams came through when dealing with data center and edge storage challenges. The ability to pivot to public clouds to ease reliance on data center infrastructure provided a huge helping hand. Organizations that already had a significant cloud footprint seemed to have an advantage in the new world of remote work.

"The pandemic has been a wake-up call to organizations," said Matt Eastwood, senior vice president of research and cloud at IDC, during a presentation at the recent Flash Memory Summit. "The pandemic forced organizations to accelerate their digital transformation out of necessity and with no warning at all."

Eastwood said "the technology headline of this pandemic is the role that technology has played" in a world where physical locations were closed. He summed it up as "a world defined by remote work and remote learning."

Eastwood said the 2020 pandemic will accelerate a move from legacy infrastructure toward the cloud. IDC forecasts that the cloud will account for two-thirds of IT spending by 2024, up from 50% in 2019. IDC also predicts that more than 50% of new infrastructure will be deployed at the edge, up from 10% today.

SearchDisasterRecovery spoke with IT teams that used their cloud storage to adopt to the sudden world of remote work, remote learning and infrastructure on the edge. The challenges they faced included making files always available for dispersed workers and providing security for hundreds of edge storage points.

Northeast Wisconsin Technical College

Northeast Wisconsin Technical College (NWTC) in Green Bay, Wis., moved its data protection to the public cloud in early 2020 to enhance its DR.

Kevin Swanson, NWTC's systems administrator, said the two-year public college chose Wasabi Technologies' public cloud because it did not charge egress fees to move data and worked well with his Commvault backup software. NWTC eliminated tape, and now uses two data centers and the Wasabi cloud for disaster recovery. Wasabi's immutable storage means the data NWTC puts there cannot be changed, protecting the school from ransomware attacks.

But soon after NWTC's trip to the public cloud, COVID-19 struck. Like other schools, NWTC switched to online learning and like most other businesses, employees began working from home. That prompted more data movement to the cloud.

"We are seeing a big portion of our data housed here on site being moved out to the cloud in some fashion," Swanson said. "We're seeing a push now from our employees that they actually want their files in the cloud now that they're working at home. They want them in Office 365 and Teams. If those files are on mapped drives, they have to be connected to our network via a VPN. They have to jump through additional hoops. It's more straightforward to go through Office 365 or Teams."

NWTC is familiar with traditional on-premises storage, using Dell EMC, HPE, NetApp, Pure Storage and other array and hyper-converged infrastructure vendors over the years. But NWTC systems engineer Brian Zimmerman said the college went to the cloud for another data protection tier.

The school replicates backups from its primary to secondary sites in Green Bay. It also ships a copy of all backup sets to tape at a remote site, but Swanson said it would recover from tape only as a "last, last, last resort."

"When we flip that switch [to the cloud], we lock that data in place so it can't be modified or deleted," Swanson said. "We have a recovery plan for every scenario, with a good set of data in each spot. In case of a disaster, we could easily get back on our feet where we have copies of our primary virtual machines in Wasabi. Those are the key things that if, if we were to lose them, it would make recovering from that disaster much more difficult if we didn't have a copy somewhere."

Swanson and Zimmerman said they would consider expanding their use of the public cloud for archiving, online learning and security.

"I see us always having some kind of physical footprints on prem, but I definitely can see us utilizing more of the cloud for ease of use," Zimmerman said. "If we get a monthlong project, we can quickly spin something up in the cloud, consume that compute and storage, knock off the development cycle and then get rid of it and spin it down quick. I can see that being a big use case."

Ithaca Energy

Malcolm Brown, IT operations manager for U.K. oil and gas company Ithaca Energy, said his firm turned to Nasuni cloud NAS in late 2018 during an infrastructure modernization project. He said the shift from on-premises to cloud storage proved valuable after Ithaca acquired Chevron Central North Sea in 2019, which more than tripled its file data volume. Moving to the cloud looked even better when the pandemic forced Ithaca's employees to work from home in early 2020. Nasuni, which stores Ithaca's file data in Microsoft Azure, also serves as a DR option.

Malcolm BrownMalcolm Brown

"If we were to lose our office in Aberdeen for any reason, we have the option of bringing it up as a native appliance in Azure," Brown said. "We can also make it available in a browser, and our people can access folders through the web. From a DR standpoint, it's very secure for unstructured data."

Brown said he experimented with Azure's file storage before deciding on Nasuni, "but the performance wasn't quite there, and it impacted cost because of egress charges with Azure."

Brown said Nasuni reduces the egress fees on Azure, although it does not eliminate them. "It's impossible to zero that out because if a user goes back for an older piece of data that's not in cache, Nasuni will put it in cache. That's an egress charge," he said.

"But Nasuni brings data back in a compressed format, so it reduces egress. All live files are in local cache, and we're not moving that data. Only deltas move into Azure."

Ithaca uses Nasuni across sites in the North Sea so its employees can share data in real time. Brown said moving to the cloud streamlined data protection, which previously consisted of disk-to-disk replication and monthly backups to tape. That involved multiple hardware devices and software applications.

"We would put it on a SAN or NAS device or even direct attached storage, and then provide that on a local network for engineers," he said. "They create files, and we back that up disk-to-disk-to-tape. It was a long process that could take weeks for big data sets, and I've had to go to tape for complete data sets and restore it to a golden set. With Nasuni, a gold master is always stored and untouched. People work on a snapshot, not the original data. We can always go back to the original if we need it."

Glenn Davis Group

COVID-19 struck months after the Glenn Davis Group design agency switched its main storage platform to Ctera's cloud-based NAS in January. Kyle Edsall, vice president of technology at the Canadian agency, said around 90% of the company's employees now work remotely. The initial reason for switching from on-premises storage was to keep hot data cached for performance while consolidating backups.

Kyle EdsallKyle Edsall

Glenn Davis uses the Ctera X-Series that runs Ctera cloud NAS software on HPE SimpliVity's hyper-converged infrastructure system. When file data is no longer considered hot, the agency moves it off to the Wasabi cloud.

This wasn't a decision that was made because of the pandemic, but man, was it ever timely that we had all that in place when we needed more agility from our storage platform.
Kyle EdsallVice president of technology, Glenn Davis Group

"This wasn't a decision that was made because of the pandemic, but man, was it ever timely that we had all that in place when we needed more agility from our storage platform," Edsall said. "It was advantageous to suddenly have a Web interface on top of every file in the organization. Otherwise, it would have been 100% everything SMB over VPN, which, depending on your bandwidth, just is not a good solution at all."

Edsall said Glenn Davis gets a performance boost because its users don't have to send files over the VPN. Its designers deal mainly with Adobe Illustrator and Photoshop files, but the Ctera system also stores administrative documents. The agency has also moved its VMware data to the Ctera HCI system.

"We keep everything that we're working on where it can be cached and it can be fast, but using the cloud is the best thing for anything that's transitioned to archive content," he said. "So, we do have a data center. We do have that infrastructure in place, but we've turned 48u of equipment into 4u [by incorporating the cloud for file storage]. We felt like the edge was absolutely the right place for us to be because we've got about 125 terabytes of data that we need access to, but a lot of it does not need to be locally housed."

Edsall said Glenn Davis previously stored data in AWS S3 buckets and ran a trial on Azure. But he said the agency picked Wasabi as its main storage because it was about one-sixth the price of the large clouds, performed better, and he didn't feel like he gave up any storage services.

Next Steps

Annual Flash Update -- The Pandemic's Impact

Dig Deeper on Disaster recovery facilities and operations

Data Backup