The benefits and pitfalls of cloud-based data management systems
Learn the benefits of cloud-based data management systems, common pitfalls and strategies when considering varying data levels and industry needs.
As enterprises move data storage, compute and applications to the cloud, cloud-based data management systems are increasingly important. Cloud service providers offer a wealth of tools on tap, but it's wise to understand the associated cloud costs before making expensive mistakes.
Most of today's enterprises have a hybrid cloud architecture, so it's important to have an appropriate enterprise data management strategy that transcends the location of data.
"The hybrid landscape will definitely be important, so we'll see vendors pivot in and capitalize there," said Daniel Elman, research manager at Nucleus Research. "IBM is the main one …, but now we're seeing AWS and Azure investing in hybrid capabilities as well. I think you'll see the data management players who are partnering with those [cloud vendors] take advantage of those capabilities."
The benefits of cloud-based data management
Cloud-based data management provides a variety of benefits:
- data governance controls
- value-added security controls
- an audit trail
- faster query performance
- built-in product upgrades
- compliance (e.g., GDPR, CCPA and the Biometric Information Privacy Act)
- automated backup and disaster recovery
- lower costs, assuming cloud costs have been optimized
Of course, tried and true data center practices, such as standardizing naming conventions, are still required, but they may not get the attention they deserve before data is migrated to the cloud.
One mistake that organizations make when migrating data to the cloud is executing a lift and shift operation, which essentially duplicates what's been done in the data center.
"We've seen client instances where the estimate of cloud consumption cost versus the actual cost is off by a factor of 10 or higher," said Goutham Belliappa, vice president of AI engineering at global professional services company Capgemini.
For example, one Capgemini client estimated a $250,000 monthly cloud spend, but the invoices were $2.5 million per month. Ironically, the client wanted to avoid renewing a contract with an on-premises data management vendor whose appliance cost about $5 million. The cost of the appliance alone would have been amortized over four years as a capital expense versus spending the same amount on two months of cloud service.
Goutham BelliappaVice president of AI engineering, Capgemini
Another potential pitfall is remaining tied to a traditional data management vendor whose progress is sluggish compared to the major cloud providers and open source projects.
"The mistake many of our clients make is trying to come up with a solution that works well in their on-prem environment that is also able to support their cloud-native environment. They end up with the worst of both worlds because they're optimizing for the least common denominator," Belliappa said.
Meanwhile, enterprises are struggling to master data governance because their data ecosystem is evolving so rapidly. Rather than attempting to accomplish too much too soon, Belliappa recommended that companies identify the four or five most desirable data governance outcomes and then create a data governance structure that supports the KPIs.
Different plans of attack
Organizations are at varying stages of data management maturity. Technology consulting firm Saggezza uses five levels:
- on-premises data management (data is siloed);
- data cataloguing;
- dashboarding and reporting;
- optimization (e.g., route optimization for a logistics company); and
- machine learning and AI.
More mature organizations tend to have a chief data officer as opposed to relying on the COO, CFO, CIO or CTO, particularly if they're a large organization. If they're a small or midsize company, they may not be able to afford the luxury.
"If they don't know what business data they have, how to grant access and govern it, we recommend they [keep] it in their data centers so they can have more control. Then, we help them mature so they can move to the cloud," said Thanneermalai Krishnappan, senior technical program manager at Saggezza. "Cloud data management provides a lot of benefits, but if you don't plan it properly, your cost is going to come back and bite you."
Consider the industry
Law firms have special requirements for data usage and security, but their infrastructures differ. Though some are concerned about cloud security, others believe the cloud is actually more secure.
"We did this because it saves our clients money, it's more secure and it's more flexible. Our partners can work from anywhere," said Heather Clauson Haughian, founder and managing partner of law firm Culhane Meadows, which has been completely cloud-based for the past eight years.
"Two or three years ago, we changed our document management system from one cloud to another, so one of the things I tell my clients is not only is it going to cost, but you also have to really know where your data is," said Haughian, who is also an International Association of Privacy Professionals Certified Information Privacy Professional.
She also underscored the importance of stakeholder support from the outset.
"If you invest in resources from the outset, making sure you educate people as to what exactly you're doing and how you're doing it, you'll get better buy in, which means you'll have a shorter migration period," Haughian said. "[Otherwise,] you'll have people who are resistant, don't know what the process is and are protective of their data because they don't think it belongs in the cloud."
The challenges of cloud data management