Organizations that want to enable faster insights and real-time decision-making can choose from several data architectures and the cloud in lieu of the traditional data warehouse. They must consider not only the new capabilities they provide, but how each approach fits their data management goals and needs.
The modern cloud data warehouse, lakehouse, fabric and mesh are all options for IT managers and architects who consider data strategy. These options add to a list that already includes the traditional data warehouse, mart and lake. Dr. Barry Devlin clarifies the difference between these approaches and how these terms can overlap or contradict each other in a series of books on cloud data warehousing.
In the first book, Cloud Data Warehousing Volume 1: Architecting Data Warehouse, Lakehouse, Mesh, and Fabric, he discusses the foundational principles of cloud data warehousing and how to create an environment to support insight-driven decision-making. Devlin draws upon more than 35 years of IT experience and industry insights; he defined the first data warehouse architecture at the IBM Dublin Lab in 1985.
Devlin challenges readers to consider not just what new technologies can provide, but whether they're necessary additions to their operations.
"This discussion of purpose and principles leads to a conceptual architecture for data warehousing," Devlin writes in Volume I. "It turns out this conceptual level of architecture remains exactly the same, whether implementation is planned on-premises or in the cloud."
He uses "seven deadly sins of data warehousing" to show that old assumptions may be subconsciously applied to design decisions, despite possibly being solved or mitigated today. These sins demonstrate the importance of constantly questioning assumptions in design thinking.
Volume II, which is expected to be published in early 2024, will dig into the architectures of the modern cloud data warehouse, lakehouse, fabric and mesh.
Here is an excerpt from Chapter 3, "Data Warehousing -- Purpose and Principles," in which Devlin analyzes how the discussion of the novelty and benefits of new infrastructure doesn't address implementation challenges or the underpinning rationale for data warehousing.