DuckDB is an open source database technology that, until now, has largely been used on premises. But that is about to change.
Today, MotherDuck revealed that it has raised $47.5 million in venture capital funding to build out a hybrid platform that will let on-premises DuckDB users take advantage of the cloud.
The 2022 startup is based in San Francisco and led by founder and CEO Jordan Tigani, who formerly worked at Google for a decade helping to lead BigQuery development. Until April of this year, he was chief product officer at distributed SQL database platform SingleStore.
DuckDB is an open source online analytical processing database that has only had limited commercial support offered from DuckDB Labs. MotherDuck is a different company than DuckDB Labs but has pledged to work with DuckDB Labs on continuing to help develop the underlying open source database technology.
MotherDuck's main goal is to provide a serverless, cloud-hosted version of DuckDB that will work in an integrated approach with on-premises users. The cloud version is currently in private preview with a public preview expected in 2023.
DuckDB competes against several different databases, including SingleStore, which is how the technology came to the attention of Tigani. It also competes against Apache Druid, which benefits from the support of commercial vendor Imply and its recently raised $100 million.
DuckDB and MotherDuck are useful for organizations that want to start with a local on-premises database, while having the flexibility to burst to the cloud when needed, said Sanjeev Mohan, analyst at Sanjmo Advisory.
Mohan also noted that some organizations face the challenge of dealing with the complexity of the modern data stack and need better usability, which is what MotherDuck aims to provide.
"With the advent of cloud, we have fixated on building ever-larger databases," Mohan said. "However, most organizations don't have extreme scale needs."
How the open source DuckDB database works with MotherDuck
Tigani first encountered DuckDB while he was working at SingleStore. He quickly realized that there wasn't a serverless cloud version of the technology, which inspired him to start MotherDuck.
Sanjeev MohanAnalyst, Sanjmo Advisory
Tigani and MotherDuck are building a hybrid model that will enable and support on-premises use of the DuckDB database that can burst to the cloud when needed for backup and high availability. The idea is to let developers use both the cloud and a local computer system to build and run a DuckDB database deployment.
"Generally, with cloud-based SaaS services, your local computer sits idle while you wait for the cloud to compute a result," Tigani said. "By moving pieces of data where they're needed, we're able to let an analytics deployment span both the local and cloud environment."
The MotherDuck platform isn't just DuckDB running in the cloud either.
The vendor is developing an optimized cloud storage layer that enables rapid synchronization with on-premises data stores. MotherDuck is working on the API to enable connectivity to and from the cloud to on-premises DuckDB deployments.
MotherDuck is also building a hybrid execution system that will understand where data is and how it can be accessed and used both in the cloud and on-premises.
The hybrid execution system is a cost-based optimizer that can help organizations decide where and when to best run a query and what data should move.
Hybrid use cases for MotherDuck
Tigani said he sees opportunity for MotherDuck in several applications, most notably in enabling scalability for existing users of on-premises deployments of DuckDB.
He also expects that MotherDuck will be useful for developers that need a local database running inside of an application but want to use the cloud when needed.
DuckDB can run in WebAssembly, which makes it small and portable -- perfect for operation on mobile phones, Tigani noted. Data can run locally on a user's phone, which saves cloud resources when needed, while still synchronizing with a cloud instance.
With MotherDuck, Tigani is using lessons he learned while working on BigQuery. One such lesson is that many database analytics workloads are not always large volumes of data. Many are less than 100 megabytes in size.
"Everybody's focusing on big data, but most people don't actually have big data," Tigani said. "So what we want to build at MotherDuck is an easy query platform that's just going to make it simple for people to understand their data and do analytics without a lot of, like, fuss."