pathdoc -

Meltano 2.0 provides a superset of open source DataOps tools

The startup and GitHub spinoff added the Apache Superset business intelligence technology to enable organizations to more easily build full data operations pipelines.

Nearly a year after spinning out from developer collaboration vendor GitLab, Meltano is out with the general availability of its 2.0 platform for DataOps.

The vendor also said that alongside the platform update, it raised $8.2 million more in funding, bringing total funding up to $12.4 million. Meltano had raised $4.2 million when the company was launched in June 2021.

The open source-based data integration vendor combines multiple tools that help organizations build out data pipelines and DataOps processes.

Meltano aims to enable organizations to easily use the different tools together, providing a dashboard interface for management and visualization that combines tools into a DataOps pipeline.

When Meltano was founded, its primary tools included the Singer project for data connectors and dbt for data transformation. With the 2.0 update, the vendor expanded the DataOps platform with support for Apache Airflow for scheduling and workflow orchestration, Apache Superset for analysis and Great Expectations for data quality.

Meltano faces no shortage of competitors in the DataOps market, including Datafold, which has a focus on data reliability; DataKitchen and its DataOps Platform; and Talend's data fabric. Even as a 2.0 release, Meltano's DataOps platform is lacking a number of capabilities. Most notably, it isn't yet available as a cloud-managed service that is easy for organizations to use without the need to install software.

How HackerOne uses Meltano for DataOps

Bug bounty platform provider HackerOne is among the users of the Meltano DataOps platform.

The company builds and maintains a data platform that provides actionable information for its users, and Meltano is at the core of the platform.

The future of DataOps is a data platform that is entirely defined in code, all the way from the data ingestion to the visualization of information.
Jobert AbmaCo-founder, HackerOne

The Meltano framework helps HackerOne's engineers rapidly develop new streams of information that are fully tested and easy to maintain, said Jobert Abma, co-founder of HackerOne.

"The future of DataOps is a data platform that is entirely defined in code, all the way from the data ingestion to the visualization of information," Abma said. "Prior to implementing Meltano at HackerOne, we had to combine and maintain a lot of tools ourselves, which led to a development stack that was hard to maintain, hard to test and hard to orchestrate deployments for. Meltano solves these problems for us."

Meltano 2.0 expands DataOps platform

Adding Apache Superset to the vendor's update, released on June 8, is perhaps the most significant component users will notice, said Douwe Maan, founder and CEO of Meltano.

Before the 2.0 update, Meltano's tools had been primarily focused on extracting and loading data with Singer and then providing transformation capabilities with dbt, Maan noted.

With support for Apache Superset, Meltano can now be used to build complete operations pipelines. The DataOps pipeline capabilities in Meltano now includes the Singer tool for data integration, dbt for data integration, Airflow for orchestration and Superset for analysis.

In Apache Superset, users have access to a popular open source business intelligence tool that has a growing community of users, Maan said. Meltano 2.0 enables organizations to more easily deploy and manage Superset as part a Meltano environment, he said.

There is also a business reason for Superset coming into Meltano.

Max Beauchemin, the original creator of Apache Superset, is an early investor in Meltano.

Beauchemin is currently the CEO and founder of Preset, which provides commercial support and services for Superset. Meltano is not bringing in any commercial software from Preset, but rather is only using the open source components, Maan noted.

Maan said that a key focus for the vendor for the rest of the year is to build a managed services capability in the cloud for Meltano. With Meltano 2.0, the platform entirely open source and available  in a self-managed format for users to run on their own.

The fully managed cloud version that is in development will be available in preview by the end of the year, according to Maan.

"The managed service will definitely broaden the total addressable market for Meltano to those teams that are a little bit less technical than the ones we're currently targeting," he said.

Dig Deeper on Data integration

Business Analytics
Content Management