Snowflake's AI capabilities fueling music data specialist

Prelude to AI

Based in Los Angeles, Luminate was formed in 2020 when Nielsen Music, Alpha Data and Variety Business Intelligence joined forces.

Now, the company's data feeds sources such as the Billboard music charts and is used to inform all aspects of movie and television production. Customers include major music labels, artist management companies, movie and television studios and music and entertainment creators.

Most of the data Luminate collects is structured, although some is also semi-structured and unstructured. Music data sources alone that Luminate currently collects from include streaming services, retailers, independent record stores and localized data from about 70 international markets. In total, the company ingests approximately four terabytes of data per day.

Currently, that data is managed with a combination of Snowflake and Amazon Simple Storage Service (S3). Meanwhile, Luminate is using capabilities in Cortex AI, Snowflake's environment for building and deploying AI tools, to develop cutting-edge applications.

Five years ago, however, Nielsen Music, Alpha Data and Variety Business Intelligence each stored their data on-premises. When they came together as Luminate, one of the new company's main initiatives was to migrate all its data to the cloud by putting it in S3.

We rebuilt everything from the ground up into what we are today, which is a combination of Snowflake and AWS. We wanted to future-proof the business. That was the main goal of what we were trying to do.

Once that migration was complete, Walker and other technology leaders met in June 2021 to develop a plan for further modernizing Luminate's data and technology infrastructures. The company was already using Snowflake on a small basis and determined to make it the core platform for its data initiatives.

"We rebuilt everything from the ground up into what we are today, which is a combination of Snowflake and AWS," Walker said. "We wanted to future-proof the business. That was the main goal of what we were trying to do."

While Luminate wound up using a combination of Amazon S3 and Snowflake to store and operationalize data, the company did explore other platforms following the June 2021 strategy meeting, he continued.

"If we were going to start a project from the ground up, we were going to look at many different options," Walker said.

Snowflake stood out, however, because it provides nearly unlimited storage, elastic compute capabilities that scale resources up or down based on demand rather than run at a steady state, and separates compute and storage to improve workload efficiency, according to Walker.

Still, Snowflake had to appeal to Luminate before Luminate made Snowflake one of its two platforms for data management. While data management is traditionally thought of in relation to industries such as finance, retail and manufacturing, music and entertainment is also a data-intensive industry. As a result, from Snowflake's perspective, Luminate is an attractive customer.

When Luminate was evaluating different data management vendors, Snowflake emphasized its data collaboration capabilities to appeal to Luminate, according to Michelene Rabbitt, global head of sports and music at Snowflake.

"The media and entertainment industry … relies extensively on data collaboration," she said. "As the [interactions with customers evolve], it's crucial for media and entertainment companies to share insights with their ecosystem partners to optimize customer journeys. Snowflake's data collaboration tools provide ways to share this data in a secure and governed manner."

In addition, Snowflake's support for structured, semi-structured and unstructured data is a way to attract a company such as Luminate that collects different data formats, Rabbitt continued.

Once Luminate started scaling its use of Snowflake, the transition from an on-premises data infrastructure to a cloud-based one was relatively simple, according to Walker. More difficult was rebuilding products and overcoming resistance to new things.

"That was more of a challenge than using Snowflake, specifically," Walker said. "Our data used to be in many very fragmented systems and siloed. Now, all of our data is together, and that has allowed us to extract some interesting insights."

Now, Luminate remains with Snowflake because the data platform vendor has demonstrated a commitment to evolving as data management and analytics trends emerge and evolve, including transitioning to a focus on AI over the past few years.

"Snowflake continues to invest heavily in its product roadmap, which allows us to do the same," Walker said. "It allows us to stay current with AI features. We can continue to innovate without reinventing the wheel."

Luminate and AI

Though it was only four years ago that Luminate's data was stuck in a fragmented array of on-premises systems, now the company is creating cutting-edge AI applications.

But before Luminate could develop Lumi, just as Luminate had to evolve, so too did Snowflake.

Before OpenAI launched ChatGPT in November 2022, marking a significant improvement in generative AI (GenAI) technology, Snowflake's primary focus was on providing a platform for data management. After ChatGPT was released, however, that focus shifted toward enabling AI development as enterprises -- including Snowflake's customers -- began showing more interest in AI because of GenAI's potential to make workers better informed and business processes more efficient.

Snowflake, however, was later than rival Databricks, tech giants such as AWS, Google Cloud and Microsoft, and many specialists to embrace AI development and provide customers with the tools to build AI applications. But when Sridhar Ramaswamy took over as CEO in February 2024, Snowflake shifted its emphasis to AI, and since then, has made it a top priority.

Now, with data management and AI development deeply intertwined, Snowflake's focus is on providing a platform that enables customers to develop AI applications trained on proprietary data in the secure environment of a single platform, according to Rabbitt.

Snowflake first introduced its Cortex AI development environment in June 2023, made it generally available in May 2024 and unveiled Cortex Agents in February 2025.

"Snowflake helps companies like Luminate move from AI experimentation to production through its unified platform, which keeps all data and AI workloads -- including agentic -- in one place," Rabbitt said.

One AI-powered tool Luminate has developed using Snowflake's platform is an anomaly detection capability, according to Walker.

Because Luminate provides industry-specific data to its customers, that data needs to be accurate and trustworthy for customers to have faith in Luminate. Anomaly detection is a means of identifying abnormal data points or events so they can be checked for accuracy before potentially providing misinformation to a customer.

Before building AI-powered anomaly detection capabilities, humans had to check data for anomalies and other potential inaccuracies. But when dealing with 4 TB of data on a daily basis, it's impossible for people to look at every potential anomaly in real time.

"Our data being correct and fully vetted is incredibly important," Walker said. "We have built a lot of complex anomaly detection across all of our data that we can now do in a fraction of the time [it would take humans]."

Anomaly detection, however, is for internal use. Lumi is Luminate's first customer-facing, AI-powered tool.

The feature has both GenAI and agentic AI capabilities, according to Walker. It acts autonomously to read through all the data Luminate collects as it comes in, determines which data is most interesting to an individual user and generates summaries of that data.

The project to develop Lumi began in June 2024, beginning with building a data foundation. Once that was in place -- a months-long task -- Luminate began building Lumi in early 2025 and embedded the feature in its customer-facing Streaming Viewership product, which measures online video consumption, in early August. Next, Luminate plans to embed Lumi in its product that measures music consumption by yearend 2025, with plans to add it to other customer-facing products going forward.

"The ability to quickly be able to get the data you're looking for without complexity is the goal," Walker said.

Lumi includes Luminate's proprietary data, AI capabilities from Anthropic's Claude large language models and was developed using Snowflake's Cortex AI. Development was complex, but a relatively smooth process, according to Walker.

"Nothing is really easy in that space, but I think it went well," he said.

Looking ahead

While adding Lumi to more products represents Luminate's near-term focus, long-term, the company plans to continue using Snowflake's AI development capabilities to build tools that enable users to uncover contextually relevant insights using natural language, according to Walker.

Snowflake's own roadmap is to add more features that simplify using AI to be better informed, according to Rabbitt.

"Our primary focus remains identifying ways to make AI more accessible for all users," she said. "We want to simplify generative AI and deepen agentic development capabilities to empower customers to build sophisticated AI apps with ease."

Toward that end, in June, Snowflake unveiled a swath of new capabilities aimed at better enabling AI development, including data engineering features aimed at making structured and unstructured data available together.

"Snowflake's roadmap is pretty well-aligned with our own," Walker said.

Meanwhile, regarding its own AI development plans, Luminate is looking to expand its use of Snowflake's Cortex Analyst to help users uncover insights from their structured data and Cortex Search to query and analyze unstructured data, according to Walker.

"We'll continue to expand our use across our ecosystem to continue to accelerate decision-making and expand self-service access to our data," he said. "We're looking to continue to unlock more contextual insights that are way too difficult or time-consuming for a person to surface."

Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than 25 years of experience. He covers analytics and data management.