Bank sees ETL processes evolve from IT to front-line users
ETL jobs -- once the sole province of IT -- take on a new form as data wrangling and self-service gain greater traction with business users of analytics.
There is a lot going on behind the big talk about self-service for front-line business analysts. In the background, a lot of new data infrastructure must be built out. The ultimate goal is to speed creation of new products and improve efficiencies. Along the way, this means changing ETL processes.
"We are working to create a platform that allows the data owners to manage the data themselves," said Alasdair Anderson, executive vice president for technology at Nordea Bank, a growing European bank built in part by mergers with other banks across Northern European countries.
Anderson said part of that build-out involves a shift in how extract, transform and load is approached. It can sometimes be a bottleneck, putting a curb on new business initiatives.
Alasdair Anderson, executive vice president for technology at Nordea Bank
A trend that some call data wrangling is meant to change ETL processes. Rather than inundating IT-based teams of ETL scripters with transformation chores, new infrastructure is intended to identify and automate commonly used data transformation workflows based on how business analysts and others typically use the data. The intended result is less work on the IT side, and more self-service capabilities for end users.
More automation for ETL processes
"ETL is moving from IT to the business," said Anderson, who leads a team of over 400 people working on data engineering at Nordea. "We don't think IT should own the data."
The drive to shift more responsibility for data transformations to business folks has led data engineers at Nordea Bank to construct a stream of new data infrastructure and tools. Included are Cloudera Hadoop, Arcadia Data visual analytics and Trifacta self-service data preparation software.
ETL is moving from IT to the business.
Alastair AndersonNordea Bank
For this IT veteran well-versed in traditional ETL and enterprise data warehouse technology, these new tools are striking in comparison.
"With the old enterprise data warehouse world, you tended to have to go with one vendor. Where you used multiple vendors' software, you found they didn't play very well together," he said. "You had a lot of integration work to do."
It's a new world
"Trifacta and the others we have chosen subscribe to the open source model," Anderson said. "It's far easier than the old world. It doesn't take you down a cul-de-sac. And you don't end up with vendor angst."
Anderson said the Trifacta data preparation tools were "an important part of solving the puzzle" when cutting the time needed for Nordea to implement Single Euro Payments Area requirements intended to streamline transactions within the eurozone.
The work Nordea is doing with self-service data analytics has enabled the bank to be more responsive to changes in the regulatory environment, he said.
Anderson added that Nordea's success in compliance can also have downstream benefits, and can contribute to the bottom line.
"The bank feels that being good at compliance is a competitive advantage -- not just a cost of doing business," he said.
Next Steps
Take a look back at the roots of the Hadoop ecosystem
Learn about ETL and data wrangling -- including telemetry data wrangling
Catch up on recent events in SQL-enabled data pipelines
