Data analytics architecture must break down higher ed silos
Siloed data isn't only a problem for businesses. It's also a big issue for many large universities -- one that their data infrastructures need to resolve for effective analytics.
Media attention often focuses on elite private colleges and universities when the topic is academics or research; we mostly hear about public ones during sports seasons. That does a major disservice to public universities, which two-thirds of U.S. students attend. They're crucial to higher education -- and they face some big challenges.
Many states have cut funding for higher education, something that directly puts public universities at risk financially. At the same time, tuition rates are rising and student debt is getting more onerous. Those and other issues make it critical for universities to update their data analytics architecture to help administrators and other users better understand what's happening before major problems occur, whether that's with budgets or student success.
It's easier said than done, though. While many schools have adapted business and computing courses to provide the fundamentals students need to become the next business and data analysts in corporations, internalizing those lessons has been more difficult.
One reason is that the university IT environment is typically even more complex than the ones in business or government. We talk a lot about data silos in businesses, but large universities have a diverse mix of administrative operations, academic departments and research arms that tend to work in a much more autonomous way. The complexity of interactions, or the lack of them, creates challenges in data management and analytics in higher education that are not for the faint of heart.
As one example, let's look at the University of Kentucky. UK has a total enrollment of about 30,000 students -- and while most of us outside of the academic environment focus on the number of students, it's important to also realize that the Lexington, Ky.-based university has more than 14,000 employees. That's not a small organization, and every enterprise data management practitioner should relate to its data analytics architecture needs.
Adam Recktenwald
"The university is facing many challenges," said Adam Recktenwald, UK's executive director of enterprise applications. "The decades-old systems providing annual, almost handcrafted reports were not something that could help us adapt to the coming changes. We need a very strong, data-driven decision infrastructure to provide information to faculty, students and administrators."
Using data to boost student performance
While there are many areas in which UK has worked to improve decision support in recent years, one of the most visible is in linking those three groups of users via a single architecture in an effort to improve student performance and retention.
Public universities, especially land grant ones like UK, have far more open admissions standards than do private schools. To better retain both the most advanced students and others who aren't as well prepared, the university needed technology that could enable everyone to understand where individual students were in their progression in order to provide timely and appropriate assistance as required. The goal is to improve the six-year graduation rate among students from about 60% to 70% -- a big jump.
Another challenge involved a bane of student lives that most of us remember well from every term when we were in college: plotting out a class schedule that fits your academic plan, and then rushing to change things when a required course is full or unavailable.
"UK provides 8,000 [course] sessions every term," Recktenwald said. "Our decision systems need to be able to understand the need for bottleneck courses -- classes students across multiple disciplines must take."
And simply determining how many separate offerings are needed for each course isn't enough, he noted: "It's just as important to integrate schedules across departments to ensure those courses have available sessions that don't conflict with sessions required for a major."
One infrastructure, lots of sources
In planning the new data analytics architecture, one critical issue was the need for it to work smoothly with a variety of data sources -- 22 systems in all from multiple software vendors. The university's financial and ERP systems are SAP-based, which helped put SAP Hana into the mix as a possible database platform at the center of the architecture. HANA demonstrated the ability to access data in real time across the different systems, and UK deployed the in-memory technology in 2012.
Among other performance gains since then, Recktenwald said UK has reduced its extract, transform and load (ETL) times by 85%, partly by switching from a standard ETL process to what he called "lift and shift." Data is extracted in bulk and moved as is into HANA, where the transformations can be done more efficiently. It's what IT people are talking about when they refer to a change from ETL to ELT -- extract, load and transform.
BI and analytics applications risk being useless without a focus on breaking down silos and accessing the full assortment of data sources.
In academic and business settings alike, business intelligence and analytics applications risk being useless without a focus on breaking down silos and accessing the full assortment of data sources. That begins with an understanding of the challenges, and it proceeds through a clear-eyed analysis of the existing information landscape and a well-thought-out plan for integrating the various source systems.
Large public universities are complex organizations with information scattered in multiple places. The University of Kentucky is one such school that recognized the need to break down its data walls so users could better leverage what was behind them. Other universities -- and businesses, too -- that are hamstrung by an inadequate data analytics architecture might be able to learn some lessons from UK's experience.
Adam Recktenwald
