Data quality techniques and best practices

What is data lineage? Techniques, best practices and tools

Organizations can bolster data governance efforts by tracking the lineage of data in their systems. Get advice on how to do so and how data lineage tools can help. Continue Reading

Data lake governance: Benefits, challenges and getting started

A data lake that isn't well governed may become more of a swamp. Here are key benefits and challenges of data governance in a data lake, plus initial steps to take. Continue Reading

How to build a data catalog: 10 key steps

A data catalog helps business and analytics users explore data assets, find relevant data and understand what it means. Here are 10 important steps for building one. Continue Reading

Top benefits of data governance for businesses

Effective data governance provides a variety of benefits to organizations, including improvements in operational efficiency, data quality and business decision-making. Continue Reading

8 proactive steps to improve data quality

Here are eight steps to take to improve your organization's data quality in a proactive way, before data errors and other issues cause business problems. Continue Reading

Data quality startup Superconductive raises $40M

The vendor behind the open source Great Expectations data quality technology is looking to build a commercial cloud service to help data professionals accurately profile data. Continue Reading

Self-service data preparation: What it is and how it helps users

Using self-service tools to properly prepare data simplifies analytics and visualization tasks for business users and speeds complex modeling processes for data scientists. Continue Reading

6 data preparation best practices for analytics applications

Amid issues of data sources, data silos and data quality, the process of collecting and prepping data for analytics applications requires a practical and effective approach. Continue Reading

Data preparation in machine learning: 6 key steps

Trustworthy analytics outcomes depend on the right data, requiring data scientists to focus on these steps when they prepare data for use in machine learning applications. Continue Reading

Anomalo brings data quality platform to Snowflake

Elliot Shmukler, co-founder and CEO of Anomalo, details how he was inspired to build his data startup after a critical data failure while he was working at Instacart. Continue Reading

Western Union looks to improve data quality with Talend

Thomas Mazzaferro, chief data officer at Western Union, outlines the financial services company's approach to cloud data migration and how Talend fits into the data architecture. Continue Reading

Data forecast for 2022: Data quality and cloud convergence

Expect data quality to be a top area of investment and activity next year as the need to trust data for operations, insight and machine learning will only continue to grow. Continue Reading

Bigeye brings new visibility to data observability

The startup continues to build out its cloud platform that enables data engineers to identify and remediate data pipeline quality and availability problems. Continue Reading

LogicMonitor introduces centralized log management platform

The data observability platform vendor's new platform enables enterprises to use AIOps and automation to find anomalies with their IT system, report them and head off widespread problems. Continue Reading

Business shift to a data monetization strategy elevates CDOs

As their focus dramatically swings from compliance issues to data monetization, chief data officers are on track to take their rightful place among C-level executives, but slowly. Continue Reading

Datafold raises $20M for data reliability engineering

Datafold's founder and CEO details the data observability challenges the startup is looking to address with its suite of data tools that provide visibility into data pipelines. Continue Reading

Knowing the right data enrichment techniques is crucial

In 'The Enrichment Game,' the latest book by 'Data Guy' Doug Needham, you'll learn why and how to cull data to gain valuable insights and a business advantage. Continue Reading

How to choose exactly the right data story for your audience

A data practitioner has two jobs: tell the right data story and in the right way to win over project stakeholders, data expert Larry Burns says in his latest book. Continue Reading

EY CTO outlines data governance challenges

Multinational professional services firm EY has taken a strategic view of how to manage and use data in a federated approach powered by a trusted data fabric. Continue Reading

How data governance and data quality work together

High-quality, reliable data is essential to the data governance process. Here are strategies to ensure data quality standards are ingrained in governance processes. Continue Reading

Quest updates Erwin Data Modeling and Data Intelligence

Quest is looking to build out what it calls a data empowerment platform with the help of data modeling and other technology from its recently acquired Erwin business unit. Continue Reading

The enterprise advantages of automated data collection

Many organizations still rely on manual data entry that wastes time and results in low-quality data. Here are the latest automated data collection techniques and their benefits. Continue Reading

Prefect raises $32M for dataflow automation technology

Data startup Prefect is looking to grow its open source project and commercial business to help organizations manage automated data pipeline requirements. Continue Reading

Cribl aims to ease data observability with LogStream update

LogStream 3.0 brings new configuration capabilities to Cribl's pipeline technology that can help organizations optimize log and metrics data. Continue Reading

How automated metadata management improves business insights

Automating metadata management can cut down time spent on tasks such as data tagging and cataloging. Explore how automated metadata management is improving data quality. Continue Reading

Building a big data architecture: Core components, best practices

To process the infinite volume and variety of data collected from multiple sources, most enterprises need to get with the program and build a multilayered big data architecture. Continue Reading

Data quality for big data: Why it's a must and how to improve it

As data volumes increase exponentially, methods to improve and ensure big data quality are critical in making accurate, effective and trusted business decisions. Continue Reading

Who belongs on a high-performance data governance team?

Putting together a high-quality data governance team can be a challenge. Explore the necessary team members and best practices for a high-performing team. Continue Reading

IoT data quality determines project profitability

The insights drawn from IoT data analysis can only improve with better IoT data quality, which data scientists measure with metrics such as accuracy, timeliness and completeness. Continue Reading

Why your data story matters and how to tell it

Data storytelling isn't just for business analysts. Find out how to build a data management story and why you need to have one in the first place. Continue Reading

Collibra acquires predictive data quality vendor OwlDQ

With OwlDQ, Collibra is bringing data quality capabilities to its data intelligence platform to help users better understand and evaluate the impact of different sources of data. Continue Reading

Pandemic exposes difficulty of data management in education

Limited resources and a shift to remote learning have shown the inequalities across school districts when it comes to data management and the negative impact this can have. Continue Reading

Data lineage documentation imperative to data quality

Understanding the detailed journey of data elements throughout the data pipeline can help an enterprise maintain data quality and improve trustworthiness. Continue Reading

The top 6 use cases for a data fabric architecture

Enterprise data fabric adoption has been on the rise as a way to ensure access and data sharing in a distributed environment. Here are the top use cases for data fabrics. Continue Reading

Data protection officer responsibilities and role importance

The data protection officer title has been growing over the last few years, and organizations are still working to grasp the full responsibilities of the role. Continue Reading

How AI data privacy can help your enterprise

Enterprises benefit in many ways from AI data privacy tools that reduce the need for manual efforts from data professionals. Read on for top use cases for the growing technology. Continue Reading

The new normal for enterprise data governance

Mastercard data exec highlights the foundational role of enterprise data governance during the pandemic era with more people working from home and new demands on businesses. Continue Reading

Key steps in the feature engineering process

Feature engineering is key to machine learning algorithms. Read on to learn how those features are created and chosen to increase the accuracy of those models. Continue Reading

Talend evolves enterprise data integration efforts

The CTO of Talend outlines the path the data integration vendor has taken in recent years as the market for big data has evolved along with the need to treat data as an asset. Continue Reading

The evolving role of the chief data officer

The job of the chief data officer is expanding to be more strategic as the need for organizations to connect and make sense of vast sums of data continues to grow. Continue Reading

Why you should consider a machine learning data catalog

A machine learning data catalog can benefit an enterprise in a variety of ways, from increasing access to necessary data to keeping your data sources up to date. Continue Reading

How to streamline your data cleansing process

Data cleansing is an important part of maintaining data quality, and the process is easier if you keep ahead of it by upholding governance and quality standards. Continue Reading

Top 7 data catalog use cases for enterprises

From data lake modernization to data democratization, there are many benefits to a data catalog. Experts talk about the top ways data catalog adoption can benefit enterprises. Continue Reading

Top 5 feature engineering tips for better models

From understanding a model's expected goal to factoring in subject matter expertise, experts talk about the best ways to improve your feature engineering. Continue Reading

Surprising insight on the data governance process

Many enterprise data governance processes have been around for a while. Read on for ways a modern approach can be beneficial, and preview a new book that can update your strategy. Continue Reading

GDPR, CCPA, cloud drive security management tool makeovers

As data protection and privacy laws like GDPR and CCPA take hold, data managers refine governance practices, while vendors enhance traditional big data security tools. Continue Reading

Improving enterprise data governance with data trust

Talend users from Covanta and AutoZone provide insight into data management challenges amid growing data complexity at the Talend Connect virtual conference. Continue Reading

Alternative data in finance holds potentials and risks

Alternative data sources, when verified as reliable and trustworthy, can be an essential tool for financial institutions when it comes to gathering information. Continue Reading

Data governance and COVID-19 data security challenges

Maintaining data governance and data security best practices is essential now more than ever. But the increase in working from home can put a strain on those practices. Continue Reading

Common data lake challenges and how to overcome them

Managing the data contained in your enterprise data lake presents many challenges. From the amount of data to data inconsistencies, here are some solutions to common issues. Continue Reading

How data governance and data management work together

Data governance is an important part of data management. These recommendations will help build a strong complementary relationship between the two strategies. Continue Reading

How to build an effective streaming data architecture

Data architecture can be tricky when it comes to real-time analytics. Clear objectives and scalability are important factors when determining the streaming data architecture you need. Continue Reading

Data governance metrics: Data quality, data literacy and more

Statistics on data quality are the most common metrics used to track data governance programs, but other types of data-related measurements also can be applied. Continue Reading

6 best practices on data governance for big data environments

Efforts to govern big data must corral a mix of structured and unstructured data. That's a challenge for most organizations. These six action items will help. Continue Reading

Breaking down data silos with strong data governance

Not having a single source of truth can be a huge issue for data professionals. But strong data governance policies can prevent data silos and lead to better-quality data. Continue Reading

Hitachi Vantara acquires data catalog vendor Waterline Data

With the acquisition of Waterline Data, Hitachi Vantara is bringing new data catalog capabilities that will expand the Lumada Data Services and DataOps portfolio. Continue Reading

What are the main features of data catalog software?

Data catalogs are curated data portals for self-service analytics users. Learn about the capabilities of data catalog tools, vendors that sell them and how catalogs work. Continue Reading

Evaluate your data governance and privacy best practices

GDPR and CCPA have put data governance in the news. With this increase in data legislation, now is a great time to review your company's data governance policies. Continue Reading

How to improve data governance for self-service analytics

Citizen data scientists and self-service analytics are on the rise as the data scientist shortage continues. Here are some data management best practices to support them. Continue Reading

Data catalog management for analytics fraught with unique demands

Data catalogs for analytics applications demand detailed assessment of user needs, cross-functional teams, ready access, continuous improvement and a self-sustaining system. Continue Reading

How to streamline feature engineering for machine learning

Structured data is necessary in machine learning, but sifting through data is time consuming. Streamlining the feature engineering process can help data scientists be more productive. Continue Reading

How AI could save the enterprise data catalog

Enterprise data catalogs record all the databases and files in a corporation, but sometimes it can become out of date. Advances in AI are helping minimize the problem. Continue Reading

DOD CDO shares 7 data management best practices

The DOD's first chief data officer, Michael Conlin, shares tips on how organizations can modernize their data management practices with the objective of matching private enterprise. Continue Reading

Align business and IT drivers through data quality best practices

Information management consultants talk about the importance of aligning business and IT drivers for a holistic architectural approach to ensuring strong data quality. Continue Reading

Information Builders exec talks data management ethics

In this Q&A, James Cotton, director of the Data Management Centre of Excellence at Information Builders, offers advice on how organizations can carry out ethical data management. Continue Reading

How to navigate today's business analytics governance challenges

Don't let a traditional analytics mindset lure you into complacency when it comes to advanced analytics governance. Here are the biggest governance roadblocks and how to avoid them. Continue Reading

Key components of an effective data virtualization architecture

Experts break down which elements -- both technical and nontechnical -- are most crucial to successfully deploying and managing a data virtualization architecture. Continue Reading

The evolution of the data preparation process and market

Organizations have long struggled with inconsistent data and other issues. Expert Andy Hayler explores how that has led to the rise of the data preparation tools market. Continue Reading

Advice on enterprise data cleansing from an SAP VP

SAP's Kristin McMahon details data cleansing best practices and explains why a good data cleanse needs continual communication, collaboration and oversight. Continue Reading

Why data silos matter: Settling ownership of data issues

Data management is often still seen as an IT task, but that can lead to data silos. Find out why the business should be in charge of its data as part of a governance process. Continue Reading

Machine learning approaches gain critical mass for data pros

Machine learning will bring change to analytics and data management, said data luminary Michael Stonebraker. Others agree managing such change will take special effort. Continue Reading

Big data management practices gain attention amid risks

Many data professionals have yet to solidify traditional data management practices, but they have a new set of challenges to overcome to ensure data privacy and avoid misuse. Continue Reading

How to navigate the challenges of the data modeling process

Data modeling and curation can help businesses more efficiently use data they've collected. There are challenges, however -- beginning with ensuring data quality. Continue Reading

8 tips to improve the data curation process

A data curation and modeling strategy can ensure accuracy and enhance governance. Experts offer eight best practices for curating data. First, start at the source. Continue Reading

Why organizations need a solid data governance strategy

The flood of data flowing into data warehouses, data lakes and other systems makes effective data governance a must for successful business analytics initiatives. Continue Reading

Tips for creating curated data sets for self-service BI users

Data curation initiatives can help streamline BI processes by reducing the amount of time users spend locating and preparing data. Get four tips for preparing data sets. Continue Reading

How data duplication in healthcare is diagnosed

Electronic health record systems have helped reduce duplicate patient data in hospitals -- but they haven't cured the problem. Find out how organizations are addressing the issue. Continue Reading

11 features to look for in data quality management tools

As the need for quality data has increased, so have the capabilities of data quality tools. Learn how collaboration, data lineage and other features enable data quality. Continue Reading

What is an enterprise data strategy?

Defining a data strategy can help focus an organization's data management initiatives -- but it isn't the same as data governance. Expert Anne Marie Smith explains why. Continue Reading

What is the best structure for data governance programs?

The optimal approach to a data governance framework includes a program team, a data governance council and stewards. Expert Anne Marie Smith explains what role each group plays. Continue Reading

5 to-dos for your GDPR compliance checklist

It's never too late to fine-tune your GDPR strategy. Expert Anne Marie Smith suggests a current state analysis of your PII protections, drafting a data privacy policy and more. Continue Reading

Why data quality tools matter in today's organizations

Data is everywhere -- but is it good? Data quality tools could answer that question and help save time and money. So why don't more organizations use the technology? Continue Reading

GDPR checklist items data managers might have overlooked

Now that the General Data Protection Regulation has been implemented, it's a good time for organizations to make sure they haven't missed any GDPR best practices. Continue Reading

How to manage data and the GDPR right to be forgotten

The General Data Protection Regulation requires organizations to delete personally identifiable information if customers ask -- which can be a challenge. Experts offer four tips. Continue Reading

Good data quality for machine learning is an analytics must

As companies add machine learning applications, they need to really understand -- and be able to improve -- their data. That's where data quality initiatives come in. Continue Reading

Data integration techniques to help keep BI data consistent

Data integration processes that aren't managed properly can create inconsistent data in BI and analytics applications. Here are some steps to avoid that problem. Continue Reading

GDPR rules can spur broader steps to protect sensitive data

Companies need to make compliance with GDPR's requirements on managing personal data a priority, but they should also work to implement wider data protection efforts in the future. Continue Reading

Data integrity protection spurs greater security spending

As hacking, ransomware and malware attacks mount, companies place big data protection and integrity among the primary reasons for increased spending on security software. Continue Reading

Security of big data fashions a whole new look with GDPR

As data managers scramble to protect their precious lakes of washed and unwashed data from the evils of hacking, malware, ransomware and botnets, there comes a privacy regulation of European Union origin that could change the way many U.S. companies protect their data from inside and outside forces. For some U.S. companies doing business in Europe and preparing for compliance, the EU's General Data Protection Regulation could be a well-intended enforcer of better data governance practices. But for businesses forced to change their old data protection habits, GDPR can be a four-letter word.

The April issue of Business Information opens with our editor's note and historical insights into the fundamental differences in attitude between the U.S. and Europe when it comes to protecting the data privacy of customers. As data breaches mount, however, the U.S. government -- willing or not -- may have to take steps beyond current regulations to ensure companies increase their security of big data.

Along those lines, our cover story examines some GDPR rules that can directly impact U.S. companies doing business overseas and, in the process, their data governance and ethics procedures. With the noted lack of maturity in data governance and security tools, we see in another feature how IT teams are addressing data security issues upfront in do-it-yourself ways when deploying big data systems.

Also in this issue, a GDPR compliance expert advises data managers on the best ways to prepare for a regulation that puts more control of information in the hands of users, the internet of things and edge computing could be compromising the security of big data and surveys show that companies place data protection among the top reasons for escalating their security spending.

 Continue Reading

U.S. data protection laws fall short in the age of big data

Data breaches and a history of data abuse led the EU to adopt GDPR, but it might take massive scale data security crises for the U.S. to legislate similar data protection laws. Continue Reading

Data managers must hustle preparing for GDPR regulations

Data privacy expert and SAP consultant Stephen Lofthouse outlines a six-step process that data managers can follow to be compliant with GDPR regulations. Continue Reading

Information architecture applied to big data streaming, AI

New technologies challenge data professionals, but taking a step back helps with hurdles. In this interview, consultant William McKnight takes a measured look at data streaming, GDPR and AI. Continue Reading

Cloud workloads, data lakes challenge information architecture

Data management options are expanding; cloud workloads are an example. That means changing your approach to information architecture, says data management expert William McKnight. Continue Reading

GDPR mandates push data quality improvement into IT spotlight

Data quality issues cost companies significant amounts of money in lost revenue and added expenses, and their impact will only get bigger when the EU starts enforcing its GDPR law. Continue Reading

Infographic: The evolution of the chief data officer role

The CDO role, which has never been rigidly defined, is undergoing a face-lift as emerging technologies present new opportunities to enterprises. Continue Reading

How to determine which data governance tool best meets your needs

This buyer's guide provides insights to help you determine if your organization could benefit from data governance software and how to navigate the buying process. Continue Reading

Examining the top data governance tools on the market

Expert John Ladley examines the leading data governance software products, comparing and contrasting their features to help you determine which tool will best meet your needs. Continue Reading

How to narrow down your choices for buying a data governance tool

Evaluating and selecting a data governance tool depends on not only features and functionality, but also how you will use the tool to add value to your organization. Continue Reading

Data quality metrics a governance must, managers say

Data governance managers who spoke during an online conference said that tracking business-oriented metrics on data quality improvement is a key to success in governance efforts. Continue Reading

How to identify master data in a multi-domain MDM program

In an excerpt from their book on managing multi-domain master data management programs, Mark Allen and Dalton Cervo explain how to identify MDM domains and your master data. Continue Reading