What is Data Quality Management?

The world is living amid the Data Age. This means that more data being is produced today than there has been in the previous 5000 years of human history — roughly 2.5 quintillion bytes of data per day. Every time someone sends an email or text, downloads an app, send does any number of seemingly trivial things, data is created, and the compounded number of these interactions by millions of people has created an explosion of data. Instead of being overwhelmed by the data, your organization can become more data-driven. A shared trait of data-driven organizations is that they have a data quality management program in place to ensure they are working with the best data possible.

Why Organizations Need Data Quality Management

From the C-level on down, organizations are becoming aware of the importance of data quality management. There are common threads that are driving the need for data quality: integrating new sources of data, particularly unstructured data, with existing systems; the financial investment and competitive pressure needed to capitalize on all available enterprise data: and the difficulty of extracting data from the silos in which it resides, among others. Harvard Business School released a study which revealed that 47% of newly created data records contain at least one critical error.  An astonishing study conducted by MIT Sloan notes that bad data can cost as much as 15-25% of total revenue.

Bad data doesn’t have to cost your organization time and money. A solid data quality management program will ensure that the integrity of data is high, and the data readily available to anyone who needs it in a secure and governed fashion. Data quality management is all about finding the right combination of having the right people equipped with the right tools following the right approach.

The people: The collaborative path to data quality

Data quality management initiatives shouldn’t have to rely on a small IT team or a couple of rockstar data personnel to execute. Data is a team sport; everyone from IT to data scientists to application integrators to business analysts should be able to participate and extract valuable insights out of constantly available, good quality data.

When embarking on a data quality management program, it’s important to work with data as a team, otherwise, you may become overwhelmed by the amount of work needed to validate trusted data. By introducing a Wikipedia-like approach where anyone can potentially collaborate in data curation, there is an opportunity to engage the business in contributing to the process of turning raw data into something that is trusted, documented, and ready to be shared.

IT and other support organizations such as the office of the CDO need to establish the rules and provide an authoritative approach for governance when it is required (for example for compliance, or data privacy.)

You need to establish a more collaborative approach in parallel, so that the most knowledgeable among your business users can become content providers and curators. By leveraging smart and workflow-driven self-service tools with embedded data quality analysis controls, you can implement a system of trust that scales.

The tools: A unified data quality management platform

There are plenty of data preparation and stewardship tools offering several benefits to fight bad data. But only a few of them cover data quality for all. These specialized, standalone data quality management tools typically have a complex user interface that requires deep expertise for successful deployment. Sure, these tools can be powerful, but if you have short term data quality priorities, you will miss your deadline.

On the flipside, you might find simple and often robust apps that can be too siloed to be injected into a comprehensive data quality process. Even if they successfully focus on the business people with a simple UI, they will miss the big part — collaborative data management. And that’s precisely the challenge. Success relies not only on the tools and capabilities themselves, but on their ability to talk to each other. You therefore need to have a platform-based solution that shares, operates, and transfers data, actions, and models together.

You will confront multiple use cases where it will be impossible for one person or team to manage your data successfully. Working together with business users and empowering them on the data lifecycle will give you and your team superpowers to overcome traditional obstacles such as cleaning, reconciling, matching, or resolving your data. Here are ways that data quality tools can support your data-driven organization:

• Analyze your data environment: Data profiling — the process of gauging the character and condition of data stored in various forms across the enterprise — is commonly recognized as a vital first step toward gaining control over organizational data.
• Share quality data safely: Selectively share production quality data using on premises or cloud-based applications without exposing Personally Identifiable Information (PII) to unauthorized people.
• Manage the data lifecycle: Data stewardship is the process of defining and maintaining data models, documenting the data, cleansing the data, and defining its rules and policies. It enables the implementation of well-defined processes covering several activities including monitoring, reconciliation, refining, de-duplication, cleansing, and aggregation to help deliver quality data to applications and end users.
• Prepare and share data quickly: Too many people are still spending too much time crunching data in Excel or expecting their colleagues to do that on their behalf. Data preparation tools allow potentially anyone to access a data set and then cleanse, standardize, transform, or enrich the data — this shared ownership ultimately drives collaboration between business and IT.

The cost of bad data quality can be counted in lost opportunities, bad decisions, and the time it takes to hunt down, cleanse, and correct bad errors. A solid data quality management program comprised of the right mix of people and technology is the best way to ensure data quality for everyone who needs it. Talend Cloud offers numerous tools to help you achieve your data quality goals and help your organization become truly data-driven.