In this fourth blog post about ISO 8000:150 we will look at the Data Quality Improvement processes that form part of the overall data quality management approach defined in this standard. For an overview of the standard, please see ISO 8000:150–A framework for Data Quality Management. The Data Quality Improvement process corrects data errors detected and eliminates the root causes of data errors :
- Data Stewardship and Flow Management is the process that analyses data flows and responsibilities and manages the stewardship of data operations
- Data Error Cause Analysis is the process that identifies root causes of data errors in order to prevent them reoccurring
- Data Error Correction is the process that corrects data that does not meet standards or data quality criteria
Whilst the Data Operations processes focus on ‘leading’ factors for data quality and the Data Quality Monitoring processes are ’lagging’ indicators of data quality problems, the Data Quality Improvement process should correct data to the levels intended.
Data Stewardship/ Flow Management
Due to the interactions of data with processes and the way that data can be shared across systems and data stores it is important to understand how data ‘flows’ across an organisation in order to be able to manage its quality effectively. The two core activities in this process are:
- Defining who is able to access and change data in various systems i.e. Stewardship assignment, and on-going management of these authorisations
- Data flow management is used to understand the relationships between the same data in different data stores and how changes in one should ‘flow’ to others. This is a component of Master Data Management and will typically involve coordination across a number of organisational units
These activities also link to a number of other data quality processes:
- There is a close link between Data Flow Management and Data Architecture Management since one defines the data relationships and the other how data flows between them. From a personal perspective, I would have seen these grouped as a single process in the standard
- This process is informed by data quality planning since this sets the framework and priorities for this activity
- Data Error Cause Analysis can take information from this process into the analysis process
Data Error Cause Analysis
Using the old analogy that you would be wasting your time cleaning the water in a swimming pool if you had not cured the source of the dirt, similarly for data, it is important to identify root causes of data errors so that these can be resolved before attempting to correct the data. The two core activities in this process are:
- Identifying and correcting root causes – a challenge here is to develop suitable criteria for which data errors to assess in order to avoid getting bogged down in analysis. Once the root cause(s) have been identified, they should be corrected
- Other data stores and systems should be assessed to determine if these root causes could also exist there, in which case they should also be corrected. This activity should help prevent repeat problems.
This process has links to a number of other data quality processes:
- Depending on the error causes identified, there may be a requirement to adjust data stewardship and data flows
- Outputs of the error cause analysis should be an input to data error correction
- Outputs of error cause analysis can be inputs to be the data quality criteria and data designs in order to further reduce the risk of reoccurrence
Data Error Correction
Data error can be identified from a number of sources, not least the data quality monitoring and data error cause analysis processes. The process involves:
- Agreeing who, how and when data errors will be corrected
- Ensuring that corrections are shared with other parties and data stores as required
For both the above activities, it is important to ensure that staff have the required authorisation to make the corrections.
These processes should help improve data to required levels in a controlled manner. The next blog post in this series considers organisational approaches relating to this framework.