Most organisations have problems with their data – not enough, too much, not good enough, missing, duplicated, inconsistent etc. Many managers and employees just want someone to ‘sort it all out’, but who is responsible?
Often there is a view that data quality is IT’s problem, but is this correct?
To illustrate the answer, I will use the concept of a warehouse as an analogy for the data:
- Warehouses typically contain shelves and racking all housed in a suitable weatherproof building
- There will usually be some fork lift trucks or other machinery to help load products/ goods onto/off the shelves and racking
- There will be loading bays where vehicles unload and load goods
- The company that set up the warehouse want to make sure that their building, racking and machinery are looked after and not damaged
- They are not particularly bothered what products are put on the shelves, so long as they are not overloaded
Now, how does that relate to the IT part of the analogy?
- IT provide the software and hardware
- IT provide the user interfaces and links between systems
- IT want to make sure that data volumes and formats do not cause hardware or software errors
- IT are not particularly bothered what the data quality is, so long as it does not cause software errors
I often state that, just because data is valid and plausible, does not mean that it is right. For example, the data about me may state I have a full head of ginger hair – this data is valid and plausible, but clearly wrong!
The people who know whether data is right or wrong are the employees who interact with the data and the thing it represents – which could be a customer, vehicle, asset or building. IT usually cannot know when data is wrong
The people who can input the ‘right’ information about an activity – what the problem was, how it was resolved etc. are the people who are part of the relevant business process. IT do not generally run the process or get involved in the activity concerned.
Enterprise data quality needs considering at an enterprise level, not at an individual data store level as explained in this blog post. Business users who create spreadsheets to store corporate data may be creating data challenges for the organisation, but they are not part of IT and these spreadsheets are likely to be ‘below the radar’.
Data quality is mainly a business problem.
IT do have a role to play in maintaining the infrastructure and interfaces between systems, ensuring there is suitable capacity for data volumes etc. However, getting data right, keeping it right, spotting when it is wrong, storing it in the correct places are all responsibilities for business users.
Our popular Data Zoo concept helps explain how different user behaviours impact data quality and the drivers for these behaviours.
How can you improve approaches to data quality?
There are many things to consider when trying to improve data quality, far too many to include in this article. The book ‘Managing Data Quality‘ by Tim King and I provides most of the information you need:
- The book explains about the challenges of managing data at an enterprise level
- Different user and organisational behaviours and their impact on data
- ISO 8000-61 as a framework for data quality management
- Implementing data quality management
We also have a related online training course for individuals and organisations to use.
Improving data quality is an organisational activity that can take time to implement – if you want to find out how we can support you on this journey, feel free to get in contact.