Sometimes a level of specialised data cleaning is needed that cannot be achieved by automated enhancement techniques
What our approach delivers:
- Clear improvements over machine-cleaned data
- Particularly suitable for business customer data
- De-duplication, correction and matching
- Standard and client-specific enhancement rules
- Supported by on-line and telephone research
How our approach works:
- Designed for each client’s data challenges
Our approach begins with a combination of understanding the organisation’s experience of its data quality challenges and our own more formal data quality assessment. From these we develop the enhancement regime. This will normally include a number of our standard cleaning elements plus other, client-specific elements.
- Driven by business rules wherever possible
As much of the enhancement regime as possible is documented in advance, in business rules that ensure complete clarity about most of what will happen to the data. They will normally be supplemented by human judgement but always form the basis of the core enhancement activity.
- Combines computer speed with human judgement
Everything that can be reliably programmed into database queries and routines is carried out early in the process. Even at this stage exception management puts any suspect situations in front of human eyes for a judgement. Later stages invariably involve a far higher level of human involvement.
- Application of carefully controlled common sense
Common sense is applied to the enhancement activity to a degree that has been agreed with the client. This may be limited to the correction of common mis-spellings and expansion of an agreed list of abbreviations. It can also be extended to making judgements about suspected duplicate records and identifying industry-specific errors.
- Supported by research where needed
The cleaning routines can be enhanced to include on-line or telephone research if required. This can involve checking organisation websites and company records to understand corporate hierarchies or collaborating on contacting individual customers to confirm and update their details.
- Learning can be fed into ongoing data capture and loading
As well as delivering the enhanced data itself our approach can document all the business rules used and the learning from any manual interventions that have been frequently applied. This can form the basis of a specification to the IT Team supporting the ongoing loading of data to prevent future problems occurring. It can also drive the development of a data capture guide for colleagues who collect and enter customer data.