Data Profiling Concept

From MIKE2 Methodology

This is a Concept Article which is used to describe a key Information Management concept. The purpose of this article is to provide a general definition of this concept.

Data Profiling is used quantitatively identify data quality issues. Data Profiling is carried out on the completeness of the fields, which determines the “usefulness” of the field for matching purposes. Incomplete fields mean that lower aggregate weights will be derived for the record, which can fail to meet the match cut-off requirements. Investigations are performed on both non-standardised and standardised fields. The purpose of investigating field patterns is to correct those patterns such that they can be standardised and used for matching, or to isolate those patterns for manual data quality improvement.

As part of this process, data profiling:

• Uncovers trends, potential anomalies, metadata discrepancies, and undocumented business practices
• Identifies invalid or default values
• Reveals common terminology used in a business area
• Verifies the reliability of fields proposed as matching criteria

Following this approach, a tools-based profiling assessment provides information about data structures and data content.

Additional Information on this Subject from Wikipedia

Wikipedia provides a detailed overview of many Information Management concepts. MIKE2.0 provides a mashup to Wikipedia to provide an enriched reference source for Concept Articles. To edit this article you need to login to Wikipedia.