Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Wiki Home
Collapse Expand Close

Members
Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Perform Data Standardisation Deliverable Template

From MIKE2.0 Methodology

Share/Save/Bookmark
Jump to: navigation, search
Under review.png
This article is a stub. It is currently undergoing major changes as it is in the very early stages of development and is only a placeholder. Please help improve MIKE2.0 by adding to this article.
This deliverable template is used to describe a sample of the MIKE2.0 Methodology (typically at a task level). More templates are now being added to MIKE2.0 as this has been a frequently requested aspect of the methodology. Contributors are strongly encouraged to assist in this effort.
Deliverable templates are illustrative as opposed to fully representative. Please help add examples to this template that are representative of the proposed output.

Data Standardisation addresses problems related to:

  • Redundant domain values
  • Formatting problems
  • Non-atomic data in complex fields
  • Embedded meaning in data

The Data standardisation process is used to get data into an agreed-to atomic form, oftentimes mapping in data from complex fields using a vendor tool. Mapping rules from the standardisation processes are ideally fed into a metadata repository.

Contents

Data Standardisation

Overview

Data standardisation brings data into a common format for migrating into target environment. Data Standardisation addresses problems related to:

  • Redundant domain values
  • Formatting problems
  • Non-atomic data from complex fields
  • Embedded meaning in data

The Data standardisation process is used to get data into an agreed-to atomic form, oftentimes mapping in data from complex fields using a vendor tool. Mapping rules from the standardisation processes are ideally fed into a metadata repository.

  • Design approach around Data Standardisation
  • Role assignments and ownership of standardised model
  • Metadata mappings from source to standardised model
  • Standardised data from source systems into a staging environment

Steps in the Process

This section provides a process-driven approach for the executing the standardisation process. These tasks are run serially; agreement on the set of common data elements may involve several iterations.

Step 1 Identify Common Data Elements for standardisation model
Objective: In this step, team members define the list of common data elements from the source system environment.
Input: Identification of data sources
Process: Data is standardized into a set of common data elements. Key examples of non-standard data include:
  • Redundant domain values
  • Formatting problems
  • Non-atomic data from complex fields
  • Embedded meaning in data
    In this step the initial set of common data elements is created for review.
Output: Identification of common data elements
Step 2 Ensure ownership and signoff of standardised model
Objective: To define ownership on the project team and ensure there is an ultimate owner the model that will be developed
Input: Definition of project roles and responsibilities
Process: Key business and technical stakeholders are identified that are relevant to data scope.
Team members are assigned stewardship responsibility for common model. Data stewards are assigned to act as reviewers of the model and act as the bridge between business and technical team.
In the Information Development organisation model, these responsibilities are already assigned. These roles would include:
  • Information Development Architect
  • Information Integration Standards Manger
  • Metadata Development Manager
  • Data Quality Manager
  • Information Repository Manager
    Workshops are help to gain final signoff on the model. This process may take multiple iterations to gain consensus. The Information Architect, assisted by Data Stewards should be the key facilitators in driving this process.
Output: Initial definition of standardised model
Step 3 Map source to standardised model
Objective: Initial source files are mapped to standardised model
Input: Signoff off on standardised model
Process: In this step, the source non-standardised data is mapped to the common model. Mapping rules would include:
  • Merge rules for producer to consumer file-entity mapping
  • Transformation rules for producer to consumer field mapping
    Mapping of complex fields may involve use of a tool for the standardisation process. For Customer and Address fields in particular, off-the-shelf algorithms may used for breaking up complex fields into atomic units.

    The standardisation process should be executed against the model and the new standardised model built.

    Ideally, these mapping rules are stored into a metadata repository. For some vendor tools, this will be a by-product of the development process.
Output: Mapping of data into standardised model
Loading into staging area of standardised model

Examples

Wiki Contributors
Collapse Expand Close