Personal tools

Partners

ETL Technology Selection QuickScan

From MIKE2 Methodology

Jump to: navigation, search

ETL Technology Selection QuickScan lists out some of the key capabilities expected from ETL product so that objective decisions can be made on the appropriate product to be used.

This tool can be used throughout the architectural process to provide comparative analysis of technology products. IT should be used during Phase 2 of the MIKE2.0 Methodology. It has initially been created for comparison against Foundation Capabilities of the SAFE Architecture but will be extended over time.

Assessment Approach

The assessment is answered by scoring each product against the assessment questions below. Scoring can be done at a team level or by a number of individuals; the assessment should be done individually for each product.

The scoring model is simple:

  • Answers are based on multiples of 2:
  • For capability rating:
    • 1 = No and not planned
    • 2 = No, in a future release
    • 3 = Yes, but has major issues
    • 4 = Yes, but has minor issues
    • 5 = Yes, ideally meets requirements
  • For importance weighting a multiplier is used:
    • 0 = no relevance
    • .5 = somewhat important
    • 1 = important
    • 2 = very important

These questions provide a 'starter set' to be used by project implementation teams.

Issues with the Approach

Clearly a wiki is not the appropriate technology to conduct an assessment. The purpose of putting content into this form was to initially gather some feedback and to drive reviews of open source products in relation to commercially available solutions.

The goal will be to move the assessment questions into a web-based model as has been done with Information Maturity QuickScan although the preference would be to create an Open Source Assessment Framework.

Assessment Questions

Listed below are the assessment questions in the tool.

Factors Categories Criteria Capability Importance Comments
Architecture Access Does the tool handle concurrent usage on overlapping data sets?      
Access Does the tool function in a c/s fashion using a shared repository?      
Access Does the tool allow repository access by 3rd party products?      
Convergence Can the outputs of data profiling be used directly in the tool?      
Convergence Do outputs feed directly into a metadata repository?      
Convergence What is the level of integration to a data re-engineering tool?      
Convergence Can jobs be componentised to be used by interfacing systems?      
Convergence Is the tool integrated with other integration products (e.g. BRE)?      
Standards Does the tool support using open standards (e.g. XML and J2EE)?      
Standards Does the tool support web services interfaces?      
Scalability Does the tool offer support for parallel processing?      
Scalability Does the tool support parallel processing by running a single job across multiple CPUs on a server?      
Scalability Does the tool support parallel processing by running a single job across multiple nodes in a cluster or grid?      
Scalability Can the tool support projected 3-year business growth volumes?      
Automation Scheduling Can reports be sent automatically to specific users?      
Scheduling Can a job be set to run at specific times through the tool?      
Scheduling Can multiple jobs be brought together in an overall sequence?      
Scheduling Can the tool be used entirely through the GUI, without scripting?      
Scheduling Does the tool have any built-in failover and restart capabilities for a sequence of jobs?      
ETL Integration Input Formats Does the tool support the required input file formats?      
Input Formats Can the tool source and load into non-relational formats?      
Integration Does the tool support connectivity to required data sources?      
Integration Does the tool provide adapters to interface application tier?      
Integration Does the tool trap and report database insert/update/delete error and warning messages?      
Integration Can the tool be used to execute operating system scripts or batch files? E.g. FTP a file before running a job.      
Advanced Features Does the tool support real-time integration?      
Advanced Features Does the tool provide off-the-shelf modules (e.g. SCD job)?      
Operations Usability Is the GUI "user friendly" and intuitive?      
Usability Can different jobs share the same transformation code without duplicating that code?      
Usability Does the tool support both generated SQL and user defined SQL for database extracts and loads?      
Usability Can the tool be readily used by business and technical users?      
Deploy Does the tool provide simple, wizard-driven installation?      
Deploy Does the tool provide comprehensive help files?      
Deploy Does the tool provide a comprehensive install guide?      
Deploy Does the tool support running on the client

’ s SOE platforms?

     
Manage Can the tool centrally manage property settings within jobs that frequently change?      
Manage Does the tool offer development collobation such as check-in, check-out and version history?      
Manage Is there a mechanism for moving components between dev, test and production environments with version history?      
Manage Is there a way to migrate between vendors (open standards)?      
Manage Is there a well-defined migration path between product releases?      
Manage Does the tool provide a mechanism for backup and recovery?      
Manage Does the tool offer logging/debuging functionality?      
Manage Does the tool have a console for run-time monitoring & ops?      
Manage Can the ops console be integrated to an Enterprise-wide console?      
Product Profile Risk Is the organisation responsible for the product well-placed in the global market?      
Risk Does the organisation have a strong financial position?      
Risk Is there a good relationship with the makes of the product in the organisation?      
Risk Is the product code available in an open source form?      
Execution Is the product in a state of maturity and widely used in the marketplace?      
Vision Does the organisation responsible for development have a product development vision?      
Vision Is this product undergoing continual improvement and development?      
Skills Does the product offer support with an active support forum?      
Skills Are there sufficient skills in the local market for the product?      
Skills Are there skills in the company with the product?      
Cost What is the estimated price of the product set?      

Powered by omCollab