End-to-End Testing

From MIKE2.0 Methodology

Activity: End-to End Testing

Objective

End-to-End (E2E) Testing validates the entire application to ensure that it satisfies previously established acceptance criteria and performs as an integrated system. The purpose of system testing is not to test all possible positive and negative conditions (reserved for functional and integration testing), but to instead execute business functions and infrastructure management (i.e.; batch processing, system security features, backup and recovery etc.) in an isolated and controlled environment to validate that a quality system is ready for production.

As the purpose of E2E Testing is to simulate a scenario seen by actual business users, test data should make sense from user perspective – the use of actual production data for E2E testing is ideal.

Major Deliverables

• E2E Test Results
• Changes to hardware/software environment (if required)

Tasks

Migrate Software to E2E Testing Environment

Objective:

Move software from configuration management environment to the E2E testing environment and ensure that all changes have been made prior to propagating software.

Input:

• Availability of a suitable and stable E2E Testing environment

Output:

• E2E Test Environment ready

Test BI Application Development Work Products

Objective:

Much of E2E Testing is done from a BI application perspective. E2E Testing of applications effectively mirrors the UAT process and in some cases it may make sense to get some of the users involved at this stage of the process.

Input:

• E2E Test Environment ready
• E2E Application Test Plans and Test Cases
• Completed Test Case Execution for BI Application Development SIT Testing

Output:

• Completed Initial Test Case Execution for BI Application Development E2E Testing

Test Integration Work Products

Objective:

The primary focus of E2E Testing is not to test the underling integration processes, but some degree of testing of integration processes during this aspect of the testing cycle may be appropriate.

During E2E Testing the integration processes should run as they would in production and in a completely automated fashion.

Input:

• E2E Test Environment ready
• E2E Integration Test Plans and Test Cases
• Completed Test Case Execution for Integration Development SIT Testing

Output:

• Completed Initial Test Case Execution for Integration Development E2E Testing

Test Information Development Work Products

Objective:

The primary focus of E2E Testing is not to test the information management processes, but some degree of testing of information management processes during this aspect of the testing cycle may be appropriate. In particular, the overall impact to data quality from an automated data management service may be tested during this stage. Data Monitoring capabilities may also be observed.

During E2E Testing the information management processes should run as they would in production and in a completely automated fashion.

Input:

• E2E Test Environment ready
• E2E Information Development Test Plans and Test Cases
• Completed Test Case Execution for Information Development SIT Testing

Output:

• Completed Initial Test Case Execution for Information Development E2E Testing

Test Infrastructure Management Processes

Objective:

E2E Testing is probably the first time that the Infrastructure Management environment will undergo any functional testing. Infrastructure Management testing will focus on ensuring the quality of the system platform, archiving, and backup and recovery processes that will be used to service the information management environment.

This will be of particular focus for larger information platform environments like Data Warehouses and will more complex in an environment that has limited downtimes. Backup and recovery processes must be thoroughly tested, and may be periodically tested post-deployment.

Input:

• E2E Test Environment ready
• E2E Software Management Test Plans and Test Cases
• Infrastructure Management Processes Developed and Unit Tested

Output:

• Completed Initial Test Case Execution for Infrastructure Management E2E Testing

Defect Resolution and Re-Testing

Objective:

This task covers re-execution of E2E test cases that had issues in initial round of E2E Testing. Due to dependencies between test cases, this may require regression testing of some test cases. Regression testing may also require some additional new test cases.

Input:

• Completed Initial Test Case Execution for BI Application Development E2E Testing
• Completed Initial Test Case Execution for Integration Development E2E Testing
• Completed Initial Test Case Execution for Information Development E2E Testing
• Completed Initial Test Case Execution for Infrastructure Management E2E Testing

Output:

• Completed Test Case Execution for BI Application Development E2E Testing
• Completed Test Case Execution for Integration Development E2E Testing
• Completed Test Case Execution for Information Development E2E Testing
• Completed Test Case Execution for Infrastructure Management E2E Testing

Depends of project metrics, but typical exit criteria out of E2E may include:

• 98% of performance test cases successfully completed

• No Severity 1 or 2 Defects outstanding including performance issues

• Agreed plans to resolve outstanding Severity 3 and 4 defects

These metrics should be agreed-upon as part of the overall Test Strategy.

Core Supporting Assets

Yellow Flags

• Defects that are functional and integrated-focused in nature mean that Functional Testing or SIT Testing was insufficient and it may have been inappropriate to progress to E2E Testing
• It is difficult to get a complete system flow-through without issues. Unlike earlier phases of testing, the backend integration should be stable at this point in the project
• User engagement is low or non-existent. At this point users should already be familiar with the system – the first time they see the functioning system should not be UAT

Key Resource Requirements

• Test Manager
• Functional Test Analysts
• Technical Test Analysts