Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Wiki Home
Collapse Expand Close

Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Agile Quality Assurance

From MIKE2.0 Methodology

Jump to: navigation, search
Under review.png
This article is a stub. It is currently undergoing major changes as it is in the very early stages of development and is only a placeholder. Please help improve MIKE2.0 by adding to this article.


= Introduction

This article on Agile Quality Assurance will describe the process and the practices used by QA Analysts on agile development teams. QA Analysts typically, unlike in normal IT development project teams, are not measured against how many defects they raise and instead focus on improving the quality of the application by engaging very early in an interation's process, often teaming with analysts during the analysis phase.

With the Agile QA process it is the case that the QA Analysts become the most knowledgeable people of the system being developed and this is necessary as the only the most knowledgeable can provide the assurance or audit of the system before it goes live.

Since agile is an adaptive methodology, the practices and the process described in the article should be used as a good starting point to be later tuned, improved and evolved in any way that adheres to the agile values and improves team's productivity, collaboration and communication.

This article is a part of the Agile Business Transformation article series and as explained the ABT practices enable feedback and hence continuous improvement, and the Agile QA Analyst's should be very involved in these feedback practices so to give their input into how the QA team and the overall project team can improve and perform better - hence via the ABT practices such as retrospectives and stand-ups the Agile QA process is continuously adpated and improved. forfait sosh rio bouygues portabilité du numéro calcul IMC rio orange

Fundamental Differences between Agile QA and usual 'Software Testing'

  1. Agile QA involves testing during the development phase and not after.
  2. Agile QA is about testing acceptance of the system for the user - therefore the QA is responsible for the overall user experience of the application, including non-functional requirements and not just the specific test case scope derived from business requirements that are specified.
  3. Agile QA requires for the tester (in agile usually a QA Analyst or "QA" for short) to also be involved in business analysis by:
    • Being involved in conversations with the developers, business analysts and the agile customer to define requirements to a very detailed level during the iteration - there detailed requirements will be the specification against which he/she will test conformance against
    • Understanding the that the QA Analyst role is one of assurance - assuring that what is being developed not only fits the requirements that are explicit but also that what is being developed is right for the user - hence the QA Analyst is involved during analysis to also provide this assurance with a very assertive mindset
  4. Agile QA value added to the software project team is not measured in terms of defects/bugs detected but in terms of contribution to overall problem solving, analysis effort with an assertiuve (testing) mindset and to assurance of quality of the application for the end user - hence this is a qualitative measure
    • The QA Analyst's goal is therefore not a sub-team goal (or local suboptimisation) but the only goal of the teams which is to deliver a high quality application in time and on budge
    • When the QA Analyst's performance is measured by how many defects she finds then she will not look for methods and ways how to improve the development process so less defects are create as this will then undermine her performance

Agile QA Process


The diagram above shows the Agile QA Process in relation to the overall iteration cycle. One thing should stand out, the testing (Acceptance and Regression Testing) happens during development unlike in more waterfall, incremental or spiral development methodologies. While developers write the code to complete a user story, the testers write the acceptance tests to test the user story. When developers are finished the testers carry out the acceptance test on the story. Finally when all acceptance tests are completed the testers update the regression test suite (only a subset of all the end-to-end tests) if necessary and after they always execute the regression tests (which should be automated) hence to finish when the development finishes. Also notice the QA Analyst's involvement in Business Analysis for the following iteration.

Agile QA Practices

Acceptance Tests

Writing Acceptance Tests

Acceptance tests must:

  1. Branch out from the business's acceptance criteria of the user story
  2. Test both the happy and sad paths of the user story through the application
  3. Test the relevant unspecified functional and non-functional (usability, security and performance) aspects of the user story
  4. Be in an assertive format using: Given the Pre-Condtion, When the Action happens, Then the Post-Conidition should hold
  5. Test the end-to-end functionality of the acceptance critera

Automated vs. Manual

In author's opinion acceptance tests should always be executed manually at first as the QA Analyst is the best for testing the quality aspects of the user story which testing tools can't test. After the acceptance test are carried out manually at first and the QA Analyst identifies it as a core end-to-end test (e.g. publishing an article in a CMS) then this test should be automated and made part of the regression suite of tests. Hence all the acceptance tests that are automated become the regression suite of tests.

Regression Tests

From a quality assurance perspective, regression tests are integration tests that ensure the system is still doing what it used to do with the new functionality upgrade that came with the completion of the last iteration. They should be completely automated as they are of very repetitive nature.

Ideally all acceptance tests after an initial manual execution should be automated and become part of the regression test suite. However this ideal scenario is not usually done because:

  1. Of the substantial cost due to repetition of work - manual acceptance test execution and then writing the automated test and then executing it for the same functionality under test.
  2. Usually "big things fail" - to give an example it is more likely that the save function (writing to the database) will fail than that the input field no longer excepts up to 200 character but only 100.
    • Hence the saving function test should be automated and in the regression over the max character entry test.

Therefore good regression tests should:

  1. Provide end-to-end testing of the core functionality at minimum (see for example CMS User Acceptance Testing - section on Test Script Strategy as an idea of core functionality in a CMS)
  2. Run at the end of every iteration
  3. Be automated so that they can be executed fast
  4. Be optimized so that all regression testing finishes just after the development finished in a iteration

Business Analysis and Requirements Audit

The QA Analyst on the agile team is, just like the whole team, focused on the team's goal - to develop a high quality application. The business analysts analyse what it needs to do, developers question what it needs to do and then write the code to do that, the QA Analysts/testers then have to assure the business stakeholders that the application does what it needs to do but they also have to question what it needs to do and help define what it needs to do.

The justification for the QA Analysts involvement in analysis, as a requirements auditor, is because of the vast and usually superior knowledge they develop of the application to anyone else on the team. The QA Analyst is always, during testing, being reminded of what the application does, how it does it and why it does it - the QA Analyst is always during regression test and end-to-end acceptance tests being reminded of existing functionality, relationships and dependencies. The business analysts don't have this knowledge reinforcement and are always pushing forward for defining new requirements which they remember but don't revisit again unless things change around the existing functionality. Hence the QA Analyst is a requirements auditor, whose superior input of what the application does now in the analysis phase will help define the least costly approach to delivering new functionality and ensure the requirements are also being assertively viewed early.

Testing Automation Tools

There are two approaches to automated functional testing: graphical user interface testing or code-driven testing. The testing tool that uses GUI testing automates the GUI interaction and testing validation is done on the responses of the GUI. The testing tool that uses code-driven testing calls public methods from libraries of the software application being build and then validates the returning values of the methods against what they should be.

Selenium (GUI driven) and FIT (code and script driven?) are two widely used automated functional testing tools. There is also the IBM's Rational Robot which is a heavyweight but extremely exhaustive testing tool. For performance testing there is also JMeter.

Regression testing should involve both functional and performance automated testing as regression tests will be repeated many times, every iteration, so this process must be automated.


Wiki Contributors
Collapse Expand Close

View more contributors