Delivery & Operations metrics at the team level (typically measure every Iteration).
Metrics | Definition | Comments | RAGs | Source | Scope |
---|---|---|---|---|---|
Test Coverage | The extent to which test cases exercise the software under test. Formula = (Number of Covered Requirements(stories) / Total Number of Requirements) * 100 |
|
GREEN>90%.
70% <= AMBER <= 90%
RED < 70% |
Jira + QTest |
MVP |
Test Case Effectiveness | The measure of how well the test cases can identify defects. Formula = (Number of Defects Found / Number of Test Cases Executed) * 100 |
|
GREEN>90%
70% <= AMBER <= 90%
RED < 70% |
Jira + QTest | MVP |
% Unit Test Coverage | Percentage of a source code covered by unit tests |
|
GREEN > 80%
AMBER > 40%
RED <= 40% |
Sonar | MVP |
Test Automation Coverage | The percentage of test cases that are automated out of the total test cases. Formula = (Number of Automated Test Cases / Total Number of Test Cases) * 100 |
Number of automated tests in QTest / number of total tests in QTest at the end of each iteration OR over time |
GREEN > 90%
70% <= AMBER <= 90%
RED < 70% |
QTest | MVP |
Test Environment Availability | The availability and readiness of test environments for testing activities. Formula = (Total Time Test Environment Available / Total Time Required for Testing) * 100 |
-------- |
GREEN > 99%
95% <= AMBER <= 99%
RED < 95% |
TBD | MVP+ |
First time to right (FTR) | First Time Right shows a percentage of resolved issues without returns. | The same as for Defect Reopen Rate, but for all issue types |
GREEN < 10%
10% >= AMBER >= 20%
RED > 20% |
Jira | MVP+ |