Purpose – The purpose of this document is to provide the standard test estimation process for waterfall projects
Maintenance of the document – Review it at least once annually and update it. QE Chapter team is a custodian of this document and is responsible for maintaining in centralized document management tool and sharing it with Spoke teams. QE Chapter team to receive project requirements (If any specific) and feedback periodically to review and update the document as required
Objective – The objective of test estimation is to have a predictable measure of the test estimation with increased accuracy. Test estimation is performed based on the below factors.
Number of test cases & complexity
Influencing factors of the project
Test design and Execution productivity
Description
Test estimation process defined in TCP for the waterfall model
Test Estimation Guidelines – Waterfall Methodology
Functional Test Estimation
Test case point (TCP) analysis is an approach for accurately estimating effort for functional testing. This approach emphasizes key testing factors to determine the effort required for the entire testing life cycle. The TCP effort estimation is divided into four stages.
Determine Test case points (TCPs)
Categorize influencing factors and apply adjustments
Consolidate the elements for testing and assign productivity to determine efforts across testing phases
Calculate overall test effort in the project
Determine Test Case Points
To determine Test Case Points:
Identify the test cases from the test requirements. For detailed test effort estimation, the BRD/TRD is the basis for arriving at the final test requirements.
Classify individual test cases into the following categories based on complexity.
Simple
Medium
Complex
| Complexity Factors | Definition |
|---|---|
No. of Transactions |
Request to or response from a database or any storage location |
Confirmation Steps |
Dependency of a test case on another test case (s). There may be cases where a series of steps/test cases must be performed before executing the test case. These steps/ test cases would directly impact the test cases. |
No. of verification points/ Steps in the test case |
The count of logical steps in a test case with an expected output to be verified. |
Establishing baseline data |
If the outcome of a test case has a critical dependency on the input test data, then the input test data is considered baseline test data. |
The following guidelines need to be applied for categorizing the test cases.
| Complexity Type | Number of Transactions | Confirmation Steps | Number of Verification Points | Baseline Test Data |
|---|---|---|---|---|
Simple |
<3 transactions |
0 |
<3 |
Not Required |
Medium |
3-6 transactions |
<3 |
3-8 |
Required |
Complex |
>6 transactions |
>3 |
>8 |
Required |
The complexity of any test case is determined by taking the highest order of any one of the parameters identified for that Test Case or scenario.
Example: If a test case has more than eight verification points but only one transaction, the test case is considered complex.
3. Determine the total number of simple, medium and complex test cases
4. Apply the adjustment weights as follows
Simple: 2
Medium: 4
Complex: 8
5. Calculate the Test Case Points (TCPs) as follows
| Parameter | Formula |
|---|---|
TCP |
(Number of simple test cases x 2) + (Number of medium test cases x 4) + (Number of complex test cases x 8) Where 2, 4, and 8 are the adjustment factors for simple, average, and complex Test Case types |
Categorize Influencing Factors and Apply Adjustments –
Factors other than complexity also influence the total effort required to generate and execute the test cases. Example: The test cases for which the business knowledge of Premium Rate Mechanism is needed may take more time/effort than test cases designed to verify a screen. To consider other influences, 16 adjustment factors have been identified to adjust the TCP and improve the estimate’s accuracy.
Note all the factors may not apply to all projects. For example, technical know-how cannot be a factor for all projects without technological changes. If any factor does not apply to any project, please assign ‘0 (zero)’ weight and DO NOT remove any factor.
The 16 adjustment factors are as follows:
F1 - Domain Knowledge & Complexity
F2 - Technical Know How
F3 - Integration with other Hardware devices such as Hand-held Devices, Scanners, Printers
F4 - Multilingual Support
F5 - Software/ Hardware Set-Up
F6 - Environment Set-Up
F7 - Build Management
F8 - Configuration Management
F9 - Preparation of Test Bed
F10 - Stable Requirements
F11 - Offshore/ Onsite Coordination
F12 - Test Data Preparation
F13 - Network Latency
F14 – Operating System Combinations
F15 – Browser Combinations
F16 – Execution Productivity
The various steps in this stage are as follows:
Calculate Adjustment for Factors F1 to F13
For each adjustment factor, “factor weight” and “complexity weight” needs to be assigned.
The Factor weight denotes the complexity of the factor concerning the project. This value will vary from 0 to 10 only.
The complexity weight considers the QE’s expertise and the complexity we foresee in the project due to this factor. This value will vary from 0 to 3 only.
Example: If the same IVR system were coming in for a new release testing (the prior release having been tested by us), the Technical Know-how complexity would be assigned 1 or 2.
After assigning the factor weight and complexity weight for each factor, calculate the Individual Adjustment factor.
Individual Adjustment Factor= (Factor Weight * Complexity Weight) / (Maximum Factor Weight * Maximum Complexity Weight)
Example - Assume a factor weight of 10 and complexity weight of 1 for the parameter’ Domain Knowledge & Complexity. The adjusted weight is calculated as 10 *1 /30 = .33.
The denominator 30 denotes the product of ‘maximum factor weight’ and ‘maximum complexity weight’; here, the maximum skill set weight is 3, and the maximum relevance weight is 10, hence 30.
Calculate Adjustment for Factors F14 to F15
For factors F14 and F15, a quantified count of operating systems and browser combinations is determined for the application. The count for F14 and F15 are multiplied to determine the adjustment value. This adjustment value will influence the effort for manual test execution.
Estimate the Effort
Estimate the effort for manual test case design.
The formula = Test Case Points determine the overall Estimate for manual test case design Count * Total Adjustment Factors impacting test design* Test Design Productivity (Person hours per TCP)
While computing the Test Case Points Count, modularization and reusability techniques used in Test Case Design should be considered.
The total Adjustment Factor impacting test design equals (1 + average of relevant individual adjustment factors influencing manual test case generation).
Test Design Productivity indicates the effort in hours required to create one Test Case Point.
Estimate the effort for manual test case execution
The formula = Test Case Points determine the overall estimate for manual test case execution Count * The Total Adjustment Factors impacting test execution * Test Execution Productivity (Person hours per TCP)
The total Adjustment Factor affecting test execution equals (1 + average of relevant individual adjustment factors influencing manual test case execution).
Test Execution Productivity indicates the effort in hours required to execute one Test Case Point.
Calculate Overall Test Effort in the Project
The Total Testing Effort = (Effort Estimate for Test Initiation + Effort Estimate for Test Planning + Effort Estimate for Test Case Design + Effort Estimate for Test automation (if planned) + Effort Estimate for Test Execution (manual as well as automation, if planned and multiple cycles of testing) + Effort estimate for Test Closure + Project Management effort + any additional buffer efforts for contingency.
The overall testing effort needs to be used to arrive at resource loading for this effort based on the testing schedule.
L1 Level Test Effort Estimation
During the initial stages of the SDLC, the testing team may have to share the L1-level test effort estimates for planning or budgeting. The BRD will be the basis for the L1 Level test estimation. But, the full information will likely not be available at that stage to perform reasonable test estimation. So, the testing team should consider and/or seek the following parameters.
Clarity on requirements, as much as possible
Changes to Architecture, as much as feasible
Magnitude/Complexity of the change
Technical complexity
Functional complexity
Test Environment
Availability
The scale of the environment, like end-to-end, based on the complexity of changes
Development methodology
Waterfall methodology
Iterative methodology
The TCP effort estimation template can also be used for this high-level estimate. The guidelines for using the TCP effort estimation template are in the following table per the TCP estimation stages mentioned above.
| Stages in TCP effort Estimation Process | Customization Required for High-Level Estimates |
|---|---|
Determine Test Case Points (TCPs) |
|
Categorize influencing various factors and apply adjustments |
Only some adjustment factors can be considered for effort estimation
|
Consolidate factors for testing and assign productivity to determine effort across testing phases. |
No changes are required at this stage |
Calculate overall test effort in the project |
No modifications are needed at this stage |