Who: Quality Engineer
When: Smoke testing is typically performed after the deployment of the application into any environment. Smoke Testing is performed to verify deployed build with minimal critical flows (High-level functionality) to ensure the application is stable enough for further development or testing and should not exceed 15 mins for execution. Automating smoke tests and integrating them with the Continuous testing pipeline for Medium velocity programs is recommended.
Tools/Technology:
UTAF
- A framework that provides a standardized approach to automate and execute smoke tests.
qTest
- A test management tool that facilitates the organization and tracking of smoke test cases and results.
Primary Purposes of Smoke Testing:
1. Ensure Application Functionality:
- Smoke testing validates that the critical functionality of the application is working as expected. It verifies if the application launches, critical links open upon clicking, and the basic navigation flow is functional within data availability.
2. Early Identification of Defects:
- : Smoke testing helps identify defects in business-critical functionality at an early testing stage. By executing a minimal set of tests, testers can quickly identify any issues or errors that may impede further testing activities.
3. Stability Assessment:
- Smoke testing ensures that the deployed build is stable for testing before handing it off to the Quality Engineering (QE) team or User Acceptance Testing (UAT) in their respective environments. It provides a quick evaluation of the application's stability and readiness for further testing.
How To Plan for Smoke Test Automation
To plan for smoke test automation, the Agile program's Squad teams should create an automation coverage matrix that includes smoke testing. If automated smoke tests are not already in place, the following steps can be followed:
Identify Manual Smoke Test Flows: Identify the manual smoke test flows for the application under the project.
Collaborate with the QE Chapter Team: Collaborate with the QE Chapter team to obtain the required automation framework and the latest version.
Estimate for Automation Development: Estimate the effort and resources required for developing the automation scripts for smoke tests.
Obtain Approvals: Get approvals for funding and resources required for the automation development.
Create Automation Scripts: Create automation scripts for the identified smoke test suite.
Incorporate in Agile Practices: Plan the automation development as part of PI (Program Increment) planning and create user stories for automation development.
Maintain and Execute Smoke Tests: Maintain the automation scripts as part of ongoing sprints and execute smoke tests manually for every deployment until the smoke tests are fully automated. However, it is recommended for Agile projects have automated smoke tests.
Smoke test cases can be selected based on various scenarios, including
| Scenarios | Scenario Examples |
|---|---|
Addressing critical business functionality/workflows |
|
End-to-end scenarios handling critical business functions
|
|
Check navigation through selected screens. |
|
If the smoke test fails, the following steps depend on the specific scenario and the severity of the failures.
| Smoke Test Results | Next Steps |
|---|---|
The smoke Test fails, and the application fails when trying to access it |
The entire code turnover is rejected |
The smoke Test fails, but some components in the application are operational. |
The code turnover is partially rejected in the areas of failure. Test cases will be executed for available functionality. |
The smoke Test passes, but test case execution identifies a series of Severity 1-Critical defects that impede test execution. |
Code turnover will be rejected for areas of functionality that failed |
The smoke Test passes, but test execution identifies numerous weaknesses of all severity levels that impede test execution |
Code turnover will be rejected for areas of functionality that failed |
Environment Health Check
The environmental health check is one of the basic tests that must be performed daily in each environment. Environment health checks validate the connectivity points at the UI, Service and DB layer of each application (Prioritized) hosted in the environment. The environment health check is automated, maintained by the QE Chapter team and executed daily. The output of this report will contain app health based on connectivity checks at all layers and share the results with all stakeholders daily.
Environment health checks are scheduled to run daily morning through the Jenkins scheduler and share the results automatically. Once the results are shared, respective owners review and act upon the issues before the testing starts so there are no delays because of any app down.
Note: Until automated, the Smoke Test and Environment Health checks will be performed manually.
