The Test requirement analysis guidelines provide an approach to perform a detailed analysis of the requirements delivered. This document will provide an overview of requirements testing, including definitions for different ambiguities and how to capture requirements defects.
Benefit of Requirement analysis:
The QE Team will benefit from the detailed analysis of requirements to
Limit Scope Creep
Increase Requirements Traceability to Test Cases
Reduce Defect Leakage to Production
Improves Estimation Accuracy
Increases velocity of Test Case creation
Overview of performing Requirement analysis:
The QE team shall engage in the requirements walkthrough given by the Product Owners/ Business Analysts. QA Specialist (Functional SME) participates in the elicitation to ensure clear, concise, measurable conditions. By providing measurable requirements, QA Specialist (Functional SME) ensures the requirements are testable.
Respective architect teams give technical spec walkthroughs, and the QE team clarifies all open questions.
Technical requirements provide the solutions for the requirements defined by the business. Functional requirements are termed the “what,” and technical requirements are termed the “how.”
In the elicitation sessions, the Product Owners/ Business Analysts focus on capturing requirements to define “what” actions will be automated in the system. Technical specifications define how the system will be modified to address the needs of business users and analysts.
As part of PI planning, the entire team participates, estimates the features and commits to the PI. QE team reviews the Features and their acceptance criteria and ensures they are testable.
Based on the features committed in PI, the Scrum QE team creates user stories for testing the features. When a User story is completed, the QE team analyzes the user stories and their acceptance criteria for any ambiguity, or it contains any poor requirement etc., to meet the definition of done without any ambiguities. The condition refers to the user story and its acceptance criteria in the below sections. Analysis practices are mentioned below.
Performing requirement analysis
When analyzing the requirement, QA Specialist (Functional SME) shall review the content of the requirement to determine if the requirement is clear, concise and measurable. The following provides an overview of the questions applied to requirement analysis.
Does the requirement state what triggers the action performed in the requirement?
Does the requirement have an expected outcome?
Does the requirement identify the user performing the action?
Does the requirement state the permissions required to perform the operation?
Are the boundaries of the requirement clear? For instance, if a report is run weekly, does the requirement state what day the report is run?
Are the business rules stated that determine whether a condition is true?
Does the requirement address error conditions?
Are error messages given?
Does the requirement address alternate flows?
Does the requirement address exceptions?
| Condition | Symptoms |
|---|---|
Ambiguous Logical Operations |
A condition is when a requirement uses specific words to indicate logical relationships between elements in a sentence.
|
Ambiguous References |
Condition when requirement uses words that can be interpreted differently based on the reader’s view, such as frequently, efficiently, monthly, periodically, etc. |
Boundary Ambiguity |
A condition when the author uses terms - among or up to. The scope of the requirement is ambiguous because the stated requirement can be interpreted in multiple ways.
|
Built-In Assumptions (Tribal Knowledge) |
A condition when the author assumes that all consumers of the document will have the same level of domain knowledge
|
Etc. |
Etcetera is not a quantifiable measurement that can be confirmed, so it is considered ambiguous. (i.e., Sentences ending with etc.) |
Omission |
A condition of an incomplete requirement
The requirement omits important information leaving readers to interpret the behaviour. |
Scope of Action |
A condition when the requirement is not clear and cannot be quantified |
Temporal Ambiguity |
A condition when the author uses terms - at the appropriate time or fast. The scope of the requirement is ambiguous because the stated requirement can be interpreted in multiple ways.
|
User Stories:
Define user stories
- A user story is a low-level requirement detail containing just enough information so that the scrum team can reasonably estimate the effort to implement it
Description of user stories
- Card – A written definition of the story (Title, definition section of the story)
- Conversation – Discussions about the story that provides the details of the story
- Confirmation – Tests that convey the details which can be used to determine the completion of the story (Acceptance criteria)
Writing user stories – Each user story should focus on six attributes
- Independent
- Negotiable
- Valuable to Users
- Estimatable
- Small
- Testable
Once you identify a user story, care should be taken that it contains all these six attributes. User stories are not written commitments. Details of the story are always discussed between the developer and the user. The story card should contain the notes of such discussions and acceptance tests to be carried out for the story.
Detailed Requirements specify the interaction between the user and the system. Detailed requirements can be written as User stories. The detailed requirements define the fundamental actions between the user and system in accepting, processing, and generating the outputs.
Detailed Documents
- Gathered and documented during the Requirements Phase and described
- What the user can do with the system – this is the interaction between the user and the system
- What the system shall do – the functionality that should change
- Describes the outcome of all scenarios to accomplish the business goal described
- The requirement describes the action performed by the system
- The Happy Path or normal flow describes the ideal path for performing a function
- Alternate flows describe business constraints, special circumstances and error conditions which fail to reach the button goal
- Dependencies on other applications for data to process new or modified transactions
- Data from an application is required to be transmitted to another application
- Each requirement must be traceable to a high-level requirement
Question Samples
- Does the happy path user requirement detail the workflow and expected outcome?
- Are user requirements created for alternate flows with the applicable workflow and expected outcome?
- Are user requirements for error conditions with the applicable workflow and expected outcome provided?
- Are valid inputs stated where required?
- Is the sequence of operations or processing steps provided?
- Does the requirement include exceptions such as responses to abnormal situations, error handling and recovery?
- Is the effect of parameter changing considered?
- Are valid output(s) stated?
- Is the relationship of outputs to inputs stated, including
- Input/ output sequences
- Formulas for input-to-output conversion
- ‘As Is’ and ‘To be’ Process Flow diagrams
- Error messages & handling of error messages
User stories (Non-Functional)
- Gathered and documented during the Requirements Phase and may include
- Guidelines for system performance
- Guidelines for usage, transaction volumes and SLA windows
- Guidelines for security
- Guidelines for disaster recovery, business continuity, etc.
- Describes the expected performance of the system under specific conditions
Question Samples- Non-functional user stories
- What are the initial estimated users of the application?
- What is the expected growth rate (in terms of users) of the application annually?
- What is the number of users accessing the site per day, peak hours, off-peak hours, Monday morning, Friday evening (or any specific day, depending on the nature of the site)?
- What is an acceptable response time for a web-enabled application that includes back-end processing?
- What is an acceptable response time for processing handled by the internal enterprise router, firewall and web server?
- What is an acceptable response time when calling an external system?
- What are peak and non-peak periods daily, weekly, monthly and annually?
- What are the transaction volumes and number of users define peak processing time?
- What are the transaction volumes and the number of users defining non-peak processing time?
- Will the users perform complex queries during peak periods?
- Will the users perform complex calculations during peak periods?
Business Rule
A business rule is a law, policy, standard or procedure by which an organization functions. It is a statement that defines or constrains some aspect of the business.
The business rules are analyzed in conjunction with user requirements, as the business rules provide parameters for the user requirement.
The determination for “measurable” will be whether the business rule provides boundaries for user requirements.
Constraint – A constraint is a limitation or restriction placed on the choices available to the project team for the design and development of the system.
The analysis for constraints is done in conjunction with user requirements, as the constraints provide parameters for the user requirement.
The determination for “measurable” will be whether the constraint provides boundaries for user requirements.
Data Requirement – A data requirement is information the system or user requests/provides to satisfy an interface or functional requirement.
Data Requirement Examples
- Membership Effective Date must be equal to or greater than 01/01/2023.
- The system will protect against unauthorized addition, deletion or modification of data
The Quality Engineers will examine the data requirements
- Data Element Name
- Data Type (e.g. alphabetic, alpha-numeric, or numeric)
- Data Definition
- Data Format
- Range of values or discrete values
- Unit of measurement
- Data capture method (manual entry, another application)
- Data source (if another application already exists)
- Data conversion rule (if applicable, in cases where data needs to be converted to another unit or format)
- Data requirements must be specified in case of a new or modified function
Reporting Requirement
- Report Name
- Brief Description
- Report format (sample template)
- Data fields
- Report generation variables (from date – to date, data element 1, data element 2, …… to be used to generate the report)
Additional tips for performing analysis
These apply to both Waterfall and Agile methodologies
How to find ambiguities – Review the user stories and acceptance criteria for ambiguities as mentioned below
Ambiguities in language result in incomplete requirements. The following list provides indicators.
Incomplete lists ending with etc., and/or, and TBD
Ambiguous words and phrases, such as generally, usually, to the greatest extent, and where practicable
Imprecise verbs such as supported, handled, processed or rejected
Implied certainty, such as always, never, all or every
Passive voice, such as the counter is set (By whom?)
Every pronoun, particularly it or it should have an explicit and unmistakable reference
Comparatives, such as earliest, latest, and highest. Words ending in “est” or “er” should be suspect.
Words whose meanings are subject to different interpretations between the business customer and IT analyst, such as instantaneous, simultaneous, achievable, complete, finish, degraded, a minimum number of, nominal/normal/average, peak/minimum/steady state, as required/specified/indicated, Coincidental/adjacent/synchronous with
In cases where a term used in a particular context can have multiple meanings, the term should be included in a glossary of the requirement document, where its meaning is made more specific.
Consistent Requirements – Quality Engineers and the team review the user stories to ensure no inconsistency. Below are the inconsistent sample types
Internal Inconsistencies Types
Conflicting terms: Two terms are used in different contexts to mean the same thing. Example: The term ‘prompt’ denotes a message to have a user input data used in one requirement, while the term ‘cue’ is used in another requirement to mean the same thing.
Conflicting characteristics: Two requirements in a BRD or feature/ user story state the software should exhibit contradictory attributes. Example: one provision states that all inputs will be via a menu interface, while another requirement says that all inputs shall be via a command prompt.
Temporal or logical inconsistency: Two parts of the BRD or user story may specify conflicting timing characteristics or logic. Example: One requirement may state that operation B can be performed only when operation A has been completed. In contrast, another requirement may conflict by saying that Operation B can be performed independently of Operation A.
Redundancy/ Duplicate Requirements – The Test Lead shall examine the user stories to find any redundancy or duplication in requirements. Redundancy itself is not an error but has the potential to lead to mistakes. For Example: When a user story containing acceptance criteria is updated, there is a possibility that one of the redundant components is updated. This automatically leads to inconsistency in requirements.
Prioritized Requirements – Quality Engineers shall verify user stories are assigned priority based on business criticality. If a focus is not offered an elicitation, Test Lead must examine Jira Backlog to determine whether all requirements have been given priority.
Measuring of requirements:
Requirements are deemed measurable when they are SMART
Specific requirements are precise
- The requirement has the appropriate level of detail and states precisely what is required
- The definition of the requirement must result in only one interpretation, no matter who is reviewing it
- Conjunctions (and, or, but) are avoided to prevent misinterpretation
Measurable requirements can be verified as complete
- The behaviour described in the requirement (in the User story) provides an expected outcome (Through acceptance criteria)
- Ambiguous adjectives/adverbs have been removed to remove a subjective interpretation of the requirement
- Test scenarios can be derived from the requirement
Attainable requirements are actionable
- The requirement is not gold-plated or too grandiose to be fulfilled
- The requirement can be achieved given the current environments
Realistic requirements are appropriate about other requirements
- Can the requirement be fulfilled given the existing systems?
- Can the requirement be satisfied within the project budget and resource constraints?
Time-Bound (Traceable) requirements have time parameters
- The requirement provides a specific time when the requirement needs to be completed or executed
- The requirement avoids ambiguous references such as quickly, fast or immediately
Projects that follow Agile and iterative model
- QE Team participates in requirement walkthrough and Technical specifications walkthrough given by the Business team, clarifies the questions related to requirements and identifies Testable requirements.
- QE team part of the scrum team participates in Sprint grooming to understand the user stories written and be clear by analyzing with points mentioned above (Ambiguity, inconsistency etc.)
- QE team will create Test scenarios against requirements/ features and review them with the business for coverage
- QE Team will create Test cases, review them with the business and upload them in qTest
QA Specialist (Functional SME) identify the dependencies on the projects that are not aligned in the same sprint or follow the waterfall model, create a timeline of end-to-end completion and track.
Track Requirements defects – If any observations or defects are identified during the requirements walkthrough by the business, those observations and defects need to be raised in Jira with Requirements as phase or root cause.
