Home | Updates | Work | Play | Photos | Contact | About

Software Quality Assurance


Home > Work > Software Quality Assurance

When focus is quality, quality rises and costs fall
When focus is cost, costs rise and quality falls
William Edwards Deming

This post introduces some high-level quality objectives common to most IT projects. It defines quality, and presents common quality objectives (Wiegers, 2003) as user stories. These can (and should) be used by project teams to derive specific quality requirements applicable to their projects.

Introduction to Quality

The International Organisation for Standardisation (specifically ISO 9000:2015) defines quality as the degree to which a set of inherent characteristics fulfils requirements. In this definition the word degree implies that quality exists on a continuum – starting at zero and moving toward, perhaps, infinity. This inference is ambiguous in that the quantifications of “good” and “bad” are not qualified.

Another popular definition of quality, as defined by Joseph Juran, is fitness for use, with fitness and use being crucial to the understanding of quality. Consumer and provider interpretations of these two terms often conflict, and so cannot be left to interpretation. Organisations often bound these two terms with specifications for their products and services.

Specifications may be explicit or implicit. Explicit specifications are prepared by the provider, and are given to the consumer. Implicit specifications are not defined, but are understood to be necessary. Examples include safety, security and fault tolerance requirements.

Quality Assurance User Stories


As a   user
I want the system to be at least n1 percent available between [start time] and [end time] on [days of week] and at least n2 percent available on [days of week] local time
So thatI can complete my assigned tasks during normal working hours, and occasionally use the system out of hours.
Measure:% Availability = MTTF / (MTTF + MTTR) * 100
Where MTTF is mean time to failure, and
MTTR is mean time to repair


As a   User
I wantat least n percent of the processor capacity and RAM available to the application to be unused during planned peak load conditions
So thatunanticipated demand spikes and future growth do not impair acceptable performance
MeasureHow well a system utilizes processor capacity, storage space, memory, or communication bandwidth (Davis, 1993).


As a   business
I wantto introduce x feature into the system, including code , modifications and testing, with no more than n hours of labor.
So thatthe system can evolve through a series of successive releases or by evolutionary prototyping
Measure:How easy it is to add new capabilities to a product, to change existing capabilities, or to remove redundant or obsolete capabilities.


As a   business, user or administrator
I wantto block unauthorized access to system functions, prevent information loss, and protect the system from tampering
So thatI can protect the interests of [business], and the privacy and safety of [entities].
Measure:Integrity is a constraining business requirement, and as such has no tolerance for error. Ensure that a threat model exists, and includes mitigations to reduce probability, and contingencies to reduce impact.


As a   user or administrator
I wantto submit data to, and access data from [location(s), external system(s) and/or service(s)], and import and export data in [format(s)]
So thatI can easily exchange data and services with other systems.
Measure:How easy it is to share or access data or services with or from other systems.


As a   user
I wantn percent of operations to complete correctly within x [time unit], and for the system to operate for a period of y [time unit] before failing
So thatBAU is maintained, and the impact of system failures is minimized.
Measure:The period of time that software executes without failure (Musa, Iannino, and Okumoto, 1987); the percentage of operations that complete correctly.


As a   user
I wantthe system to continue to function properly when confronted with invalid inputs, defects in connected software or hardware components, or unexpected operating conditions
So thatthe system can recover gracefully from problem situations and is forgiving of user mistakes.
Measure:The degree to which the system continues to function with confronted with invalid inputs, defects in connected software or hardware components, or unexpected operating conditions.


As a   user
I wantto spend x amount of effort in preparing input for, operating and interpreting the output of the system
So thatfunctions are easy to complete, and complete within acceptable timeframes.
Measure:The effort required to prepare input for, operate and interpret the output of the system (Constantine and Lockwood, 1999).

Developer Stories


As a   developer
I wantan average time of n1 [time unit] to fix a problem, and to fix n2 percentage of problems correctly
So thatI am encouraged to take actions to the intent, and not the letter, of the goal.
Measure:The average time required to fix a problem, and the percentage of fixes that are made correctly.


As a      developer
I wantto select design and coding approaches that enhance the system’s portability
So thatthe system isn’t tied to a single technology stack or vendor’s products.
Measure:The effort required to migrate a piece of software from one operating environment to another.


As a   developer
I wantto develop modular, well-documented software that is independent of a specific application and operating environment, and somewhat generic in capability
So thatthe relative effort required to convert a software component for use in other applications is minimal.
Measure:Reusability is difficult to quantify. Identify which components are suitable to use in other applications (typically components that address cross-cutting concerns such as monitoring, logging, communication, and authentication/authorisation), and estimate the relative effort required to convert those components for use in other applications.


As a   developer
I wantto write software modules in which the maximum cyclomatic complexity does not exceed n
So thatthe number of logic branches and loops are kept within a range that makes them easier to test, to understand and to maintain.
Measure:Count cyclomatic complexity. Do not measure automated test coverage (remember that the objective is quality, and not coverage).


< Back | ^ Back to top

All content copyright © Michael Wittenburg 1995 to 2021. All rights reserved.
Merch (t-shirts designed by my twin)