Software Testing is a way to assess quality of software while reducing the risk of failure. The testing process includes planning, analysis, design, implementation and reporting. Software Testing has a single purpose of evaluating the software under test. According to the International Software Testing Qualifications Board, there are seven principles in software testing:
- Testing shows the presence of defects, not their absence
- Exhaustive testing is impossible
- Early testing saves time and money
- Defects cluster together
- Beware of the pesticide paradox
- Testing is context dependent
- Absence of errors is a fallacy
The following examines the value each of the software testing principles bring to the lean software development life cycle (SDLC).
Testing shows the presence of defects, not their absence
Like a beautiful work of art, software is imperfect. While testing reduces the presence of defects, it cannot show that there are zero defects. However, testing can greatly reduce the number of undiscovered defects.
Exhaustive testing is impossible
A simple Rubik’s cube has 43 quintillion different combinations. Testing all combinations of preconditions, inputs, outputs, validation, success and error messaging of a system of software components is not feasible. To create the testing plan, the tester will conduct an analysis of the application using testing techniques and prioritization methods. This will focus test efforts to ensure the software meets the minimum agreed upon criteria for quality.
Early testing saves time and money
Static and dynamic test strategies should be deployed early and often. The goal is to catch issues as early as possible in the software development life cycle to reduce cost. Professionals in the industry often refer to this strategy as shifting testing left. Static testing will include white box testing techniques, while dynamic testing employs both white box and black box testing techniques.
Defects cluster together
Software code contains individual methods and functions and components, or modules. A small number of modules typically contain the majority of the defects. These high-defect modules are most often responsible for operational failures. Identifying the fragile modules is an important to in conducting a risk analysis. This risk analysis can be used to focus the testing effort, thereby reducing risk.
Beware of the pesticide paradox
When farmers use the same pesticide year after year, the pesticide loses it’s effectiveness. As a result, each year the same pesticide is used, fewer and fewer of the pests are killed. For this reason, farmers cycle through different pesticide formulas. Likewise, when the same tests are used repeatedly during software testing, they no longer find new defects. Sometimes this has a good outcome, such as running an automated regression suite. However, this can also give a false sense of security since the defects have likely shifted. To detect new defects the software tester should modify existing tests, test data, and add new tests.
Testing is context dependent
A software tester’s approach to testing is different given the known conditions. Safety critical software is tested differently than a game app because the risk is different. The risk of failure maybe life or death critical in a medical device, while a defect in a game app will cause an inconvenience to the user. Likewise, testing in an agile development environment is different than testing in a waterfall process. The agile process strives to derive fast feedback (testing takes place earlier in the SLDC), while the waterfall process may strive for a more predictable time line.
Absence of errors is a fallacy
Thoroughly testing all of the requirements and fixing all of the defects discovered does not mean the application will meet the needs of its users. Nor will this process ensure that the application is superior to competitor’s software. Software testers strive to verify the software has the expected outcomes. Testers also check to see the app reacts properly when users do unintended actions. However, it is the product owner (or project manager or business analyst) who should understand the end users. They should then pass along that information to the technical team, which would result in a user-friendly application that does the things the user needs the software to do.
