Software Testing Techniques

CIS 375
Bruce R. Maxim
UM-Dearborn

1
 Testing Objectives
• Testing is the process of executing a
program with the intent of finding errors.
• A good test case is one with a high
probability of finding an as-yet
undiscovered error.
• A successful test is one that discovers
an as-yet-undiscovered error.

2
 Testing Principles
• All tests should be traceable to customer
requirements.
• Tests should be planned before testing begins.
• 80% of all errors are in 20% of the code.
• Testing should begin in the small and progress
to the large.
• Exhaustive testing is not possible.
• Testing should be conducted by an
independent third party if possible.

3
 Software Defect Causes
• Specification may be wrong.
• Specification may be a physical
impossibility.
• Faulty program design.
• Program may be incorrect.

4
 Types of Errors
• Algorithmic error.
• Computation & precision error.
• Documentation error.
• Capacity error or boundary error.
• Timing and coordination error.
• Throughput or performance error.
• Recovery error.
• Hardware & system software error.
• Standards & procedure errors.
5
Software Testability Checklist - 1
• Operability
– if it works better it can be tested more efficiently
• Observability
– what you see is what you test
• Controllability
– if software can be controlled better the it is more
that testing can be automated and optimized

6
Software Testability Checklist - 2
• Decomposability
– controlling the scope of testing allows problems to
be isolated quickly and retested intelligently
• Stability
– the fewer the changes, the fewer the disruptions to
testing
• Understandability
– the more information that is known, the smarter
the testing can be done

7
 Good Test Attributes
• A good test has a high probability of
finding an error.
• A good test is not redundant.
• A good test should be best of breed.
• A good test should not be too simple or
too complex.

8
 Test Strategies
• Black-box or behavioral testing
– knowing the specified function a product is
to perform and demonstrating correct
operation based solely on its specification
without regard for its internal logic
• White-box or glass-box testing
– knowing the internal workings of a product,
tests are performed to check the workings
of all possible logic paths
9
 Basis Path Testing
• White-box technique usually based on the program
flow graph
• The cyclomatic complexity of the program computed
from its flow graph using the formula V(G) = E – N +
2 or by counting the conditional statements in the
PDL representation and adding 1
• Determine the basis set of linearly independent paths
(the cardinality of this set is the program cyclomatic
complexity)
• Prepare test cases that will force the execution of
each path in the basis set.

10
 Control Structure Testing - 1
• White-box techniques focusing on control
structures present in the software
• Condition testing (e.g. branch testing)
– focuses on testing each decision statement in a
software module
– it is important to ensure coverage of all logical
combinations of data that may be processed by
the module (a truth table may be helpful)

11
 Control Structure Testing - 2
• Data flow testing
– selects test paths based according to the locations
of variable definitions and uses in the program
(e.g. definition use chains)
• Loop testing
– focuses on the validity of the program loop
constructs (i.e. while, for, go to)
– involves checking to ensure loops start and stop
when they are supposed to (unstructured loops
should be redesigned whenever possible)

12
 Graph-Based Testing - 1
• Black-box methods based on the nature of the
relationships (links) among the program objects
(nodes), test cases are designed to traverse the
entire graph
• Transaction flow testing
– nodes represent steps in some transaction and links
represent logical connections between steps that
need to be validated
• Finite state modeling
– nodes represent user observable states of the
software and links represent state transitions
13
 Graph-Based Testing - 2
• Data flow modeling
– nodes are data objects and links are
transformations of one data object to
another data object
• Timing modeling
– nodes are program objects and links are
sequential connections between these
objects
– link weights are required execution times
14
 Equivalence Partitioning
• Black-box technique that divides the
input domain into classes of data from
which test cases can be derived
• An ideal test case uncovers a class of
errors that might require many arbitrary
test cases to be executed before a
general error is observed

15
 Equivalence Class Guidelines
• If input condition specifies a range, one valid
and two invalid equivalence classes are defined
• If an input condition requires a specific value,
one valid and two invalid equivalence classes
are defined
• If an input condition specifies a member of a
set, one valid and one invalid equivalence class
is defined
• If an input condition is Boolean, one valid and
one invalid equivalence class is defined

16
 Boundary Value Analysis - 1
• Black-box technique
– focuses on the boundaries of the input domain
rather than its center
• Guidelines:
1. If input condition specifies a range bounded by
values a and b, test cases should include a and b,
values just above and just below a and b
2. If an input condition specifies and number of values,
test cases should be exercise the minimum and
maximum numbers, as well as values just above
and just below the minimum and maximum values

17
Boundary Value Analysis - 2
3. Apply guidelines 1 and 2 to output
conditions, test cases should be designed
to produce the minimum and maximum
output reports
4. If internal program data structures have
boundaries (e.g. size limitations), be
certain to test the boundaries

18
 Comparison Testing
• Black-box testing for safety critical systems in
which independently developed
implementations of redundant systems are
tested for conformance to specifications
• Often equivalence class partitioning is used to
develop a common set of test cases for each
implementation

19
 Orthogonal Array Testing - 1
• Black-box technique that enables the
design of a reasonably small set of test
cases that provide maximum test
coverage
• Focus is on categories of faulty logic
likely to be present in the software
component (without examining the
code)
20
 Orthogonal Array Testing - 2
• Priorities for assessing tests using an
orthogonal array
– Detect and isolate all single mode faults
– Detect all double mode faults
– Multimode faults

21