Part Four
Verification, Validation
and Testing
This is concerned with the checking processes which are required
to ensure that software meets its specification and the user’s
requirements. It introduces verification and validation and covers the
overall testing process in a program.
Contents
1. Verification and Validation
2. Program Testing
3. Unit Testing
4. Integration Testing
5. System Testing
6. Test Tools
7. Test Team
Part Four Verification, Validation and Testing
Introduction to Software Engineering
Objectives:
To describe the process of program testing.
To introduce the different strategies in testing the system.
To have an understanding on defective software’s.
To explain how a program testing may be represented as guidelines in having good
quality software.
To propose some general design principles which should be followed by developers
responsible for testing the system.
Develop love and respect.
4.1 Verification and Validation
Verification and validation (V & V) is the generic name given to checking processes
which ensure that software conforms to its specification and meets the needs of the
software customer. The system should be verified and validated at each stage of the
software process using documents produced during the previous stage.
Verification and validation are sometimes confused. They are in fact, different
activities. The difference between them is briefly summarized:
Validation: Are we building the right product?
Verification: Are we building the product right?
Verification involves checking that the program conforms to its specification.
Validation involves checking that the program as implemented meets the expectations of
the software customer.
4.2 Program Testing
The testing process is described in terms of who performs the test, what steps are
involved, and what types of test can be used. We examine in depth the testing of individual
modules. After you, as programmer, have eliminated errors.
Defective Software
Software Error
o Types of Errors
Introduction to Software Engineering | For CICT Classroom Use Only
62
Introduction to Software Engineering
1. Algorithm Error. This is a type of error in which a program module’s
algorithm or logic does not produce the proper output for a given input
because something is wrong with the processing steps.
2. Syntax Error. It is needed to be sure that we have properly used the
constructs of the programming language. Sometimes the presence of a
seemingly trivial syntax error can lead to disastrous results.
3. Computation and Precision Errors. Occurs when the implementation of
a formula is wrong or does not compute the result to the required degree
of accuracy.
4. Documentation Errors. Occurs when the documentation does not match
what the program actually does.
5. Stress or Overload Errors. Occurs when these data structures are filled
past their specified capacity.
6. Capacity or Boundary Errors. Occurs when the performance of a system
becomes unacceptable as the activity on the system reaches its specified
limit.
7. Timing or Coordination Errors. Occurs when the code coordinating
these events is inadequate.
8. Throughput or Performance Errors. Occurs when the system does not
perform at the speed prescribe by the requirements.
9. Recovery Errors. Can occur when an error is encountered and the system
does not behave as the designers desire or as the computer requires.
[Link] and System Software Errors. Can arise when the
documentation for the supplied hardware and software does not match
their actual operating conditions and procedures.
[Link] and Procedures Errors. May not affect the running of the
programs, but they may foster an environment for the creation of errors
as the system is tested and modified. By failing to follow the prescribed
standards, one programmer may make it difficult for another to
understand the programming logic or to find the data descriptions
needed to solve a problem.
Purpose of Testing
We test a program in order to demonstrate the existence of an error. Because
our goal is to discover errors, we can consider a test successful only when an error
is discovered. Once is found, “debugging” or error correction is the process of
determining what causes the error and making changes to the system so that the
error no longer exists.
o Attitude towards testing
New programmers are not accustomed to viewing testing as a
discovery process. Many software engineers adopt an attitude known as
egoless programming, where program modules are viewed as the
Introduction to Software Engineering | For CICT Classroom Use Only
63
Introduction to Software Engineering
components of a larger system, not as property of those who wrote them.
When an error is found, the egoless development team is concerned whith
determining the cause of the error and correcting it, not with placing blame
on a particular developer.
o Who performs the test?
Even when a system is developed with an egoless approach. It is
sometimes difficult for us to remove our personal feelings from the testing
process. Thus, we often use an independent test team to test a system. In this
way, we avoid conflict between personal responsibility for errors and the
need to discover as many errors as possible.
Furthermore, an independent test team can participate in reviewing
the modules throughout development. The team can be part both the
requirements review and the design reviews. After having tested each
module individually, the team can continue to test as the system is integrated
and presented to the customer for acceptance.
o Stages of testing
Module or unit testing verifies that the modules functions properly
with the types of input expected from studying the module design. A unit
testing is done in a controlled environment whenever possible so that the
test can feed a predetermined set of data to the modules being tested and
observe what output data are produced.
Integration testing is the process of verifying that the components
of a system work together as describe in the program design and system
design specifications.
Function test evaluates the system to determine if the functions
describe by the requirements specification are actually performed by the
integrated system.
Performance test compares the system with the remainder of the
software and hardware requirements. If test is performed in the customer’s
actual working environment, a successful test yields a validated system.
Customer and system designer joint together to performed
acceptance testing, in which the system is checked against the customer’s
requirements description. When the acceptance testing is complete, the
accepted system is installed in the environment in which it will be used.
Introduction to Software Engineering | For CICT Classroom Use Only
64
Introduction to Software Engineering
Installation test is performed to make sure that the system will be
used as it should.
4.3 Unit Testing
Now we examine the problem of testing individual modules after they have been
coded. Given our goal of finding errors in the modules, how do we start? Unit testing of a
program module is done in the same way. First you and your peers examine the code for
errors. When you find no more errors in this manner, the module is compiled and run with
test data to search for other errors.
Program review
This is similar to the ones in which the system and program design
description are reviewed prior to implementation. A team, composed of you as the
programmer and three or four other technical experts, studies the program. The
technical experts can be other programmer, technical writers, designers, or project
supervisors. Whereas the design review teams included the customer’s
representatives, a program review team contains no one from the customer’s
organization.
Types of Program Review:
1. Program Walkthroughs. This kind of review where you present your code and
the accompanying documentation to the review team, and the team comments
on the correctness of the program. During a walkthrough, you lead the session
and control the discussion.
2. Program Inspection. This is similar to walkthrough. Whereas walkthrough is an
informal presentation to a review team, a program inspection is a formal review
in which the review team checks the program against a prepared list of concerns.
The team can review the algorithms and computation for correctness and
efficiency.
Testing programs
Proving programs correct is a goal to which software engineers aspire;
consequently, much related research is being done to develop method and
automated tools. However, in the near future, development teams are more likely to
be concerned with testing their software that with proving their programs correct.
4.4 Integration Testing
When we are satisfied that individual program module are working correctly and
meet out established objectives, we combine the modules into a working system. This
integration is planned and coordinated so that when an error occurs, we have some idea of
what caused it.
Bottom-up approach
Introduction to Software Engineering | For CICT Classroom Use Only
65
Introduction to Software Engineering
One popular approach for merging modules to test the larger system is called
bottom-up testing. When this method is used, each module at the lowest level of the
system hierarchy is tested individually. Then, the next modules to be tested are
those that call the previously tested modules. This approach is followed repeatedly
until all modules are included in the testing. This approach is useful when many of
the low-level modules are general-purpose utility routines that are invoked often by
others or when the design is object oriented.
Test E
Test B, E, F
Test F
Test A, B, C, D,
Test C E, F, G
Test G Test D, G
Top-down approach
Many developers prefer to test using a top-down approach. In this way, top
down is the reverse of bottom-up. The top level, usually one controlling module, is
tested by itself. Then, all modules called by tested module(s) are combined and
tested as a larger unit. This approach is reapplied until all modules are incorporated.
others or when the design is object oriented.
Test A, B, C, D,
Test A Test A, B, C, D E, F, G
Introduction to Software Engineering | For CICT Classroom Use Only
66
Introduction to Software Engineering
Modified top-down testing
This approach tests each level’s modules individually before the merger
takes place.
Test B Test E
Test A, B, C, D,
Test A Test C Test A, B, Test F E, F, G
C, D
Test D Test G
Big-bang testing
When all modules are tested in isolation, it is tempting to mix all of them
together as the final system and see if it works the first time.
Test A
Test B
Test C Test A, B, C, D,
E, F, G
Test D
Test E
Test F
Test G
Sandwich testing
This combines a top-down strategy with a bottom-up one to form a sandwich
testing approach. The system is view as having three layers, just sandwich does; a
target in the middle, the levels above the target, and the levels below the target.
Introduction to Software Engineering | For CICT Classroom Use Only
67
Introduction to Software Engineering
Test E
Test B, E, F
Test F
Test A, B, C, D,
Test G Test D, G E, F, G
Test A
Modified-sandwich testing
This allows upper-level modules to be tested individually before merging
them with others for testing.
Test E
Test B, E, F
Test F
Test B
Test D Test A, B, C, D,
E, F, G
Test G Test D, G
Test A
4.5 System Testing
Testing can be viewed as a two-step process. First, we must be sure that the
implementation works as the programmers’ process. Then, when the modules are
combined into a working system, we insure that the system functions according to the
prescriptions of the requirements document.
System testing verifies that a system solves the problem as defined by the
requirements documents.
Introduction to Software Engineering | For CICT Classroom Use Only
68
Introduction to Software Engineering
Steps in system testing
1. Function testing. Effective function test should have a high probability of
detecting an error. Function testing is performed in carefully controlled
situations. It compares the actual performance of the system with its
requirements, so the test case for function testing is developed from the
requirement document.
2. Performance testing. This examines how well the calculation is done; response
to user commands, accuracy of the result, and accessibility of the data are
checked against the customers’ performance prescription.
Test coverage:
o Stress test. Evaluates the system when stressed to its limits over a short
period of time.
o Volume tests. Address the handling of large amounts of data in the system.
o Configuration test. Analyzed the various software and hardware
configurations specified in the requirements. A configuration test evaluates
all possible configurations to make sure that each satisfies the requirements.
o Compatibility tests. This are needed when a system interface with other
systems. We find out whether the interface functions performs according to
the requirements.
o Regression tests. This are required when the system being tested is
replacing an existing system. The regression test guarantees that the new
system’s performance is at least as good as that of the old.
o Security test. Insures that the security requirements are met.
o Timing test. Evaluates the requirements dealing with the time to respond to
a user and time to perform a function.
o Environment test. This look at the system’s ability to perform at the
installation site.
o Quality test. Evaluates the reliability, maintainability, and availability fo the
system.
o Recovery test. Address response to the presence of errors or to the loss of
data, device, or power.
o Maintenance test. Address to the need for diagnostic tools and procedures
to help finding the source of errors.
o Documentation test. Insures that we written the required document. Thus, if
user’s manual, maintenance guides, and technical documents are needed, we
verify that these materials exist and that the information they contain is
consistent, accurate, and easy to read.
o Human factor tests. Investigate requirements dealing with the user
interface to the system.
Introduction to Software Engineering | For CICT Classroom Use Only
69
Introduction to Software Engineering
3. Acceptance testing. When function and performance tests are complete, we are
convinced that the system meets all requirements specified during the initial
stages of the system development. Acceptance test is done to make sure that the
customer and users concur to the results.
Types of Acceptance Test
o Benchmark test. This is performed by the actual users or a special test team
exercising system functions. In either case, the testers are familiar with the
requirements and able to evaluate the actual performance.
o Pilot test. It installs the system on experimental basis. Users exercise the
system as if it had been permanently installed. In-house test is called alpha
test and the customer’s pilot test a beta test.
o Parallel test. If a new system is replacing an existing one or is part of a
phased development, this test can be used for acceptance. The new system
operates in parallel with the previous version. This testing incorporates a
user-administered combination of compatibility and function testing.
4. Installation testing. The final round of testing involves installing the system at
user sites. If acceptance testing has been performed on site, installation may not
be needed. However, if acceptance testing conditions were not the same as
actual site condition, additional testing is necessary.
4.6 Test Tools
Several tools are available to help us administer the variety of test involved in
system testing. Most are automated and capture data that can be of use in evaluating a
system’s performance.
Simulator. This allows us to concentrate on evaluating one part of a system while
portraying the characteristics of other parts. This is particularly useful is a special
device is located on site but testing is being done at another location.
Monitor. A device that captures data passing from one device or process to another.
We can use a monitor to save a “snapshot” of the condition before and after an error
occur.
Analyzer. It goes one step beyond a monitor; it does not only capture data but also
evaluates the according to some prescribe criteria.
4.7 Test Team
We have primary responsibility for function and performance testing, while the
customer plays a large role in acceptance and installation tests. However, the test team for
all tests is drawn from both staffs, as shown in Figure 4.7.1.
Introduction to Software Engineering | For CICT Classroom Use Only
70
Introduction to Software Engineering
No programmer from this project are involved is system testing; they are too
familiar with structure and intention of the implementation, and the may have difficulty
recognizing the differences between implementation and required function.
Profesional
Testers
Users Analysts
Test Team
Configuration
System
Management
Designers
Specialists
Figure 4.7.1
Members of the Test Team
Professional testers. They organize and run the tests. They are involved from the
beginning designing test plans and test cases as the project progresses. They also
work with the configuration management team to provide documentation and other
mechanism for tying test to the requirements, design modules, and implementation
modules.
Analysts. If they were involved in the original requirements definition and
specification, they are useful in testing because they are familiar with the problem
as defined by the customer. They also have good feeling for the needs and goals of
the customer.
System designers. In the test team, they add perspective of intent. They designers
understand what we proposed as a solution to the customer’s problem; they know
the constraints imposed on the solution by the overall design. The designer also
know how the system is divided into functional subsystems and understand how the
system is supposed to work.
Configuration management specialists. As error are discovered and changes
requested, they arranges for the changes to be reflected in the document,
Figure 4.7.1 requirements, design, implementation, or anything else affected by the change.
Test team membersUsers. They are the best qualified to evaluate issues dealing with appropriateness of
audience, ease of use, and other human factors. User of the proposed system are
essential if they were not present when the system requirements were first defined
by the customer. A user is likely to be intimately familiar with customer’s problem
Introduction to Software Engineering | For CICT Classroom Use Only
71
Introduction to Software Engineering
because of daily exposed to it, and can be invaluable in evaluating the system to
verify that it solves the problem.
EVALUATION NO. 04
1. Discuss the difference between verification and validation and explain why
validation is a particularly difficult process.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
2. What kind of performance test might be required for a word processing system? For
an automatic bank teller system? And for an office payroll system? Explain your
answers.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Introduction to Software Engineering | For CICT Classroom Use Only
72
Introduction to Software Engineering
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
3. Measurements of reliability, maintainability, and availability are often placed in the
system requirements document. Give examples of systems that might be able to
tolerate low levels of these qualities. Give examples of system that can tolerate only
high levels of these qualities.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Introduction to Software Engineering | For CICT Classroom Use Only
73
Introduction to Software Engineering
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Introduction to Software Engineering | For CICT Classroom Use Only
74
