Content

Introduction & Programming

SOFTWARE TESTING with Testing, Verification and Validation
ESP8266 Test Strategies for Conventional and OO types
Unit Testing
Integration Testing
Validation Testing
SystemTesting
Security Testing, Stress Testing, Performance Testing
Metrics for Source Code, Metrics for Testing
Assistant Professor
Kusum Lata Dhiman Debugging Process, Debugging Strategies
Computer Science & Engineering

Testing, Verification and Validation Testing, Verification and Validation

Testing is the practise of running a programme with the goal of discoveri Verification is the software system or component evaluation process to
ng mistakes. determine.
The existence of bugs but never their absence can be show by research. whether the products of a given development phase satisfy conditions
The design of a programme to discover its faults. imposed at the beginning of that phase
Method of proving the correctness of the programmes is verification. Testing is a series of procedures conducted to test any feature of a piece
of software.
Validation is the method of discovering mistake by running software in
Testing is a tool used to reveal software defects and to assess that the
a specific world.
software has achieved a specified degree of quality with regard to
Debugging is diagnosing the bug and fixing it.
selected attributes.

Difference between Verification & Validation Debugging

We check whether we are We check whether the Debugging, or localization of faults, is the method of
developing the right product developed product is right. (1) The fault or flaw is located,
or not. Validation is also known (2) code upgrade
Verification is also known as dynamic testing. (3) the code is re-tested
as static testing. It is a process of checking the Errors: An error on the part of a software developer is a mistake,
It is a process of checking the software during or at the end of misconception, or misunderstanding.
work-products whether the the development cycle to decide Faults: As a consequence of an mistake, a fault (defect) is inserted into
product meets the specified whether the software follow the the programme.
requirements. specified business requirements. It is a software phenomenon that can cause it to behave inappropriately,
Quality assurance comes Quality control comes under not as per its specification.
under verification testing. validation testing. Failures: A failure is the inability of a software system or component
Verification is done before After verification testing, under specified performance specifications to perform its necessary
the validation testing. validation testing takes place. functions.

Test Case Test or Series of test

In a practical sense, a test case is a test-related item which contains the A test is a collection of associated test cases or a set of associated test
following data:
cases and test procedures.
1. A selection of inputs for experiments. Test Suite: Group of associated tests that are database-related and
There are data items that the code under test has obtained from an
external source. typically run together is often referred to as test suite
Hardware , software, or humans may be the external source. Oracle test: A test oracle is a document or piece of software that helps
2. Conditions for execution. These are the conditions needed to run the
test, such as a certain database state, or a hardware system config. testers to evaluate whether or not a test has been passed.
3. Expected outputs. These are the results stated to be provided by the Test Bed: A test bed is an area containing all the hardware and software
code being evaluated.
used to test a part of the software or software system.
Performance of Software Software Testing Principle
Quality relates to the degree to which specified specifications are met A definition can be described as:
by a system , system part, or process. 1. A theory, or inference, general or fundamental.

Quality relates to the extent to which a system,part of the system or 2. A behavioural law or code.
process meets customer need or specifications 3. The rules or facts of nature that underlie an part of the system or
Metric: A metric is a quantitative measure of the extent to which a given process that meets customer need or specifications
attribute is possessed by a system , system part, or method. The Principles are as follows:
Service metric: scale of software, LOC 1. Testing is the method of using a selected set of test cases to exercise a
Metric of process: prices, time software component with the goal of (I) exposing defects and (ii)
Quality metric: A quality metric is a quantitative measure of the degree assessing consistency.(Testing shows presence of Defect)
to which a given quality attribute is present in an object. 2. If the test target is to detect defects, then a successful test case is one
that has a high degree of defects. probability of revealing a yet
undetected defect(s). (Defect Clustering)

Software Testing Principle Test Strategies for Conventional and OO Software

3. Test findings should be meticulously inspected.(Absence of Error)
Conventional research techniques for applications are-testing unit testing.
4. The predicted performance or outcome must be contained in a test
case. Integration checks, device tests and tests for validation are used.
5. For both valid and invalid input conditions, test cases should be
The first one focuses on any feature of software that is implemented in
created.(Exhaustive testing is impossible)
6. A community that is independent of the production group should carry the source code.
out the research.
The second one based on the design of software architecture.
7. Tests have to be reusable and repeatable.(Pesticide Paradox)
8. It should be prepared for early testing.(Early testing) The third one checks as a whole the other system and applications.
9. In the software life cycle, testing activities should be incorporated.
The validation of specifications is specified by the fourth one against the
10. A innovative and demanding job is research.(Testing is context
Dependent) built software.
Objective methods' input and output are not basis of testing.

Unit Testing Advantages of Unit Testing

It is a testing technique using which individual modules are tested.
It is concerned with functional correctness of the standalone modules.. Reduces Defects in the Newly developed features or reduces bugs when
To focus on a relatively small segment of code and aim to exercise a high changing the existing functionality.
percentage of the internal path. Reduces Cost of Testing as defects are captured in very early phase.
The main aim is to isolate each unit of the system to identify, analyze and Improves design and allows better refactoring of code.
fix the defects. Unit Tests, when integrated with build gives the quality of the build as
It is used to find out the defect at the early stage of the testing. well.
It is normally performed by software developers themselves or their Codes are more reusable. In order to make unit testing possible.
peers. Unit testing increases confidence in changing / maintaining code.

Integration Testing Bottom-up Integration Test

Integration testing is the second level of the software testing process It is a strategy in which the lower level modules are tested first.
These tested modules are then further used to facilitate the testing of
comes after unit testing. higher level modules.
units or individual components of the software are tested in a group. The process continues until all modules at top level are tested.
Once the lower level modules are tested and integrated, then the next
The focus of the integration testing level is to expose defects at the time level of modules are formed.
of interaction between integrated components or units. Allow early testing aimed at proving feasibility
Emphasize on module functionality and performance
Once all the components or modules are working independently, then we Advantages: No test are needed
need to check the data flow between the dependent modules. Errors in critical modules are found early
Disadvantages: Test drivers are needed
It consist of two types Top down and Bottom up Integration testing. Interface errors are discovered late
Drivers and Stubs Example
Stubs and Drivers are the dummy programs in Integration testing used to Let us see one sample example of a banking application, as we can see in
facilitate the software testing activity.
These programs act as a substitutes for the missing models in the testing. the below image of amount transfer.
They do not implement the entire programming logic of the software First, we will login as a user P to amount
module. transfer and send Rs200 amount, the
But they simulate data communication with the calling module while confirmation message should be displayed on
testing. the screen as amount transfer successfully.
Stub: Is called by the Module under Test. Now logout as P and login as user Q and go to
Driver: Calls the Module to be tested. amount balance page and check for a balance
in that account = Present balance + Received
Balance. So the integration test is successful.
Also, we check if the amount of balance has
reduced by Rs200 in P user account. [Link]

Validation Testing Validation Testing

As a tester, you need to evaluate if the test execution results comply
Validation testing is testing where tester performed functional and non- with that mentioned in the requirements document.
Any deviation should be reported immediately and that deviation is
functional testing. thus called a bug.
Functional testing includes Unit Testing (UT), Integration Testing (IT) and Tools like HP quality Centre, Selenium, Appium, etc are used to perform
validation test.
System Testing (ST). we can store the test results there.
Non-functional testing includes User acceptance testing (UAT). A proper test plan, test execution runs, defect reports, reports &
metrics are the important deliverables to be submitted.
Validation testing is also known as dynamic testing, where we are
ensuring that "we have developed the product right."
It also checks that the software meets the business needs of the client.

Advantage of Validation Testing Alpha Testing

During verification if some defects are missed then during validation It is done when product development is on the verge of completion.
process it can be caught as failures.
If during verification some specification is misunderstood and Slight changes in design can be made after conducting the alpha test.
development had happened then during validation process. This testing methodology is performed in lab surroundings by the
while executing that functionality the difference between the actual
result and expected result can be understood. developers.
Validation is done during testing like unit testing, integration testing, Here developers see things in the software from users point and try to
system testing, load testing, compatibility testing, stress testing, etc.
Validation helps in building the right product as per the customer s detect the problems.
requirement and helps in satisfying their needs. These testers are internal company or organization's employees or may
be a part of the testing team.
It is done early at the end of software development before beta testing.

How to do Alpha Testing Advantage of Alpha Testing

Firstly need to first review the design specification and functional One of the benefits of alpha testing is it reduces the delivery time of the
project.
requirements. It provides a complete test plan and test cases.
Then develop a comprehensive test plan and test cases. Free the team member for another project.
Every feedback helps to improve software quality.
After that execute the test cases in order to find log defects and fix those It provides a better observation of the software's reliability and
defects. accountability.

Finally retest once the issues are solved for smooth functioning of
software.
Beta Testing Features of Beta Testing
Beta Testing can be defined as the second stage of testing any product Beta testing used in a real environment at the user's site.
Beta testing helps in providing the actual position of the quality.
before release. Testing performed by the client, stakeholder, and end-user.
A sample of the released product with minimum features and Beta testing always is done after the alpha testing, and before releasing
it into the market.
characteristics. Beta testing is black-box testing.
It is being given to the intended audience for trying out or temporarily Beta testing performs in the absence of tester and the presence of real
users
using the product. Beta testing generally is done for testing software products like utilities,
The beta test is being carried out by real users in the real environment. operating systems, and applications, etc.

This allows the targeted customers to dive into the product's design,
working, interface, functionality, etc.

Advantage of Beta Testing Disadvantage of Beta Testing

Beta testing offers numerous benefits to testers, software developer, as Software engineer has no control over the process of the testing, as the
well as the users. users in the real-world environment perform it.
In the assistance of this type of testing, it enables developers, testers to
test the product before its release in the market. This testing can be a time-consuming process and can delay the final
The Beta testing focuses on the customer's satisfaction. release of the product.
It helps to reduce the risk of product failure via user validations. Beta testing does not test the functionality of the software in depth as
Beta testing helps to get direct feedback from users. software still in development.
It helps to detect the defect and issues in the system, which is It is a waste of time and money to work on the feedback of the users
overlooked and undetected by the team of software testers. who do not use the software themselves properly.
Beta testing helps the user to install, test, and send feedback regarding
the developed software.

System Testing System Testing

System Testing is a type of software testing that is performed on a System Testing is carried out on the whole system in the context of either
complete integrated system system requirement specifications.
It is used to evaluate the compliance of the system with the It also checks for functional requirement specifications or in the context of
corresponding requirements. both.
In this testing, integration testing passed components are taken as input. System testing tests the design and behaviour of the system and also the
System testing detects defects within both the integrated units and the expectations of the customer.
whole system. System Testing is basically performed by a testing team that is
The result of system testing is the observed behaviour of a component or independent of the development team.
a system when it is tested. It has both functional and non-functional testing.

System Testing Example System Testing Example

A car manufacturer does not produce the car as a whole car. Each The whole car needs to be checked for different aspects as per the
component of the car is manufactured separately, like seats, steering, requirements defined like if car can be driven smoothly, breaks, gears,
mirror, break, cable, engine, car frame, wheels etc. and other functionality working properly, car does not show any sign of
After manufacturing each item, it is tested independently whether it is tiredness after being driven for 2500 miles continuously, color of car is
working the way it is supposed to work and that is called Unit testing. generally accepted and liked, car can be driven on any kind of roads like
Now, when each part is assembled with another part, that assembled smooth and rough, sloppy and straight, etc and this whole effort of
combination is checked if assembling has not produced any side effect testing is called System Testing and it has nothing to do with integration
to the functionality of each component and whether both components testing.
are working together as expected and that is called integration testing. The example worked the way it was expected and the client was
Once all the parts are assembled and the car is ready, it is not ready convinced about the efforts required for the system test.
actually.
Types of Testing System Testing Process
Test Environment Setup:
Create testing environment for the better quality testing.
Create Test Case:
Generate test case for the testing process.
Create Test Data:
Generate the data that is to be tested.
Execute Test Case:
After the generation of the test case & test data, test cases are executed.
Defect Reporting:
Defects in the system are detected.
Log Defects:
Defects are fixed in this step.
Retest:If the test is not successful then again test is performed.
[Link]

Recovery Testing Recovery Testing

Recovery testing a type of system testing which aims at testing whether a A system or software should be recovery tested for failures like :
Power supply failure
system can recover from failures or not. The external server is unreachable
The technique involves failing the system and then verifying that the Wireless network signal loss
Physical conditions
system recovery is performed properly. The external device not responding
To ensure that a system is fault-tolerant and can recover well from The external device is not responding as expected, etc.

failures, recovery testing is important to perform.

A system is expected to recover from faults and resume its work within a
pre-specified time period.
Recovery testing is essential for any mission-critical system.

Security Testing Security Testing

Security testing is a type of software testing that intends to uncover Integrity
vulnerabilities of the system. Authentication.
It determine that its data and resources are protected from possible Authorization.
intruders. Availability
It falls under non functional testing. Non-repudiation.
Security testing is a testing technique to determine if an information
system protects data and maintains functionality as intended.
It also aims at verifying 6 basic principles such as:
Confidentiality

Security Testing Stress Testing

There are four main focus areas to be considered in security testing Stress Testing is a type of software testing that verifies stability &
(Especially for web sites/applications):
reliability of software application.
Network security: This involves looking for vulnerabilities in the network The goal of Stress testing is measuring software on its robustness and
infrastructure.
System software security: This involves assessing weaknesses in the error handling capabilities under extremely heavy load conditions.
various software the application depends on. It ensures that software doesn't crash under crunch situations.
Client-side application security: This deals with ensuring that the client
cannot be manipulated. It even tests beyond normal operating points and evaluates how software
Server-side application security: This involves making sure that the works under extreme conditions.
server code and its technologies are robust enough to fend off any
intrusion. Stress testing is to determine the limit, at which the system or software
or hardware breaks.
Stress Testing Stress Testing
It also checks whether the Stress testing is also extremely valuable for the following reasons:
system demonstrates effective To check whether the system works under abnormal conditions.
error management under
extreme conditions. Displaying appropriate error message when the system is under stress.
The application under testing System failure under extreme conditions could result in enormous
will be stressed when 5GB data
is copied from the website and revenue loss.
pasted in notepad. It is better to be prepared for extreme conditions by executing Stress
Notepad is under stress and
gives 'Not Responded' error. Testing.
Eg:During festival time, an
online shopping site may
witness a spike in traffic. [Link]

Performance Testing Types of Performance Testing

Performance Testing is a software testing process used for testing the Load testing - checks the application's ability to perform under
anticipated user loads. The objective is to identify performance
speed, response time. bottlenecks before the software application goes live.
Stability, reliability, scalability and resource usage of a software Stress testing - involves testing an application under extreme workloads
to see how it handles high traffic or data processing. The objective is to
application under particular workload. identify the breaking point of an application.
The main purpose of performance testing is to identify and eliminate the Endurance testing - is done to make sure the software can handle the
expected load over a long period of time.
performance bottlenecks in the software application. Spike testing - Tests the software's reaction to sudden large spikes in the
uncovers what needs to be improved before the product goes to market. load generated by users.
Volume testing - Under Volume Testing large no. of. Data is populated in
Poor performance testing are likely to gain a bad reputation and fail to a database and the overall software system's behaviour is monitored.
meet expected sales goals. Scalability testing -To determine the software application's effectiveness
in "scaling up" to support an increase in user load

Metrics for Source Code Metrics for Source Code

A measure provides a quantitative indication of the extent, amount, These are measurements of the source code that make up all your
dimension, capacity or size of some attributes of a product or process . software.
Measurement is the act of determining a measure. Source code is the fundamental building block of which your software is
The metric is a quantitative measure of the degree to which a product or made.
process possesses a given attribute. Measuring it is key to making sure your code is high-caliber.
example, a measure is the number of failures experienced during testing. The number of lines of code must be looked while measuring source code
Measurement is the way of recording such failures quality.
A software metric may be an average number of failures experienced per Equally important is to track the percentage of comments within the
hour during testing. code, which will tell you how much maintenance the program will require.

Development Metrics Software Testing Metrics

These metrics measure the custom software development process itself. Software Testing Metrics are the quantitative measures used to estimate
Gather development metrics to look for ways to make your operations the progress, quality, productivity and health of the software testing
more efficient and reduce incidents of software errors. process.
Measuring number of defects within the code and time to fix them tells The goal of software testing metrics is to improve the efficiency and
you a lot about the development process itself effectiveness in the software testing process.
Start by tallying up the number of defects that appear in the code and To help make better decisions for further testing process by providing
note the time it takes to fix them. reliable data about the testing process.
If any defects have to be fixed multiple time then there might be a A Metric defines in quantitative terms the degree to which a system,
misunderstanding of requirements or a skills gap. system component, or process possesses a given attribute.
 Debugging Strategies Debugging Process
The important technique to find and remove the number of errors or The steps involved in this process are:
bugs or defects in a program is called Debugging. Identifying the error
It involves identifying the bug, finding the source of the bug and Identifying the error location
correcting the problem to make the program error-free. The developer can locate the code error in the program and remove it
This process should be done before releasing them into the market. using this process.
The developer can locate the code error in the program and remove it Retest the code to find if its get repeated. i.e the error that have
using this process. encountered.
It plays a vital role in the entire software development lifecycle. It plays a vital role in the entire software development lifecycle.

Refrences
[1]. Roger Pressman, Software engineering- A practitioner s Approach, McGraw-
Hill International Editions
[2]. Yogesh Singh, Software Testing, Cambridge University Press.
[3]. William E. Perry, Effective Methods for Software Testing, Second Edition, John
Wiley & Sons
[4]. Ron Paton, Software Testing, second edition, Pearson education.
[5] Dohorthy Graham foundation of Software Testing,ISTQB Certification
[6] Softwaretesting. Tutorialspoint.
[Link] testing