TYPES OF TESTING

Why System Testing is important?

System Testing is a crucial step in Quality Management Process.

……..- In the Software Development Life cycle System Testing is the first level where
………..the System is tested as a whole
……..- The System is tested to verify if it meets the functional and technical
………..requirements
……..- The application/System is tested in an environment that closely resembles the
………..production environment where the application will be finally deployed
……..- The System Testing enables us to test, verify and validate both the Business
………..requirements as well as the Application Architecture

Prerequisites for System Testing:

The prerequisites for System Testing are:
……..- All the components should have been successfully Unit Tested
……..- All the components should have been successfully integrated and Integration
……….Testing should be completed
……..- An Environment closely resembling the production environment should be
………..created.

When necessary, several iterations of System Testing are done in multiple environments.

Steps needed to do System Testing:

The following steps are important to perform System Testing:
……..Step 1: Create a System Test Plan
……..Step 2: Create Test Cases
……..Step 3: Carefully Build Data used as Input for System Testing
……..Step 3: If applicable create scripts to
………………- Build environment and
………………- to automate Execution of test cases
……..Step 4: Execute the test cases
……..Step 5: Fix the bugs if any and re test the code
……..Step 6: Repeat the test cycle as necessary

What is a ‘System Test Plan’?

As you may have read in the other articles in the testing series, this document typically describes the following:
………- The Testing Goals
………- The key areas to be focused on while testing
………- The Testing Deliverables
………- How the tests will be carried out
………- The list of things to be Tested
………- Roles and Responsibilities
………- Prerequisites to begin Testing
………- Test Environment
………- Assumptions
………- What to do after a test is successfully carried out
………- What to do if test fails
………- Glossary

How to write a System Test Case?

A Test Case describes exactly how the test should be carried out.

The System test cases help us verify and validate the system.
The System Test Cases are written such that:
……..- They cover all the use cases and scenarios
……..- The Test cases validate the technical Requirements and Specifications
……..- The Test cases verify if the application/System meet the Business & Functional
………..Requirements specified
……..- The Test cases may also verify if the System meets the performance standards

Since a dedicated test team may execute the test cases it is necessary that System Test Cases. The detailed Test cases help the test executioners do the testing as specified without any ambiguity.

The format of the System Test Cases may be like all other Test cases as illustrated below:

· Test Case ID

· Test Case Description:

o What to Test?

o How to Test?

· Input Data

· Expected Result

· Actual Result

Sample Test Case Format:

Test Case ID	What To Test?	How to Test?	Input Data	Expected Result	Actual Result	Pass/Fail
.	.	.	.	.	.	.

Additionally the following information may also be captured:
……..a) Test Suite Name
……..b) Tested By
……..c) Date
……..d) Test Iteration (The Test Cases may be executed one or more times)

Working towards Effective Systems Testing:

There are various factors that affect success of System Testing:

1) Test Coverage: System Testing will be effective only to the extent of the coverage of Test Cases. What is Test coverage? Adequate Test coverage implies the scenarios covered by the test cases are sufficient. The Test cases should “cover” all scenarios, use cases, Business Requirements, Technical Requirements, and Performance Requirements. The test cases should enable us to verify and validate that the system/application meets the project goals and specifications.

2) Defect Tracking: The defects found during the process of testing should be tracked. Subsequent iterations of test cases verify if the defects have been fixed.

3) Test Execution: The Test cases should be executed in the manner specified. Failure to do so results in improper Test Results.

4) Build Process Automation: A Lot of errors occur due to an improper build. ‘Build’ is a compilation of the various components that make the application deployed in the appropriate environment. The Test results will not be accurate if the application is not ‘built’ correctly or if the environment is not set up as specified. Automating this process may help reduce manual errors.

5) Test Automation: Automating the Test process could help us in many ways:

a. The test can be repeated with fewer errors of omission or oversight

b. Some scenarios can be simulated if the tests are automated for instance
simulating a large number of users or simulating increasing large amounts
of input/output data

6) Documentation: Proper Documentation helps keep track of Tests executed. It also helps create a knowledge base for current and future projects. Appropriate metrics/Statistics can be captured to validate or verify the efficiency of the technical design /architecture.

Summary:

In this article we studied the necessity of ‘System Testing’ and how it is done.

What is Regression Testing?

Introduction:

This article attempts to take a close look at the process and techniques in Regression Testing.

What is Regression Testing?

If a piece of Software is modified for any reason testing needs to be done to ensure that it works as specified and that it has not negatively impacted any functionality that it offered previously. This is known as Regression Testing.

Regression Testing attempts to verify:

- That the application works as specified even after the changes/additions/modification were made to it

- The original functionality continues to work as specified even after changes/additions/modification to the software application

- The changes/additions/modification to the software application have not introduced any new bugs

When is Regression Testing necessary?

Regression Testing plays an important role in any Scenario where a change has been made to a previously tested software code. Regression Testing is hence an important aspect in various Software Methodologies where software changes enhancements occur frequently.

Any Software Development Project is invariably faced with requests for changing Design, code, features or all of them.

Some Development Methodologies embrace change.

For example ‘Extreme Programming’ Methodology advocates applying small incremental changes to the system based on the end user feedback.

Each change implies more Regression Testing needs to be done to ensure that the System meets the Project Goals.

Why is Regression Testing important?

Any Software change can cause existing functionality to break.
Changes to a Software component could impact dependent Components.

It is commonly observed that a Software fix could cause other bugs.

All this affects the quality and reliability of the system. Hence Regression Testing, since it aims to verify all this, is very important.

Making Regression Testing Cost Effective:

Every time a change occurs one or more of the following scenarios may occur:
- More Functionality may be added to the system
- More complexity may be added to the system
- New bugs may be introduced
- New vulnerabilities may be introduced in the system
- System may tend to become more and more fragile with each change

After the change the new functionality may have to be tested along with all the original functionality.

With each change Regression Testing could become more and more costly.

To make the Regression Testing Cost Effective and yet ensure good coverage one or more of the following techniques may be applied:

- Test Automation: If the Test cases are automated the test cases may be executed using scripts after each change is introduced in the system. The execution of test cases in this way helps eliminate oversight, human errors,. It may also result in faster and cheaper execution of Test cases. However there is cost involved in building the scripts.

- Selective Testing: Some Teams choose execute the test cases selectively. They do not execute all the Test Cases during the Regression Testing. They test only what they decide is relevant. This helps reduce the Testing Time and Effort.

Regression Testing – What to Test?

Since Regression Testing tends to verify the software application after a change has been made everything that may be impacted by the change should be tested during Regression Testing. Generally the following areas are covered during Regression Testing:

- Any functionality that was addressed by the change

- Original Functionality of the system

- Performance of the System after the change was introduced

Regression Testing – How to Test?

Like any other Testing Regression Testing Needs proper planning.
For an Effective Regression Testing to be done the following ingredients are necessary:

- Create a Regression Test Plan: Test Plan identified Focus Areas, Strategy, Test Entry and Exit Criteria. It can also outline Testing Prerequisites, Responsibilities, etc.

- Create Test Cases: Test Cases that cover all the necessary areas are important. They describe what to Test, Steps needed to test, Inputs and Expected Outputs. Test Cases used for Regression Testing should specifically cover the functionality addressed by the change and all components affected by the change. The Regression Test case may also include the testing of the performance of the components and the application after the change(s) were done.

- Defect Tracking: As in all other Testing Levels and Types It is important Defects are tracked systematically, otherwise it undermines the Testing Effort.

Summary:

In this article we studied the importance of ‘Regression Testing’, its role and how it is done.

Challenges in Testing Web Based Applications

In this tutorial you will learn about Challenges in Testing Web Based Applications Introduction, Why testing Web Applications is different? Factors effecting Testing of Web Applications, Why technology platforms affect testing? Challenges in Testing Web Based Web Applications, Summary.

Introduction:

Web based Applications are increasingly becoming more feature rich, important and also the most popular means for developing commercial systems. Most companies opt for developing web based software wherever possible. This helps in catering to large number of end users. The deployment of the apps (once the infrastructure is in place) is fairly easy.

The web based applications are powerful and have the ability to provide feature rich content to a wide audience spread across the globe at an economical cost.

Hence it is a daunting task to test these applications and with more and more features testing these apps is becoming even more complex.
In this article we will study the challenges faced when testing these applications

Why testing Web Applications is different?

Testing web applications is different because of many factors scenarios affecting the performance and user experience.
Web applications can typically cater to a large and a diverse audience. Web Applications can also be exposed to wide range of security threats. Web applications may open up illegal points of entry to the databases and other systems holding sensitive information.

To ensure that the web application works reliably and correctly under different situations these factors need to be accounted for and tested.

Hence a lot of effort needs to put in for Test Planning and Test Design.
Test Cases should be written covering the different scenarios not only of functional usage but also technical considerations such as Network speeds, Screen Resolution, etc.

For example an application may work fine on Broad Band internet users but may perform miserably for users with dial up internet connections.
Web Applications are known to give errors on slow networks, whereas they perform well on high speed connections.
Web pages don’t render correctly for certain situations but work okay with others.

Images may take longer to download for slower networks and the end user perception of the application may not be good.

Factors effecting Testing of Web Applications:

As mentioned above Web Applications can have a lot of variables affecting them such as:

- Numerous Application Usage (Entry – Exit) Paths are possible
Due to the design and nature of the web applications it is possible that different users follow different application usage paths.

For example in an online banking application a user may directly go to “Bill Pay” page and other users may check account balances, view previous transactions and then “Pay the Bills”.
Generally a large number of usage paths are possible and all are supposed to work well.
All these Permutations and Combinations need to be tested thoroughly

- People with varying backgrounds & technical skills may use the application
Not all applications are self explanatory to all people. People have varying backgrounds and may find the application hard to use. For instance a Business Intelligence application with “Drill-Down-Reports” may work out for certain users but not for others.

Although this affects the design of the applications, but these factors should be tested in usability testing of the applications

- Intranet versus Internet based Applications
Intranet based applications generally cater to a controlled audience. The developers and architects can make accurate assumptions about the people accessing the apps and the hardware/software/technical specifications of the client machines.

While it may be difficult to make similar assumptions for Internet Based Applications
Also the intranet users can generally access the app from ‘trusted’ sources whereas for internet applications the users may need to be authenticated and the security measures may have to be much more stringent.

Test Cases need to be designed to test the various scenarios and risks involved.

- The end users may use different types of browsers to access the app
Typically for internet based applications users may have different Browsers when accessing the apps. This aspect also needs to be tested. If we test the app only on IE then we cannot ensure if works well on Netscape or Fire-Fox. Because these browsers may not only render pages differently but also have varying levels of support for client side scripting languages such as java-script.

- Even on similar browsers application may be rendered differently based on the Screen resolution/Hardware/Software Configuration

- Network speeds:
Slow Network speeds may cause the various components of a Webpage to be downloaded with a time lag. This may cause errors to be thrown up.

The testing process needs to consider this as important factor specially for Internet based Applications

- ADA ( Americans with Disabilities Act)
It may be required that the applications be compliant with ADA. Due certain disabilities, some of the users may have difficulty in accessing the Web Applications unless the applications are ADA compliant. The Application may need to be tested for compliance and usability

- Other Regulatory Compliance/Standards:
Depending on the nature of the application and sensitivity of the data captured the applications may have to be tested for relevant Compliance Standards. This is more crucial for Web Based Applications because of their possible exposure to a wide audience.

- Firewalls:
As mentioned earlier Applications may behave differently across firewalls. Applications may have certain web services or may operate on different ports that may have been blocked. So the apps need to be tested for these aspects as well.

- Security Aspects:
If the Application captures certain personal or sensitive information, it may be crucial to test the security strength of the application. Sufficient care need to be taken that the security of the data is not compromised.

Why technology platforms affect testing?

Technology platform upon which the Web Application was built also creates different strengths and weaknesses. Different Test Automation tools & packages are available for different Technology Platforms. This can influence the Test Strategy and the way in which Test Execution is done.

Challenges in Testing Web Based Web Applications:

To ensure that sufficient Test Coverage is provided for Web Based Applications and to provide a secure, reliable application to the user the above factors need to be considered.

For internet based applications accessibility and security can be important issues.
On one hand Organizations would like to cater to their users around the world on the other hand they could end up exposing their security holes all over.

Testing could be the last chance to ensure the safety of the data and the organization.

Summary:

In this article we studied the various challenges in Testing Web Based Applications. Why Testing Web based Applications is different and different factors affecting the testing.

What is User Acceptance Testing?

Introduction:

This article attempts to explain the process of User Acceptance Testing.

Once the application is ready to be released the crucial step is User Acceptance Testing.

In this step a group representing a cross section of end users tests the application.
The user acceptance testing is done using real world scenarios and perceptions relevant to the end users.

What is User Acceptance Testing?

User Acceptance Testing is often the final step before rolling out the application.

Usually the end users who will be using the applications test the application before ‘accepting’ the application.

This type of testing gives the end users the confidence that the application being delivered to them meets their requirements.

This testing also helps nail bugs related to usability of the application.

User Acceptance Testing – Prerequisites:

Before the User Acceptance testing can be done the application is fully developed.
Various levels of testing (Unit, Integration and System) are already completed before User Acceptance Testing is done. As various levels of testing have been completed most of the technical bugs have already been fixed before UAT.

User Acceptance Testing – What to Test?

To ensure an effective User Acceptance Testing Test cases are created.
These Test cases can be created using various use cases identified during the Requirements definition stage.
The Test cases ensure proper coverage of all the scenarios during testing.

During this type of testing the specific focus is the exact real world usage of the application. The Testing is done in an environment that simulates the production environment.
The Test cases are written using real world scenarios for the application

User Acceptance Testing – How to Test?

The user acceptance testing is usually a black box type of testing. In other words, the focus is on the functionality and the usability of the application rather than the technical aspects. It is generally assumed that the application would have already undergone Unit, Integration and System Level Testing.

However, it is useful if the User acceptance Testing is carried out in an environment that closely resembles the real world or production environment.

The steps taken for User Acceptance Testing typically involve one or more of the following:
…….1) User Acceptance Test (UAT) Planning
…….2) Designing UA Test Cases
…….3) Selecting a Team that would execute the (UAT) Test Cases
…….4) Executing Test Cases
…….5) Documenting the Defects found during UAT
…….6) Resolving the issues/Bug Fixing
…….7) Sign Off

User Acceptance Test (UAT) Planning:
As always the Planning Process is the most important of all the steps. This affects the effectiveness of the Testing Process. The Planning process outlines the User Acceptance Testing Strategy. It also describes the key focus areas, entry and exit criteria.

Designing UA Test Cases:
The User Acceptance Test Cases help the Test Execution Team to test the application thoroughly. This also helps ensure that the UA Testing provides sufficient coverage of all the scenarios.
The Use Cases created during the Requirements definition phase may be used as inputs for creating Test Cases. The inputs from Business Analysts and Subject Matter Experts are also used for creating.

Each User Acceptance Test Case describes in a simple language the precise steps to be taken to test something.

The Business Analysts and the Project Team review the User Acceptance Test Cases.

Selecting a Team that would execute the (UAT) Test Cases:
Selecting a Team that would execute the UAT Test Cases is an important step.
The UAT Team is generally a good representation of the real world end users.
The Team thus comprises of the actual end users who will be using the application.

Executing Test Cases:
The Testing Team executes the Test Cases and may additional perform random Tests relevant to them

Documenting the Defects found during UAT:
The Team logs their comments and any defects or issues found during testing.

Resolving the issues/Bug Fixing:
The issues/defects found during Testing are discussed with the Project Team, Subject Matter Experts and Business Analysts. The issues are resolved as per the mutual consensus and to the satisfaction of the end users.

Sign Off:
Upon successful completion of the User Acceptance Testing and resolution of the issues the team generally indicates the acceptance of the application. This step is important in commercial software sales. Once the User “Accept” the Software delivered they indicate that the software meets their requirements.

The users now confident of the software solution delivered and the vendor can be paid for the same.

What are the key deliverables of User Acceptance Testing?

In the Traditional Software Development Lifecycle successful completion of User Acceptance Testing is a significant milestone.

The Key Deliverables typically of User Acceptance Testing Phase are:

1) The Test Plan- This outlines the Testing Strategy

2) The UAT Test cases – The Test cases help the team to effectively test the application

3) The Test Log – This is a log of all the test cases executed and the actual results.

4) User Sign Off – This indicates that the customer finds the product delivered to their satisfaction

Summary:

In this article we studied the process of ‘User Acceptance Testing’.