Advertisment

Software Testing and QA

author-image
PCQ Bureau
New Update

It is not sufficient to simply write a piece of software and distribute it to someone. Software bugs and improperly tested code cost millions of dollars in damages and millions more in time and money to fix. Whether the application be personal, commercial or something as specifically written as ERP software, it needs to be tested properly and its quality accounted for.

Advertisment

While small development teams might do the testing part too themselves, in larger organizations it is commonplace to have dedicated testing teams in place. These testing groups use a mix of software-commercially available as well as some home-grown solutions-along with many human-touch methods to both provide proper feedback to the developers and make sure what goes to the customer is quality assured.

Direct Hit!
Applies to:
Software developers and testers
USP:
Understand the variety of testing mechanisms and get the right perspective on software quality assurance
Links:
http://www.softwareqatest.com 

Software testing happens in several phases, some of it done by the developers themselves, some by the team leaders who check for integration issues and others by the public at large. Self testing, where it happens, should be restricted purely to the build cycle itself and never be done in the production stages of development. 

Advertisment

Self testing: a bad idea

During a build cycle, a lot of self testing (testing carried out by the developer himself) happens. However, full-scale testing should be done by separate groups. The reason is, when we as the developer test a module, we rarely create conditions of total randomness. We don't even consider providing values in a different way. How would the code react if someone entered a telephone number where an e-mail address is asked for? How does such a thing affect something apparently disconnected down the line-in the reporting module, for instance? What happens if in a wizard we select an option, click the Back button a couple of times and then re-run it? Does the software remember our previous settings intelligently? This is why many organizations prefer to engage testers at large (public testing) to check out their software. When this is done, a lot more conditions, both deployment as well as input values, can be checked out and the actual results compared against expected behavior.

Again, the scenarios used to test the applications can make a huge difference to what is seen during testing. Is the software being tested on a real server where one is asked for or are we merely simulating it on an ordinary PC? Are we really testing that new N'Gage game on a prototype unit or on an emulator?

Dozens
of different testing types may be employed. Here are a few main
types
Testing
type
Presumptions Description
White
Box
Code
and execution paths are known
Testing
checks if execution happens along known and predicted paths for a
given set of input. Most build-cycle testing is of this category
Black
Box
Code
and execution paths are not known
Testers
check for general bugs in execution and interface
Unit
Testing
Done
directly on a piece of code
A
piece of code is taken and directly tested using manual or automated
methods
Integration Done
on code, usually by team leaders
Different
modules of the application are checked to see how well they work
together
Functional No
knowledge of the software is required
Interface,
usability and functionality of the application is checked
Sanity Done
by the developers themselves
On-the-spot
evaluation by developer teams to determine if software is stable
enough to pause a build-cycle for testing
System Done
on an entire application
The
entire application is taken as a single unit and tested
Load Special
software is needed
Tested
under conditions of extreme load and stress to check the performance
of the  software
Usability Interface
and functional
General
usability and satisfaction of its purposes is tested
Alpha
and Beta
Usually
public
Versions
of the software are given out to predetermined individuals to have
them tested under unknown conditions of load and usage
Advertisment

Testing scenarios

In more situations than not, testing happens under controlled environments. For example, when we at PCQuest receive a product for testing, we have a set of conditions we recreate for that product, based on the type of product. However, in a real-life scenario, this would not be the case and a completely random set of conditions will prevail. But since it is not possible to create every type of deployment scenario, we simplify the comparison process by having conditions we have tested under before and hence have results we can compare. Thus we can say that if the user has deployment scenario 'A' and is running module 'B' and has provided the input set 'C', 'D' will happen. If instead 'E' happens, we log it as an unexpected case.

This is why there are different types of testing that happen at different stages of the development and build cycles.

Test types

Testing processes are of different types: White Box, Black Box, Unit, Integration, Functional, Sanity, System, Load, Usability, Alpha and Beta. See the table for a quick comparison.

Most of these test types happen in an undocumented and automatic fashion. For example, when we add a new screen, we would naturally run and test that screen. But we usually fail to document the achieved results formally, thinking it too inconsequential. Generally, only the Unit, System, Load and Alpha/Beta testing happens formally.

Advertisment

Some of these happen by hand, where someone sits in front of a terminal, runs the application and notes down what he sees. Others are done using scripts, tools and automated testing software.

QA tools

Depending on the type of testing being done, a number of software tools may be used. These tools make it easy to analyze performance and provide the tester with a numerical score that can be used to compare other tests against. Some commonly used tools are WinRunner, LoadRunner, JUnit, FxCop, Visual Studio Analyzer, IBM Robot and CA's QA Center.

Depending on the language and platform of development, IDE-integrated debuggers and tools are also used.

Advertisment

However, as with everything else, ideal results are only achieved when 'best practices' are followed. When development companies are sure that they can deliver on the quality front, they provide something that has come to be known as 'quality assurance'.

Software and quality assurance

This is something like life assurance, given out by insurance companies. Basically, the software vendor will guarantee that for a fixed period of time, they are under a contract with you to keep your software absolutely up to date at a nominal cost. Just as with life insurance, where you assure a sum of a few lakh rupees for a comparatively paltry monthly premium, you would pay the software vendor a fixed sum of money every year. 

Subscribing to such policies would help keep large deployments trouble-free and up to date at nominal cost and logistical hassles.

Advertisment

Sujay V. Sarma

Advertisment

Stay connected with us through our social media channels for the latest updates and news!

Follow us: