Quality Testing of Websites

Not long ago websites were mostly HTML, with a few scripts. This changed to

dynamic pages with the advent of technologies such as JSP and ASP. Then came

XML, Web services and slightly different but XML-based technologies such as AJAX

and MXML. Websites now are no longer static with a defined set of page flows or

navigations. There is a lot that keeps happening even within a page,

asynchronous requests for example. Therefore, the way websites are tested has

also changed. There are tools that test a particular aspect such as unit testing

a component in a page, and there are complete suites for Web testing. We look

into new ways of Web testing, and two of the many available products for Web and

Web services testing, to explore their pros and cons with new paradigms in Web

testing.

New ways to test



A website can be complex, with rich interface of pages using multiple
technologies. Pages generated by objects for response to request are present.

The objects that make up business logic are there. So is database access and

concurrent users. Then you also have to take care of objects or scripts serving

ads, asynchronous requests and add to all this, page navigation. Sounds like a

recipe for disaster if a smooth function of all of this is not assured! Well it

is better to have a look into the newer quality parameters. You need to conduct

sets of tests, periodically and preferably automatically. Your tests should in

any case suffice the basic aspects such as browser independence, testing for

fonts as not all browsers support exactly the same font set, etc.

The newer aspects to take care of include testing components that would

update asynchronously in the page for their behavior in isolation. Check error

messages generated from these components also. Do this because you cannot afford

to kill the entire page if such component generates error. Especially, if you

are doing business through your websites, as this will be hazardous. You should

then test how components interact with other components, if any. In addition,

test how all of this behaves, ie, the entire page. Web services need to be

checked for security along with the regular functional and load testing, as this

service might be facilitating a transaction. This is something that, if

neglected, will surely lead to a disaster. Lastly, your tests should also keep

track of 'traditional' aspects of concurrency, database access, functional

and load testing. You never know if your website gets numerous hits with

requests right from the word go!

Lisa



It is a tool for testing Web services. It lets you drill down to every nook

and corner of your WSDL file. We evaluated it using the in-built demo server. We

used the freely available Lisa WS-Testing for our review. You need to register

for receiving a license through mail to activate this product. The installation

and activation is straightforward once you acquire the license. To start with,

you have to create a test case document and then record the steps. Let's look

into major parts of the tool and how it can help you in Web services testing.

Test case wizard



We took a sample Stock Quote Web service for our review. The process is pretty
straight forward and easy. The tool requires you to first create a 'Test Case

Document,' after which you can add various steps to your test case depending

on whether you want to go for functional testing, load testing or even testing

for security in your Web service. The tool starts a wizard containing a list of

actions, including an option for creating your own test action if the ones

provided in the wizard do not match to your taste. You can deploy the '.jar'

archive for making it available to the tool quickly through the 'hot deploy'

command in the System menu. To sum it up, the wizard is very simple yet allows

you to configure each step to the granularity level that you want.

The Test Manager wizard in Lisa allows you to edit every step for providing additional configurations

Assertions and filters



Assertions are applied to constrain a particular step in case you want to
evaluate a particular variable or property or even apply a check for some

condition. For example, while we tested the stock quote service we used an

assertion to compare the received value with actual value. You can add multiple

assertions to a step. Filters on the other hand allow multiple evaluation

criteria for conditional evaluation. They are particularly useful during dynamic

evaluation. For example, when the output of one-step goes as a property input to

another step, or if the output of a step needs evaluation with assertions before

the next step executes. These assertions and filters work in the same way with

any data set, atomic values or complex data types such as objects and XML

documents.

We were particularly impressed by the way the tool handles Web services

testing, while keeping the process simple for the user. It also has the ability

to pull data from a variety of datasets, for example in our stock quote web

services we pulled data from a JDBC data source. That is not all; you can also

pull data from Excel files, CSV files, and even simple text and XML documents.

Add to that the tools ability to evaluate assertions and filters, and test Web

services for security. We feel such tools can help a great deal if added to

development life cycle right from the early stages. Definitely, a good solution

to go for your Web services testing needs. Moreover, if you are not satisfied

with just the WS-testing, you can go for the commercially available suite that

has a lot more features to add to the ones we brought out in this review.

Advent Net QEngine



This complete suite covers almost all aspects of Web testing, is feature rich,
and runs on the browser. Advent Net QEngine covers Web performance and

functional testing as well as Web services performance and functional testing.

You can initially download the trial version for evaluation. With this suite you

can configure your tests once and it then does unattended testing. So, you don't

have to worry about both the repeated and automated part of testing we discussed

earlier. We used a few demo applications built on Java solutions and one with

pages containing AJAX enabled JSF components to test the product.

Functional testing



The functional testing suite aims at not only rigorous testing to ensure smooth
functionality, but also automating the testing process. To start with, it is

loaded with features to cover a wide range of functional issues that need to be

checked for a web application. In context of the newer factors that we have to

take care of, an application might contain a page wherein a particular element

might be updating itself asynchronously or even in a rollover fashion. For

example, an AJAX control might link to different pages at different stages of

its states depending on user input. We liked the way QEngine handled this as you

don't have to record all states for such a control, you can simply record one

of these and the suite provides 'Dynamic Functions.' Using these you need to

first record just one of the inputs to the control and then handle all other

possibilities.

Other than this the software allows you to take care of changes in elements,

whether it is a location change or property change. While you are recording for

automation everything is saved in a 'GUI Map' file. To configure changes in

property, all you have to do is edit this GUI map file to edit the value of

element's property. Not only this, the property changes based on user input

can also be configured using in-built methods while you are editing scripts for

functional testing. The other options provided in functional testing include

taking off exceptions, if they occur during a playback while testing, and usage

of XPath based expressions for filters.

The suite provides a detailed report for the results and also allows you to drill down to a specific result that you want to view. You can do that by simply clicking on the region in the chart you want to drill down to

Performance Testing



Like in functional testing over here also, you create suites that contain all
the dataset, configuration files and reports generated during test processes.

For performance testing, the suite allows you to create multiple 'Business

Case.' So, practically you can check the performance for as many scenarios as

you can think of while evaluating performance. The precision of the test results

is only dependent on the precision of the analyzed work load.

The performance testing process using QEngine is also straight forward. You

have to record user scenarios first, which is done by simply choosing a 'New

Script' option. You then parameterize your script which is basically passing

values from the dataset, cookies, a constraint or even a URL. The dataset can be

used to parameterize different set of data for controls such as form fields, and

even populating different values for same element in different playbacks.

For example we checked out login controls with 2 different value sets in two

repeated playbacks, using the dataset. The degree of dynamics provided is

equally impressive in almost all scenarios as you can also dynamically re-write

URLs, and simulate multiple users for usage by different users in each run of

your test scripts, while performing load testing.

The reports generated at the end for both functional and performance testing

are also very elaborate and provide drill down to look into a specific result.

That ways you can even track how a single component behaved while seeing the

overall results. While all these features are no less than impressive, we did

miss a few features offered in similar solutions, for example some support to

test for security.

Overall, a good buy keeping in mind the freedom it provides with scripts. The

interface is not simple but that is understood as you would rather prefer

spending some time on a tool that can be quite helpful than going for an

on-the-fly tool that does not allow you to do much.