by April 3, 2007 0 comments

SOA solutions and blueprints require quite a few critical decisions during
design and implementation. The same applies to testing these solutions, as it is
not just functionality and performance one has to take care of. Testing SOA
solutions also requires testing interfaces and services that might be bringing
together diverse systems or platforms and, of course, their performance and
related aspects. And as it happens with any new technology, there is not much of
maturity in this process and only a few choices of products or suites are
available that help in SOA testing. In this article, we look into some of the
vital points of SOA testing and also evaluate the Parasoft SOA Suite for the

Direct Hit!
Applies To: Quality Analysts
USP: Know how to go about testing SOA implementations
Primary Link:

Google Keywords: SOA testing, integration testing

What to test for
As SOA systems are a sea of services, ie, a complex integration of either Web
services, mid-tier services or even legacy systems again exposed as services,
the testing process requires looking beyond the conventional code and GUI
aspects. We are not saying that you need not touch the conventional aspects, but
the nature of SOA solutions is such that these tests do not remain the only
important thing to take care of. The presence of services is not the only factor
complicating SOA testing. Process flows in a SOA solution might involve multiple
and diverse systems integrated to provide a flow that can utilize these diverse
systems. In other words, the services implemented might be executed in a way
that they interact in a particular order or path with various systems. In
process terms, you have to test a complete integration framework keeping in mind
business processes that run covering your existing intact systems from one end
to the other. Also, the performance, vulnerability and functional tests should
also be performed.

So even if you realize that it’s more about testing the integration
framework, what exactly to test in that? Well, here the interfaces are such that
they create links between applications or systems, not within them. This brings
out another critical aspect of testing. Most of the businesses have applications
or systems that are catering to particular business units. Thus, your tests
should start with unit testing the individual modules, then test the interfaces
for the integration they provide and then the processes on the whole. Testing
the processes and services is required as the test run will invariably include
bits from diverse systems integrated by them and give you a first hand look into
how correct the overall processes are.

The challenges
Now that we know what is to be tested, let us look into how to go about it and
what challenges project managers face while doing SOA testing. As we mentioned
earlier one should start with testing individual modules for their outputs.
Project managers advocate two ways to testing SOA-one is testing smaller parts
of the integrated system, or in other words, a part of the entire process flow
and then testing the entire process and the second is end-to-end testing.

We feel testing integration in parts is certainly a better way as jumping to
a complete end-to-end testing can open up a Pandora’s box. It gets tough to
analyze the flaws in integration if revealed by the end-to-end test as you have
a huge set of things going wrong, or it might be a particular module or a
particular transaction by the service or connectivity between either the
services or the service and the underlying legacy system. So it is better to
zero down on most of the issues we just stated early in the development cycle
rather then getting lost into a sea of wrong things.

We know that any testing cycle has
the element of repeatability. And here’s where the horror story starts as the
number of process flows and execution steps’ combinations can be killing. Thus,
when you get down to regression testing, keep in mind that firstly it is not a
building from blocks kind of solution being tested but an assembly of systems or
applications. Secondly, it is better to go for a regression library that can be
executed after any changes are made. This is always the better way to go because
it is more manageable.

The biggest problem you need to tackle in testing such solutions is the
availability of the ‘underlying’ systems or application or even components. For
instance, your integration might be bringing together, say, an internal
application, one from a specific vendor or even data being received from data
sources. All of their availability becomes highly important while going for
integration testing in parts and end-to-end testing as well. A good way to
tackle this is to simulate the existing applications or systems to avoid delays.
So regardless of the scale or size of the project and the processes being
created, it is always important that your SOA testing also follows the SOA
philosophy, keeps in mind the diverse layer of systems and the processes that
run integrating them. And, of course, you have to test for the processes running
on top of these rather than checking the underlying system for output.

Parasoft SOATest suite

The Parasoft SOATest suite aims at easing out SOA testing by providing a
diversely configurable, automated suite for testing out everything from WSDL
files to custom SOAP headers. We checked out the suites capabilities on the
same stock quote Web service that we have used for testing Web testing
suites earlier and a few other sample Web services. Read on to find out what
we feel about this suite.

Testing Web services: Let’s first look into the WSDL tests that the suite
provides. You can author a WSDL test either from the wizard or create an
empty case and then add tests to it. Now once you add the URL for your Web
service, you can choose from the available options of static tests that
check the schema and semantics. You can also further check your Web service
for interoperability and even create regression test to check its
performance over a sustained period. Other than these, you can unit test the
individual operations being implemented by your Web service by using the
Unit testing feature that the suite offers. Here also, much like many tools
available it is more of a GUI-based system using which your tests can be
created without much of a hassle. You can combine your tests in a test-case
document to include all the test scenarios that test the performance and
functionality of your Web-service implementation.

The graphical output allows easy analysis
test results and quickly editing the tests to
change settings

Functional and load testing: The functional testing support is pretty
elaborate in Parasoft. It creates suites by reading through your WSDL file
and also UDDI registries amongst others. Thus, our sample Web service had
suites created automatically while we were checking for functional testing.
You can then customize these suites for fine tuning or generalizing the test
suites. Once all the test cases are done, the software also allows for
converting them into regression test. The regression library that we talked
about a little earlier is something that this tool can provide you without
much of a fuss. In short, you can configure tests that can cover every
aspect of functional testing so that you feel comfortable that all possible
aspects have been checked out while emulating client for your service(s).
For load testing, the tool provides four options of load conditions for
emulation. You can decide the load pattern from the available choices of
‘Bell Curve’, a ‘Linear Increase’ curve, ‘Steady Increase’ curve or a Buffer
Test, which provides variable loads over a period of time.

The tools works pretty well for Web services, and can also be customized
for testing CORBA or JMS-based services other than WSDLs. However, we could
not test these due to time constraints.

Bottom Line: Overall, a decent buy for your testing and governance
development needs.

Price: Professional Ed Rs 2,15,775; Enterprise Ed Rs 8,09,775 (1
yr free upgrades and unlimited support with both editions)
Key Specs: SOA Governance Development, CORBA, JMS support
Pros: Easy configuration, automated and manual testing
Cons: None
Contact: Parasoft Software, Bangalore, Tel: 41498070, E-mail : SMS Buy
130499 to 6677

No Comments so far

Jump into a conversation

No Comments Yet!

You can be the one to start a conversation.