How to Go about SOA Testing

SOA solutions and blueprints require quite a few critical decisions during
design and implementation. The same applies to testing these solutions, as it is
not just functionality and performance one has to take care of. Testing SOA
solutions also requires testing interfaces and services that might be bringing
together diverse systems or platforms and, of course, their performance and
related aspects. And as it happens with any new technology, there is not much of
maturity in this process and only a few choices of products or suites are
available that help in SOA testing. In this article, we look into some of the
vital points of SOA testing and also evaluate the Parasoft SOA Suite for the
same.

What to test for

As SOA systems are a sea of services, ie, a complex integration of either Web
services, mid-tier services or even legacy systems again exposed as services,
the testing process requires looking beyond the conventional code and GUI
aspects. We are not saying that you need not touch the conventional aspects, but
the nature of SOA solutions is such that these tests do not remain the only
important thing to take care of. The presence of services is not the only factor
complicating SOA testing. Process flows in a SOA solution might involve multiple
and diverse systems integrated to provide a flow that can utilize these diverse
systems. In other words, the services implemented might be executed in a way
that they interact in a particular order or path with various systems. In
process terms, you have to test a complete integration framework keeping in mind
business processes that run covering your existing intact systems from one end
to the other. Also, the performance, vulnerability and functional tests should
also be performed.

So even if you realize that it's more about testing the integration
framework, what exactly to test in that? Well, here the interfaces are such that
they create links between applications or systems, not within them. This brings
out another critical aspect of testing. Most of the businesses have applications
or systems that are catering to particular business units. Thus, your tests
should start with unit testing the individual modules, then test the interfaces
for the integration they provide and then the processes on the whole. Testing
the processes and services is required as the test run will invariably include
bits from diverse systems integrated by them and give you a first hand look into
how correct the overall processes are.

The challenges

Now that we know what is to be tested, let us look into how to go about it and
what challenges project managers face while doing SOA testing. As we mentioned
earlier one should start with testing individual modules for their outputs.
Project managers advocate two ways to testing SOA-one is testing smaller parts
of the integrated system, or in other words, a part of the entire process flow
and then testing the entire process and the second is end-to-end testing.

We feel testing integration in parts is certainly a better way as jumping to
a complete end-to-end testing can open up a Pandora's box. It gets tough to
analyze the flaws in integration if revealed by the end-to-end test as you have
a huge set of things going wrong, or it might be a particular module or a
particular transaction by the service or connectivity between either the
services or the service and the underlying legacy system. So it is better to
zero down on most of the issues we just stated early in the development cycle
rather then getting lost into a sea of wrong things.

We know that any testing cycle has

the element of repeatability. And here's where the horror story starts as the
number of process flows and execution steps' combinations can be killing. Thus,
when you get down to regression testing, keep in mind that firstly it is not a
building from blocks kind of solution being tested but an assembly of systems or
applications. Secondly, it is better to go for a regression library that can be
executed after any changes are made. This is always the better way to go because
it is more manageable.

The biggest problem you need to tackle in testing such solutions is the
availability of the 'underlying' systems or application or even components. For
instance, your integration might be bringing together, say, an internal
application, one from a specific vendor or even data being received from data
sources. All of their availability becomes highly important while going for
integration testing in parts and end-to-end testing as well. A good way to
tackle this is to simulate the existing applications or systems to avoid delays.
So regardless of the scale or size of the project and the processes being
created, it is always important that your SOA testing also follows the SOA
philosophy, keeps in mind the diverse layer of systems and the processes that
run integrating them. And, of course, you have to test for the processes running
on top of these rather than checking the underlying system for output.