Business and Technical Services Orchestration v System Integration

Andy Mulholland, CTO, Capgemini

The idea of reusable services for mainstream purposes started with SOA, Services Orchestrated Architecture, and I would say that the hangover of the failure of SOA to make any real difference is affecting the approach to using services today. So what went wrong? Quite simply that almost always the so called services were in fact built and deployed in a clustered manner to make an application because that's the way things were done. Today the advent of clouds, hosting, web services, etc. make for a mature environment in which a genuine shift to a different environment is under way, and that's what this article is about.

The role and capabilities of a system integrator are generally understood when applied to building solutions involving enterprise applications. Enterprise applications are broadly defined by three technical characteristics, usually linked to enterprise architecture; close-coupled, state-full and deterministic. A services environment delivered by a cloud has none of these characteristics, instead being; loose-coupled, stateless and non-deterministic. Accordingly, solutions are built in a very different manner, usually referred to as orchestration. Just as the term 'system integration' is a broad term covering complex detail in exactly how it can be achieved, so is 'orchestration' too broad a term to really provide any real understanding of the complexities that it must address.

A services environment has two distinct pieces; the business services brought together to build unique processes by combining existing, or new service elements, (the term 'service' being used to indicate small, individual pieces of code that provide a single function); and technology services that provide the common functional environment, or platform, of capabilities to 'run' the business services. It is a vitally important characteristic of providing services from a cloud computing model, whether public, private or hybrid, that this separation is understood and maintained. The sheer scale of the number of services that will be operating, together with the frequency of change expected, as well as the increasing amount of external services that will require the ability to run on demand, requires the automated provisioning approach.

Some definitions, including simple popular ones such as on Wikipedia, state that cloud is about scale and as such 'automated workflows are essential'. Others remark that the decoupling, or abstraction, of the business services from the technology services is the true definition of 'agile computing' in support of the 'agile enterprise'. A correctly built cloud solution will place the technology services with one, or more, IaaS, Infrastructure as a Service, or PaaS, Platform as a Service, providers and free the business service solution provider to orchestrate the business solution without any concern as to where, or how, it will be run, allowing the focus to remain solely on the business activities and alignment.

This is wholly and completely different from the building and deploying of an application where the provisioning, and optimizing, of a server are key tasks. In much the same way, business services orchestration assembles processes from standardized services that were intended to be integrated, as opposed to system integration being focused largely on overcoming the inconsistencies and barriers between applications and their attendant computational technology systems. Virtualization of servers may simplify and reduce the time and cost of deployment by reusing existing servers better, but in client-server applications the relationship with the server remains an important characteristic.

Clearly the more complex side is the design and construction of the IaaS and PaaS layers to provide technology services, and the key step to achieving this is the use of service catalogues. Though not new, service catalogues gain a new, expanded and significantly more important role, and must increasingly be based on industry standards to ensure that they are able to support not only internally developed business services, but the increasing numbers of services from external sources and processes.

Service catalogues cluster together the individual elements of technologies, such as a server, network, security appliances etc. to provide building blocks that will support different types of business services. As an example, the service catalogue set to support an external set of business services used by customers would be different to the service catalogue set used for secure processes with nominated partners which would include a more comprehensive set of governance and security elements. Correctly established, enterprise service catalogues provide the base line infrastructure that is common to, and shared by, all business users and the business services.

Service catalogue sets can be used for internal traditional IT functions such as establishing a service catalogue set to support testing. Service catalogues can also incorporate internal and external computational resources, or even direct different business services to run on different resources mixing internal and external resources so that security services could be run internally, and routine business services of an intra-business process would run externally. IBM and HP are establishing service catalogues to support vertical sector intra-business processes in markets such as financial services, smart cities, and retail, but this will still necessitate individual users to both build and deploy their own business services, and service catalogues to cover their elements of the extended processes.

At an enterprise level service catalogues enable 'business agility' which is usually defined as the ability to create, change, deploy, and pay for, business beneficial capabilities to compete in the market place through the front office, as opposed to automate stable back office procedures with traditional IT. The decoupling of the technology element together with the automation of provisioning enables business users to use simple visual tools to choose, combine, and edit business processes without the need to consider these elements, or even the enterprise level governance requirements. In addition, developers are able to focus on the creation of libraries of business services that will provide a richness capability, or to react rapidly to write and add new services in response to changing market conditions.

An enterprise can expect that it will have tens of thousands of business services in use covering its own and those of its customers, suppliers, partners and industry. An effective set of technology services to support and safely enable these in a fully automated manner is essential. Conversely, the freedom to respond agilely to the market with new services, or processes, is equally essential because it delivers the new business value. The role of the systems integrator will continue to be required to support the enterprise applications based on transactional client-server, but a new set of skills is required to support services integration and enable enterprises to change their business model.