Building Transformations Using BizTalk Server

In this increasingly connected world, no application is an island anymore.

If you look at any typical enterprise, it will have multiple applications each

having different sets of data. When these applications are created, they are

designed to operate independently. For example, an enterprise will have an

application to manage employee details. The enterprise will also have a pay

processing system. When a change is made in the employee database, the

information also needs to flow into the pay processing system. When such a

requirement comes, an enterprise can't just throw away current applications and

build new one.

The first time an enterprise needs two such applications to talk to each

other, they create a proprietary interface to integrate them. Now as the

integration needs keep on increasing, they keep adding these proprietary

interfaces till in becomes a messy spaghetti of such interfaces. To solve this

problem, a class of platforms were designed. These platforms work as integration

brokers between all the different applications in the enterprise. These

platforms translate the message format of one application to the data and

message format of other application. Microsoft's BizTalk Server connects

heterogeneous systems - that means you can use BizTalk Server to not only

connect application on Microsoft platforms and also to connect applications

which are on third party platforms.

A key challenge in all integration products is to translate messages coming

from one application into the format understood by the other application.

BizTalk Server has solved the problem by introducing a visual tool called

BizTalk Mapper.  BizTalk Mapper enables developers to define transformations

between two messages interactively. In BizTalk Server, messages are handled as

XML Schemas (XSD). BizTalk Mapper generates Extensible Stylesheet Language

Transformations (XSLT) that is used by BizTalk Server's runtime engine to

transform messages.

Mapper basics



In BizTalk Mapper, you start with an incoming schema representing the

message needing transformation and destination schema representing the message

which will be generated after transformation. Some of the transformation can be

done by assigning elements from one schema directly to the elements in other

schema. You can do this in the BizTalk Mapper by just drawing a line (called

link) between these schema elements. BizTalk Mapper also provides tools for

scenarios where a whole structure is to be mapped from one schema to another.

Not all transformations are straight forward; some require manipulation of

input data for generating the correct output. For these manipulations BizTalk

Mapper provides some out of the box shapes called "functoids." Functoids take

inputs either from the input schema or other functoids and perform specified

operations on them. For example, you can use a "String Concatenate" functoid to

combine two strings. This functoid can be used when, let's say, the first name

and last name coming in from the input schema needs to be combined because the

output schema only takes a full name.

BizTalk Mapper provides a wide variety of functoids - logical operations,

string manipulation, mathematical operations, date/time manipulation, scientific

operation, database operations, scripting, etc. Apart from these functoids,

BizTalk Mapper also provides an SDK that allows you to easily create your own

custom functoids.

BizTalk Mapper provides a wide variety of functoids - logical operations, string manipulation, mathematical operations etc.

After creating the map using the graphical tool, you can compile it to

generate XSLT, which BizTalk Server uses to transform the messages. You can also

validate the map to ensure that the correct output is generated. You can also

test maps against sample input messages.

What's New in BizTalk Server 2010



The complexity of maps created in BizTalk Mapper has been increasing at a

rapid pace with the increasing size of messages being processed and increasing

complexity of the integration solutions. The complexity of the maps made it

difficult for users to create maps in graphical tools and a nightmare to

understand/maintain them. Because mapping is the biggest development work in

typical integration solutions, improving the usability of the mapper was a key

focus area to enhance developer productivity for BizTalk Server 2010. The

BizTalk Server team collaborated with Microsoft Research to identify

improvements and features that can have great impact on developer productivity.

One of the problems in big maps was the amount of background noise that

distracted you from task at hand. BizTalk Mapper solves the problem by first

hiding and/or dimming out the links that are not relevant to the page area of

the map you are currently viewing. The links where both ends are not currently

in view are now hidden. Even the links where only one end is currently in view

are dimmed out by using a thinner dashed line types. When you select any schema

node, link or a functoid in the map, BizTalk Mapper also brings that in

prominence along with all the relationships associated with it. This in turn

allows you to focus on the selected object and its relationships while reducing

all the background noise. If the relationships are not in the current view,

Mapper scrolls the mapping grid surface to bring it in view. This drastically

reduces the amount of scrolling you had to do previously, to view end-to-end

relationships. After the relationships are in view, you can now hover over

different functoids and links to understand the end to end transformations.

"Relevance View" for schemas is another improvement that makes it easier for

you to focus on only the relevant schema nodes when looking at a map. Because a

map developer is only interested in nodes that are actually part of the map,

hiding the other nodes makes it easier for the developer to understand the full

map better. You can turn on this feature on either the source or destination

schema using toolbar buttons. When the unused nodes are hidden, they are still

easily accessible through tooltips. You can also expand a collapsed node to

override the default collapsing of all non-relevant nodes.

Moreover, all the configurations and settings related to a functoid have been

consolidated at a single location making the configuration task much easier and

understandable. BizTalk Mapper also provides visual cues similar to IntelliSense

for functoid configuration errors.

The new incremental search will help you find the relevant object quickly.

You can not only search through the schema nodes, but also through functoids.

You can also select which properties of functoids you would like to search.

When you have big schemas and are looking to have direct links from the

source schema to the destination schema, one of the biggest issues is to find

the relevant nodes in the schemas. Now in BizTalk Mapper if you select any

element in the source or destination schema and invoke "Indicative Matching"

mapper, it will suggest you the possible matches in the other schema. These

matches are generated based on a complex algorithm, which was developed in

collaboration with Microsoft Research. This algorithm looks at various

parameters like how words are shortened by dropping alphabets, relative position

in the schema tree, etc. to generate possible matches.

Summary



The enhanced mapper will be released in BizTalk Server 2010. BizTalk Server

2010 comes with other features like a centralized Setting Dashboard for easier

maintenance of configuration, an enhanced Trading Partner Management, new out of

box components like FTPS, etc. BizTalk Server 2010 was released for public Beta

on May 20th, 2010. You can download it and get more information on other new

features from

http://www.microsoft.com/biztalk.

Deepak Jain, Senior Program Manager, BizTalk Server, Microsoft India

Development Center, Hyderabad