Message Creation in BizTalk - solution uploaded

A few weeks ago I published this post about some experiments me and Randal Van Splunteren did around message creation.

Not surprisingly I was asked to post the solution we've used and so I have uploaded it here

 

Have fun! (let me know if anything's missing or unclear, it's been a while since I ran this...)

Fun with Message Creation in BizTalk

Back in Match I posted this entry about creating messages "from scratch" in BizTalk.

The post started a bit of an online discussion and a slightly more intensive offline discussion about the various ways to create messages and the differences between them.

As part of that discussion, Randal van Splunteren and I have exchanged some emails and Randal took the time and effort to create a test solution to compare the performance characteristics of the various methods which I have helped validating.

Randal has been kind enough to let me summarise our findings in this blog (and it only took me 6 months...buy I have my excuses) so here it is -

The scenario we've used to test is as follows -

There is one main orchestration that takes in a ‘command’ message using a file receive location; in this command message you can define the method to create a message:

• Map with Defaults (1)
• Map with xsl (2)
• Assignment with serialization (3)
• Assignment with resource file (4)
• Using undocumented API (5)

The first four options create messages according to the four methods I described in my blog post; the fifth one uses the CreateXmlInstance API suggested by Randal as a comment on my original post.

In the command message you can also set the number of messages that must be created;

Finally you can set if the method should use caching; we've implemented a very simple caching mechanism for the assignment and undocumented API methods (caching the generated instance in all three methods so it can be re-used in subsequent calls); for the map methods the caching parameter is ignored because BizTalk has its own caching for those methods.

When a particular test is finished the main orchestration writes out a ‘report’ message (again using file adapter) which contains the number of elapsed ticks the test took.

I've ran all the scenarios 5 times and averaged the results, between each test I have restarted the host to get as much like-for-like comparison as I could, so these numbers would not reflect true runtime performance of a live server but only the difference between the approaches; initially I ran all the tests creating 1 message at a time, here are the results:

msgs	Map using defaults	Map using xsl	Assign using serialisation	Assign using resource	Assign using API
1	13,243,663	12,687,314	8,153,346	8,135,461	36,374,565
1	13,385,005	12,888,630	6,905,139	8,620,287	36,468,805
1	12,837,338	13,943,338	9,272,362	8,815,033	37,723,069
1	15,630,602	13,298,954	6,679,173	8,027,708	35,877,260
1	12,729,576	12,765,337	7,113,975	9,174,668	36,919,198
Avg	13,565,237	13,116,715	7,624,799	8,554,631	36,672,579

or to put it graphically -

Then I ran all the tests again, this time creating 100 messages at a time -

msgs	Map using defaults	Map using xsl	Assign using serialisation	Assign using resource	Assign using API
100	15,195,199	15,254,912	9,158,223	8,951,018	231,352,547
100	14,421,621	16,523,637	9,259,892	8,700,856	226,704,695
100	15,199,198	15,010,499	8,476,670	10,222,202	232,357,798
100	16,725,023	15,684,085	9,110,269	9,866,252	227,806,462
100	15,349,885	14,475,857	9,101,879	10,295,228	226,928,786
Avg	15,378,185	15,389,798	9,021,387	9,607,111	229,030,058

Last I ran the 3 non-mapper versions with the caching enabled -

# messages	Assign using serialisation (cached)	Assign using resource (Cached)	Assign using API (cached)
100	9,696,044	9,478,015	41,350,100
100	8,288,120	10,087,574	37,410,620
100	9,156,289	10,473,718	36,493,118
100	8,715,621	10,001,671	40,628,198
100	8,289,295	9,951,817	37,919,237
Average	8,829,074	9,998,559	38,760,255

So, what I have spotted?

well, to start with, comparing my results with those Randal had I learnt that my laptop is much slower then his machine...(but you can't see that from the results, nor, I suspect, do you care...)

But seriously -

• It is interesting to see how, with the exception of the API scenario, there is very little difference between the generation of 1 message and the generation of a 100.
• It is quite obvious that the API call is much slower then the rest, but that does not surprise me considering the amount of work involved (getting the schema from the database, generating the instance off the XSD retrieved...)
• For that reason, it is also quite obvious that this method was the most beneficial from the use of the cache (but was still significantly slower then the others) as the cache prevented the repeating access to the database and the xml generation.
• On the same token, caching did not make a very significant difference in the other scenarios, but again- I wouldn't consider that surprising (as there's very little work involved)
• And of course - it is clear that using assignment shape to create messages using either serialisation or a resource file is indeed the fastest way (serialisation being a little faster on my machine)

I hope you find this useful and again - many thanks to Randal for all his effort in helping me get this out.

Creating a message "from scratch"

A question I get asked repeatedly is how to create a message in an orchestration “from scratch”, i.e. – when the message is not meant to be created from any other message.

Initially one might think this does not make sense, but I find myself doing it quite often actually; two typical scenarios are when one has to create a message before branching the process to satisfy the compiler (and avoid the “use of unconstructed message” error), a second is when one needs to return a message that simply contains few values obtained from, say, a database call, or some calculation etc.

In this case often it is simpler to create the shell of the message with the elements/attributes required and then use xlang’s xpath function to push the values into the right places.

Whatever the scenario is, there are, I believe, two options to create empty messages (corresponding to the two options to create messages in orchestrations in general, actually) –

1. Using a map
2. Using a message assignment shape (and a helper method).

The first one is quite obvious – you create a map, pick any message you may have in the process (you are likely to have at least one message in your process already) - use that as your input message and the message you want to create as your output message; then you’re facing two alternatives – the first one is to use xsl – you can create an xsl file that effectively has the XML you want to create hard coded in it and instruct the map to use that xsl.

The input message is completely ignored (and so is completely irrelevant) and the output message is always the way you want it to look like. You are not sensitive to any changes in the schema of your input message.

The other alternative, which I like less, is to actually use the mapper; in this alternative you would probably map the root node of your input message to the root node of your output message and then set up default values in the output message for any element/attribute that you wish to include in your output message.

The reason I believe this is not as good as the first alternative is that it is less obvious how the output message would look like; one has to follow up on the nodes to see what is set (or test the map) to see the output while in the xsl alternative one look at the xsl is usually enough to show what the output is going to look like.

Either alternative you choose – I used to think that using the mapper is a better option (as opposed to using the message assignment shape which I will describe shortly) – mostly because I thought this is a more standard way to create messages, and so it is more obvious, looking at the process and the project, where such constructions take place but mostly,I believed, if you’re using xsl files to create the output it would be very easy to spot, read and change them in the solution when necessary (simply find the xsl files and change them, no need to look for anything else).

Thanks to several people, but mostly to Ben Gimblett with whom I work with in one of the projects I'm involved with and who has insisted not to follow my advice and use helper methods to create messages, I now agree that using helper classes is the better way, mostly because using them you don’t have to use a dummy input message (as you do in the map) – which, I have to agree, can get quite confusing to anyone trying to understand the process but also because, I suspect (but have not tried to prove), a helper class will perform better than the mapper option.

When using a helper class you again have to alternatives –

You could have a message assignment shape in which you call a method that returns a .net type (class) that has been generated (using xsd.exe) from your schema.
All you need to do in your method now is create an instance of that generated return type – populate whatever members you require and return it.

In the assignment shape you assign the return value from the method to your message[part] and so BizTalk will take care of the serialization and will convert the class to the schema and because the class and the schema both represent exactly the same thing the serialisation would work just fine.

The benefit of this approach over the mapper option is mostly that there's no need for any dummy input message, no need to write xml or xsl; only very simple (and quite minimalist) code is required.

The downside – you need to generate those .net classes to represent any schema you wish to return, and maintain them as your schemas evolve.

The second alternative is simpler on the one hand as it does not involve generating and maintaining classes; it does, however, require a bit more wiring –

It starts the same way – a message assignment calls a method whose return value is assigned to the constructed message[type].

The difference is that the method does not return a strong type; instead it returns an XmlDocument whose contents are loaded from compiled resource within the assembly.

The function takes in the name of the xml that needs to be used to create the constructed message, retrieves the resource from the resources in the assembly, loads it into an xml document and returns it to the caller.

I find that this last approach works best for me in most cases – all the generated xmls are in one place which makes them easy to maintain (which I liked in the xsl option), there’s very little co-ordination that’s required – only the name of the xml file (or any other key one wishes to use) must be known to the caller and the xml resource should match the schema – but as it is stored in one location AS XML this is very easy to achieve and maintain.

Update: I've done some performance comparison between the different methods, the results of which can be read here