The lost property

You've created your pipeline component, added a few public properties for good measure (to make it a bit more flexible of course).

You went on to create a pipeline with your component, set all the properties you've introduced and deploy it to BizTalk Server.

You then configure a port to use the pipeline (and the contained component) and test your scenario. all is sweet.

But then - you decide to add one more property to your component.

You quickly go and change the component, adding the public property you've missed, and, if you're anything like me, you decide to skip the whole: update pipeline, remove from port, undeploy, redeploy, re-configure port etc. and simply GAC the updated component planning to set it's value through the admin console to save the pain (and time).

However, when you open the port in the admin console and going to set the newly added property in the pipeline configuration you are into a surprise - your new property does not appear in the UI.

You double check everything - check your code, re-build, re-gac and re-open the admin console, but it's all the same - the property is simply not there. you bang your head against the wall. twice.

And then it hits you - it's not the component! it's the pipeline!

A pipeline "source code" is essentially an XML document. pipeline components, and their properties, are XML fragments within this XML.

When you add a component to the pipeline, its information, including all known properties (and their values) are added to the pipeline's XML.

And yes - you've guessed it right - when you're editing a pipeline's configuration through the admin console, the source of the generated UI you see is that XML, not the actual components' assemblies in the GAC.

If, like me, you have just GAC-ed a new version of the component and went to the admin console to configure it, you're up for a disappointment as your newly added property will simply not be there.

you will be forced to re-deploy the pipeline (unless you are happy to change xmls in the management database, that is - and you shouldn't be).

Extracting values from a message in the pipeline

Another question I was ask recently is how to extract a fields value from a message in a pipeline.

One example I’ve seen goes something like this –

   XmlTextReader xtr = new XmlTextReader("books.xml");
   XPathCollection xc = new XPathCollection();
   int onloanQuery = xc.Add("/books/book[@on-loan]");
   XPathReader xpr  = new XPathReader(xtr, xc);

Then, in order to get the value the stream needs to be read -

while (xpr.ReadUntilMatch())
{        
   Console.Write("{0} was loaned ", xpr.GetAttribute("on-loan"));
}

you can, of course, replace the Console.Write with any action required on the data located, you should also have a check to see exactly which xpath was hit (if you have more then one in the collection)

The problem with this is, as most of you may well know, that it requires that the component reads the entire stream in it's execution.

This, actually, has two disadvantages – one is performance - assuming this is a receive pipeline BizTalk will have to read the stream anyway to write the message to the message box (ina send port the send adapter will do the same). If we could avoid the need to read the stream ourselves, and simply event on the stream as BizTalk’s internals read it we would significantly improve the performance of our pipeline.

Secondly – since we’ve read the stream, we may have a problem now when we go back to return the stream to the pipeline; some streams we might receive are not seekable (such as anything coming from the HTTP or SOAP adapters) and so we can’t simply rewind them and we surely can’t return a stream pointing at the end of the message to the pipeline. It is enough to read Charles Young’s great series of articles about receive pipelines in BizTalk 2004 (http://geekswithblogs.net/cyoung/articles/12132.aspx) to see what sort of issue you might face.

Although some of these issues have since been address the underlying problem remains and that is that by reading the stream we have to then return a “touched” stream to the pipeline which may or may not cause issues, and as we can’t always be sure in what context our component will be used (can you assume send vs. Receive? Can you assume a particular adapter will be used, can you assume port maps will not be used?) we should look for a better way to do this. Luckily such a way exists that helps in most circumstances – BizTalk’s XPathMutatorStream.

Luckily for me Martijn Hoogendoorn already wrote about it a couple of years ago, check his blog entry here I just thought I’d point this out.
Obviously this stream was designed to allow replacement of values, but nothing prevents you from setting the output parameter to the input parameter and thus avoid any changes.

As you can see in the post using this stream means you never actually need to read the message’s stream yourself, you simply need to wrap the original stream with an instance of the xpath mutator stream, add the xpaths you want to the collection and return the wrapped stream with the message. When BizTalk will read the message’s stream it will now be reading your stream which would raise the appropriate event when your xpaths are being hit. Marvellous!

From a performance point of view I have not looked at the implementation of the stream so I can’t say for sure it is faster than reading the stream completely in your pipeline, but my gut feeling says it is, but definitely from robustness perspective this solution is much better as it eliminates all the problems one might encounter by reading a message’s stream in the pipeline.

When is this solution not good – when you need an entire XmlNode. This stream would be able to return a single value (element or attribute I believe), but if you need the entire contents of an XmlNode (with child elements or various attributes) it would not server you well.

Loading custom pipeline component properties and performance

Needing to implement a pipeline component with properties slightly more complext than the odd simple type property I went back to Saravana Kumar's great whitepaper - Understanding Design-Time Properties for Custom Pipeline Components in BizTalk Server.

Before I go any further I should say I think this is a great whitepaper, as are most, is it's definitely worth reading.

I did, however, have one small reservation I thought is worth sharing -

Somewhere around page 21 Saravana shows how to save and load properties which are collections; in his example he uses Xml Serialisation and desrialisation to switch between the in-memory collection to the xml persistable form and back.

Here are the code snippets from the white paper -

public virtual void Load(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, int errlog)
{
object val = ReadPropertyBag(pb, "CorrelationPropertiesCollection");
if (val != null)
{
string corrPropertiesList = (string)val;

XmlTextReader xml = new XmlTextReader(new StringReader(corrPropertiesList));
XmlReaderSettings settings = new XmlReaderSettings();
settings.IgnoreProcessingInstructions = true;
XmlReader reader = XmlReader.Create(xml, settings);

XmlSerializer ser = new XmlSerializer(typeof(CorrelationPropertiesCollection));
CorrelationPropertiesCollection obj = (CorrelationPropertiesCollection)ser.Deserialize(reader);
CorrelationSettings = obj;
}
}

public virtual void Save(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, bool fClearDirty, bool fSaveAllProperties)
{
StringBuilder sb = new StringBuilder();
StringWriter sw = new StringWriter(sb);

XmlWriterSettings setting = new XmlWriterSettings();
setting.OmitXmlDeclaration = true;
XmlWriter writer = XmlWriter.Create(sw, setting);

XmlSerializer ser = new XmlSerializer(typeof(CorrelationPropertiesCollection));
ser.Serialize(writer, CorrelationSettings);

object val = sb.ToString();
pb.Write("CorrelationPropertiesCollection", ref val);
}

The problem with this is that XmlSerialisation is quite an expensive operation, and, quite counter-intuitively (in my view), the Load method is being called in every execution of the component; so having this sort of logic in the load can really slow down the pipeline execution.

With all of that in mind, what I would say - if performance is not a critical issue, doing this is a great clean approach; if, on the other hand, low latency is important, serialisation should be avoided.

In our case we've decided to implement our own string parsing logic.
We didn't go as far as implementing ISerializable or anything like that, but simply added ToString and Parse methods to our collection that converted the collection to a delimited string and back. While this still involves some processing work on the load I suspect it is much quicker than serialisation