Yossi Dahan [BizTalk]

Google
 

Monday, September 07, 2009

On ASMX, WCF, namespaces and generated schemas (in BizTalk 2006)

Recently we’ve started to consume a new version of a web service we’ve been using for a while.
We’ve known that, as a whole, not much had changed, only that they have now moved to WCF; they would have migrated their classes to VS 2008 but would expose pretty much the same functions, using pretty much the same parameters.

Still – it appears that BizTalk now insists on generating multiple schemas for the web reference, and as more of the service is moved across more schemas are introduced.

This caused Oleg a fair amount of pain as, when new schemas would be introduced, they would re-order the existing schemas, so reference1.xsd (in the web reference) would suddenly become reference2.xsd, which in turn break out maps.

The process of finding out the logic behind which schemas are created was fairly short and simple, but as I’ve documented it I thought I might as well share it -

Initial observation revealed that whilst the ASMX services’ WSDL file contains all the schemas needed, the WCF services using import statements in the WSDL file; the schemas exist in separate ‘files’.

The ASMX services always uses the XmlSerializer, WCF services use the DataContractSerializer by default, but can be configured to use the XmlSerializer if required.

Here’s a walk thorugh of the scenarios we’ve compared (using BizTalk 2006) -

Standard WCF project, DataContractFormat
We’ll start by comparing the standard WCF sample project generated when you create a new WCF service application in Visual Studio 2008 -

[ServiceContract]
public interface IService1
{
[OperationContract]
string GetData(int value);

[OperationContract]
CompositeType GetDataUsingDataContract(CompositeType composite);
}

[DataContract]
public class CompositeType
{
bool boolValue = true;
string stringValue = "Hello ";

[DataMember]
public bool BoolValue
{
get { return boolValue; }
set { boolValue = value; }
}

[DataMember]
public string StringValue
{
get { return stringValue; }
set { stringValue = value; }
}
}



Looking at the WSDL generated, 3 schemas are imported –

1.       The usual generic types


2.       The definition of the compositeType type


3.       The definition of the service’s messages (GetData, GetDataResposne, GetDataUusingDataContarct, GetDataUsing DataContractResponse)





Adding a web reference to this service from a BizTalk 2006 project we can see it represents this fairly accurately -


We can see all 3 schemas downloaded from the service, but within the reference.map generated code  a single reference.odx defined the methods in the form of ports and web-messages, and reference.xsd defineds the compositeType schema.



Equivalent project in an ASMX service

I’ve created an equivalent ASMX service, which looks like this –



[WebService(Namespace = "http://tempuri.org/")]
public class Service1 : System.Web.Services.WebService
{

[WebMethod]
public CompositeType HelloWorld(CompositeType composite)
{
CompositeType response = new CompositeType();
response.StringValue = "Hello World";
return response;
}
}

public class CompositeType
{
bool boolValue = true;
string stringValue = "Hello ";

public bool BoolValue
{
get { return boolValue; }
set { boolValue = value; }
}

public string StringValue
{
get { return stringValue; }
set { stringValue = value; }
}
}



Publishing this service I can see it’s WSDL contains (does not use import, but that proved to be insignificant) a single schema that represents the service’s messages and the compositeType definition.



Consume this service from a BizTalk 2006 project and only the WSDL file is downloaded (there are no ‘external schemas’ to worry about) but within the reference.map pretty much the same odx and xsd files are generated, no real difference between ASMX and WCF here.



Next I’ve looked at changing the serializer the WCF service works with from DataContract to XmlSerializer –



Standard WCF project, XmlSerializerFormatter

Now we will change the serializer to XmlSerializer  by adding XmlSerializerFormatAttributre to both the service and the data contracts  



   [ServiceContract]
[XmlSerializerFormat]
public interface IService1
{


…and



[DataContract]
[XmlSerializerFormat]
public class CompositeType
{


The WSDL in this case includes only one import, for a single schema representing both the service messages and the compositeType schema (basic types are not exposed) and BizTalk now only has one schema downloaded, but again – the reference.map code remained identical (one ODX, one schema)



How will adding a second namespace affect these behaviours? Lets investigate -



WCF project, two namespaces DataContractFormat

To demonstrate this I’ll add another data contract - AnotherCompositeType, specify an explicit namespace for it and include it as a second parameter to the GetDataUsingDataContract operation -



[DataContract(Namespace="HttpL://SomeNamespace")]
public class AnotherCompositeType
{
bool boolValue = true;
string stringValue = "Hello ";

[DataMember]
public bool BoolValue
{
get { return boolValue; }
set { boolValue = value; }
}

[DataMember]
public string StringValue
{
get { return stringValue; }
set { stringValue = value; }
}
}

[OperationContract]
CompositeType GetDataUsingDataContract(CompositeType composite, AnotherCompositeType anotherComposite);



Using DataContractFormat again, but with two classes, representing two different namespaces, we’re now getting yet another schema - the fourth one - representing the added data contract (if the namespaces of both data contracts were the same, the DataContractFormat would have included them in the same schema)



On the BizTalk side, the reference.map code now also contains a second schema, one describes the original CompositeType, and a second describes the second type – AnotherCompositeType and here as well – were the two types in the same namespace, a single schema would exist, describing both.



Let’s look at the same again, using the XmlSerializerFormat



WCF project, two namespaces XmlSerializerFormat

Adding the XmlSerializerFormat, I also have to remember to include the XmlRoot attribute to set the namespace, as the serializer does not look at the DataContract attribute -



[DataContract(Namespace = "http://SomeNamesapce")]
[XmlSerializerFormat]
[XmlRoot(Namespace = "http://SomeNamesapce")]
public class AnotherCompositeType
{
bool boolValue = true;
string stringValue = "Hello ";

[DataMember]
public bool BoolValue
{
get { return boolValue; }
set { boolValue = value; }
}

[DataMember]
public string StringValue
{
get { return stringValue; }
set { stringValue = value; }
}
}



Now the WSDL for this service, using the XmlSerialiserFormat imports two schemas – one for the service messages, and the CompositeType schema, all reside in the same namespace, and a second for AnotherCompositeType which is defined in a separate namespace



Consuming this from BizTalk and again I’m getting two schemas – one for each namespace.

So far – switching between DataContractFormat and XmlSerializerFormat made no difference to the generated code under reference.map, but it did change the way the WSDL is constructed (import vs. embededed schemas) and therefore the downloaded components (wsdl and schemas, vs. wsdl only)



Note - another thing I’ve noticed is that when a new schema needs to be generated under the reference.map code, as a result of a change to the service, updating web reference does not seem to do so; I had to delete the web reference and re-add it to see the newly added schema.



Last  - let’s look at how the ASMX service behaves with two namespaces –



ASMX service with two namespaces

I’ve added the second class, and added it as a parameter to my web method



[XmlRoot(Namespace = "http://AontherNamespace")]
public class AnotherCompositeType
{
bool boolValue = true;
string stringValue = "Hello ";

public bool BoolValue
{
get { return boolValue; }
set { boolValue = value; }
}

public string StringValue
{
get { return stringValue; }
set { stringValue = value; }
}
}


And, still consistent – when consumed from BizTalk 2006 I’m getting only the WSDL downloaded (two schemas are embedded) but the reference.map code contains two schemas – one for each namespace.



To summarise -



Using the DataContractFormat you will always get one schema for generic types, one schema for the service’s messages, and then one schema for each namespace any other types are declared in (0..n)



Using the XmlSerializerFormat  schemas are embedded in the WSDL file, and you would get one per xml namespace used.



As far as BizTalk generated code is concerned, however, there’s no difference between the two.



 



What this meant to us – well – we understand better, but there’s still not much we can do.



In our case – we control the service, and – in fact - we know that the only reason we encounter multiple xml namespaces in the service contract is because the various classes exist in several .net namespaces, and they have not supplied the DataContract attribute on any class, they have certainly no supplied the namespace parameter to that attribute, which meant the .net namespace was used as the xml namespace, resulting with multiple namespaces and therefore multiple schemas.



One that team added the attribute, and used a consistent xml namespace throughout, our immediate problem was solved; however - had it been a third party’s service we would not have that luxury and we would have had to update our code whenever we update our web reference, even if only new types were added in a backwards compatible way (as the schema ordering may have changed)



On that – it’s probably easier to simply rename the schemas (and the underlying .net types) under the reference.map generated code rather than the referencing maps and messages.


Labels: , ,

Wednesday, March 18, 2009

ASMX, WCF and enums woes

We’ve been slowly migrating our services from asmx to WCF, but as we’re still using BizTalk 2006 with no support for WCF we’ve been exposing endpoints configured for basicHttpBinding and consume them using the SOAP adapter.

Generally speaking things have been going well, although we completely gave up on the idea of moving the services to WCF and NOT have to change the client, until yesterday we’ve stumbled into a serialisation issue –

The SOAP adapter, as part of its work deserialises the request message arriving through the send port into t he web service proxy class it generated, before calling the web service (which would result in the class now being serialised back into xml, which is another story); that deserialisation failed.

The error message was clear enough and indicated it failed to deserialise an enum parameter the service was expecting, and that ran a bell – I posted on exactly that back in September, but after carefully checking and re-checking everything we could swear that our message (which was now suspended) matches perfectly the schema generated by the add web reference wizard; what’s going on then??

After chasing our tail for a short while we brought up reflector to the rescue and found out the cause of our woe is a combination of a difference in behaviour between WCF and ASMX and the use of BizTalk – here are the details –

Consider the following asmx web method  –

[WebMethod]
public string GetDataUsingDataContract(CompositeType.someEnum myEnum)
{
return "Hello World";
}

With CompositeType being

public class CompositeType
{
public enum someEnum
{
Value1,
Value2
}
}

(..and pretend CompositeType has many more things, but these are irrelevant to this topic)


The definition for myEnum in the WSDL looks like

<s:element minOccurs="1" maxOccurs="1" name="myEnum" type="tns:someEnum" />

Where the type tns:someEnum looks like

<s:simpleType name="someEnum">
<
s:restriction base="s:string">
<
s:enumeration value="Value1" />
<
s:enumeration value="Value2" />
</
s:restriction>
</
s:simpleType>

As a result the definition of the enum in a proxy generated via the add web reference VS 2005 option (which is what BizTalk would use) looks like –

[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "2.0.50727.3053")]
[System.SerializableAttribute()]
[System.Xml.Serialization.XmlTypeAttribute(Namespace="http://tempuri.org/")]
public enum someEnum
 {
Value1,
Value2,
}

All makes sense.


Now, let’s look at what WCF does in the same case; consider the following service –

[ServiceContract]
public interface IService1
{

[OperationContract]
string GetDataUsingDataContract(CompositeType.someEnum myEnum);
}

[DataContract]
public class CompositeType
{

public enum someEnum
{
Value1,
Value2
}
}

The WSDL generated looks like

<xs:simpleType name="CompositeType.someEnum">
<
xs:restriction base="xs:string">
<
xs:enumeration value="Value1" />
<
xs:enumeration value="Value2" />
</
xs:restriction>
</
xs:simpleType>
<
xs:element name="CompositeType.someEnum" nillable="true" type="tns:CompositeType.someEnum" />

The key difference is that the name of the class containing the enum has made it into the type name for the enum, which never happened in the ASMX version.


As a result the proxy is generated as such -

[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Runtime.Serialization", "3.0.0.0")]
[System.Runtime.Serialization.DataContractAttribute(Name="CompositeType.someEnum",
Namespace="http://schemas.datacontract.org/2004/07/WcfService1")]
public enum CompositeTypesomeEnum : int
{
[System.Runtime.Serialization.EnumMemberAttribute()]
Value1 = 0,
[System.Runtime.Serialization.EnumMemberAttribute()]
Value2 = 1,
}

Again – note the name given to the element now contains the class name and, crucially, a dot (‘.’).


On it’s own – nothing to malicious – although it’s another nail in the coffin for the idea that you can substitute web service with WCF service, configured them to use basicHttpBinding and all should be the same (ok – am I the only one still wishing this was possible?)


Enters BizTalk.


When you use the add web reference wizard to add a reference to the WCF service, BizTalk generates all the schemas and proxy for you, which is what you would use to create requests going to the service (and process responses).


Because the WSDL of the WCF service contains the longer name of the enum (with the class name, the dot and the enum name) the .net proxy generated is identical to the one created for the WCF service above; the schema, however, is generated incorrectly!


BizTalk “kindly” decides that having dots in the element name is not a good idea and removes it so the schema generated looks like this –

<xs:schema xmlns:tns="http://schemas.datacontract.org/2004/07/WcfService1" elementFormDefault="qualified" 
targetNamespace="http://schemas.datacontract.org/2004/07/WcfService1"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="CompositeTypesomeEnum" type="tns:CompositeTypesomeEnum" />
<
xs:simpleType name="CompositeTypesomeEnum">
<
xs:restriction base="xs:string">
<
xs:enumeration value="Value1" />
<
xs:enumeration value="Value2" />
</
xs:restriction>
</
xs:simpleType>
</
xs:schema>

“CompositeTypesomeEnum”??????


Well, we’ve seen this, and created a message with exactly that element, which – of course – the SOAP adapter failed to deserialise into

[System.Runtime.Serialization.DataContractAttribute(Name="CompositeType.someEnum", 
Namespace="http://schemas.datacontract.org/2004/07/WcfService1")]
public enum CompositeTypesomeEnum : int
{
[System.Runtime.Serialization.EnumMemberAttribute()]
Value1 = 0,
[System.Runtime.Serialization.EnumMemberAttribute()]
Value2 = 1,
}


The solution was fairly simple – we’ve simple change our xsl to put the element name as the .net proxy requires it, and not as the schema describes it, and it all worked well.

Labels: , ,

Tuesday, September 23, 2008

Calling a web service with an enumeration parameter from BizTalk Server

This was waiting in the drawer for ages now and somehow never quite made it onto the blog...

Most of our web service's methods take one enum or another, which generally works well, but it did take us a bit of experimentation to figure out how to call them from a BizTalk orchestration; here are the details -

Take for example this version of a HelloWorld web service: 

[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
public class Service1 : System.Web.Services.WebService
{
   [WebMethod]
   public string HelloWorld(MessageType message)
  {
     return "Hello World; " + message.ToString();
   }

   [System.Xml.Serialization.XmlType
       (AnonymousType=false,Namespace="http://MyNamespace")]
   public enum MessageType
   {
     Hello,
     GoodBye
   }
}

(I’ve made the enum an anonymous type and given it a namespace to represent real world scenarios from my experience, see http://www.sabratech.co.uk/blogs/yossidahan/2007/02/anonymoustypes-and-serialization.html )

Looking at the test page for this web service you get an example request that looks like this –

POST /Service1.asmx HTTP/1.1
Host: localhost
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: "http://tempuri.org/HelloWorld"
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">  <soap:Body>
    <HelloWorld xmlns="http://tempuri.org/">
      <message>Hello or GoodBye</message>
    </HelloWorld>
  </soap:Body>
</soap:Envelope>

Notice how the parameter name is message (as the name of the parameter of the method?)

And indeed if I use HttpAnalyzer to examine the traffic “on the wire” I can see the request message going through as –

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <HelloWorld xmlns="http://tempuri.org/">
      <message>GoodBye</message>
    </HelloWorld>
  </soap:Body>
</soap:Envelope>

Which makes perfect sense.

Moving to a BizTalk project - I add a web reference from my BizTalk project to generate the proxy i.e port type, schemas, message types etc.

The request message-type generated has a part for my enum, as expected, however, unlike what I'd expected,  to pass an enum value to the web service, you don't wrap the enum value in the parameter name as it appears on the wire (and in the test page), which would have been -

      <message>Hello</message>

but instead the value is wrapped in an element that matches the enum type definition, as such -

<ns0:MessageType xmlns:ns0="http://MyNamespace">Hello</ns0:MessageType>

This is quite obvious once you look at the schema that got generated -

<xs:schema xmlns:tns="http://MyNamespace" elementFormDefault="qualified" targetNamespace="http://MyNamespace" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="MessageType" type="tns:MessageType" />
  <xs:simpleType name="MessageType">
    <xs:restriction base="xs:string">
      <xs:enumeration value="Hello" />
      <xs:enumeration value="GoodBye" />
    </xs:restriction>
  </xs:simpleType>
</xs:schema>

but I just didn't bother initially, expecting to need to build the parameter the way it appears on the test page.

So in order to call the web service from a BizTalk orchestration you would need to create a message with the correct xml in an expression shape;

I use (with doc being a variable of type XmlDocument) -

doc = new System.Xml.XmlDocument();

doc.LoadXml("<ns0:MessageType xmlns:ns0=\"http://MyNamespace\">Hello</ns0:MessageType>");

and then I assign it to the request

WSRequest.@message =   doc;

(where WSRequest is a message of the web message type generated as part of the add web reference procedure, and @message is the part that was created to represent the enum parameter)

Labels: , ,

Saturday, September 13, 2008

Yet another example for a service's / xml bad design?

One of my clients have been using this 3rd party's software for some time now (I won't name neither organization for obvious reasons).

In their organisation both BizTalk and this software are key components and so they both play part in many scenarios.

Unfortunately, and quite surprisingly, this 3rd party's software doesn't have have any adequate support for integration and the main way to interact with it is pushing some data into 'import tables' in the software's database and then running some EXE to process them. quite horrendous.

To make matters worse this EXE can only allowed to run once at any single time so most implementations require some form of queuing and singleton pattern.

Recently that third party released a version with *some* support for web services, and so, excited by the great news the development team went on to implement a process using the newly introduced web service.

The service that had been exposed relates to the product's reporting features - users can create custom reports using the product's UI and then, using web services, export the result of these queries.

I would have expected, based on past experience, a generic schema in the WSDL that would describe a report's output, irrespective of the fields returned, something like -

<row>

<column fieldName=".."></column>

<column fieldName=".."></column>

.

.

</row>

But the vendor's schema included some generic field from their domain (report name, time of execution, etc.) and then rows and fields based on the specific report generated.

Because the report's structure is not known at design time its fields do not exist in the WSDL, instead a report node exist with an xs:any declaration to include the actual report's contents.

Personally I prefer to avoid xs:any if possible, and I think a schema describing a generic report's result could have been created, but that's not the main problem; the main problem, in my view, is that the fields added underneath the report element (as well as the report element itself), which are generated by the application, all belong to the same namespace.

Because they could not predict the names report designers will use for fields it was more than possible to create an element with duplicate meaning which is a bad idea overall and also causes quite a bit of headache when one needs to create schemas for BizTalk to accurately describe the response of such a service.

One thing I will grant them is that they have though of defining LAX validation for the report, so duplicate elements will not cause validation errors.

Labels: , ,

Thursday, December 27, 2007

The message box as a service boundary

For the last 18 months or so I've been working on a very exciting, and quite large, BizTalk implementation here in the UK, I'll leave the full details of it for now, but I can tell you that it involves all the nice buzzwords we keep hearing about SOA, SaaS, S+S, ESB at least to some degree (and with various level of quality, if we're honest)

Anyway, as you can imagine we're using web services quite extensively - we expose a lot of them, and consume even more; some are internal to the company (but cross teams, although not so much platforms) and many are external (which do cross platform as well)

The reasons to use service oriented architecture should be very clear to everyone by now, as are the famous four tenants of SOA.

In out implementation we've abstracted the calls to all the internal web services through utility orchestrations which would take a message in our canonical format , convert it to the service's format, call the web service and transform the response to the canonical format before returning it to the calling process; this way we can re-use those transformations, and have a central place to deal with each request, apply error handling, etc.

From the parent process we then use call orchestration to initiate these utility orchestrations passing in the request cannonical message and receiving the response canonical message as an out param, which is quite efficient (when initiating orchestrations through the call orchestration shape the request does not go through the message box)

As we're doing the transformations in these utility processes, we consider them to be in the boundary of our process, and not, obviously, within the boundary of the called service, for this reason we call the web service from the process rather than the actual assembly behind the web service.

When we, within the utility process call the web service, what actually happens is that the request message (now in the WS' format) gets published to the message box, being picked up by the send port which would pass it to the SOAP adpater which, in turn will serialises it and transmits it over the wire to the service; the service then deseralises the message on the other end before executing whatever code needs to be executed and the entire process now repeats in the opposite direction.

In this case the service boundary is the web service endpoint.

A few weeks ago I had what I thought was a brilliant idea - why not treat the message box as the service boundary!?

If I had a process that takes in the service's format of the message using a directly bound receive shpae and a filter, execute the code internally (as we're now inside the service boundary we can use the service code directly from expression shape, no need to go through a web service) and when finished publish the response back to the message box (in it's own format), I could have simply published a request message for that service, and get the response published back for me; correlation should be used, but this can be handled using self-correlating ports or a correlation set.

The client process would do pretty much the same - it would use a utility process to transform the canonical format to the service's format and publish the request. it would then use correlation to receive the response and transform it back to the canonical format before retunrning it to the calling process(synchronously).

What would we save? - following this approach for at least some of our internal services can save us the need to serialise the messages over the network; in the web service case we have to go through the message box from the process to the send port anyway, so going through the message box from one process to aonther would not make a difference, but all the network traffic and the work by the SOAP adapter (which is far from being efficient) can be saved.

This was a good idea (I thought anyway), but I suspect it won't work, as it has two main flaws (and I will be extremely happy to get some ideas around those) -

Firsly - both subsystems will need to exist on the same BizTalk group so that they share the same message box and so we could use pub/sub to exchange messages between them (on it's own this is not necessarily a problem, but it is the main cause for the next one, which is the big one)

Secondly - the schemas will have to be shared -

When you're adding a web reference to a web service from a standard .net project a proxy gets generated for you; that proxy will include a local version of all the classes used by the web service (these will be in YOUR code namespace rather then ther service's but will serialize to the same XML).

Equally - when you add a web reference in a BizTalk project, you get schemas generated so you can create messages to send and receive to/from the web service; these will be in the service's XML namespace as they have to represent the XML supported by it, and here lies the problem.

If both the service implementation and the client implementation are on the same BizTalk group, the schemas will have to be shared as there's no way to deploy two schemas using the same root node and namespace and we all know that sharing schemas is a bad idea as it strongly couples the implementation together and that pretty much renders the idea useless (this, confusingly I suspect, means we're sharing a class and not a contract).

Of course one could play around with the idea of having two BizTalk groups and communicating between them, and although you can choose better transports than SOAP for that internal communication I suspects that brings us closer to simply calling the web service and so I'd rather stay with that standrad approach.

Labels: , ,

Thursday, September 20, 2007

Limitations of web services support in BizTalk

Are listed in MSDN under the somewhat confusing title "Considerations When Consuming Web Services"

We were hit by the last one this week -

The Add Web Reference do not support the Web Services Description Language (WSDL) import element

The Add Web Reference fails when you add Web references for the WSDL file with the import element.


It didn't quite fail, but it did not import all the schemas it should have (it did some)

Labels: ,

Friday, September 07, 2007

Another dive into the SOAP adapter's behaviour

Being keen to fine tune our solution's performance we turned to look into the message exchanges between the SOAP adapter and one of the web services we're usign quite extensively.

Two things became obvious very quickly (well, to be fair - they may have not been without some very useful tips from our colleagues working for the 3rd party providing that service) -

  • All request messages we've sent specify the "Expect: 100-continue" HTTP header
  • Neither of the request set out use pre-authentication, although we explicitly set credentials on the send port

so we promptly turned to look into them -

100-continue

I'm no network expert; I hold only the basic understanding of HTTP you'd expect from someone dealing with messaging and integration; the way I understand it then is that if a requester sets the "Expect:100-continue" HTTP header on a request, the client would sends the HTTP headers only to the server and then wait to receive a response with "100-continue" status.

It is only when the "100-continue" status is returned from the server that the client sends the actual request.

I also understand that, in order to support older implementations of HTTP (or those that do not support 100-continue), if the client does not receive 100-continue it will send the request anyway, but only after the delay introduced while waiting for the server to respond.

What all that means is about 200ms delay to every message sent from BizTalk.

I'm sure Microsoft have given a lot of consideration to this before deciding this would be the default behaviour of web services in .net but to us, communicating with a long-term partner over a good leased line, it would have been nice to be able to turn it off and skip the extra step.

Unfortunately, although spending some time on this we could not, as of now, find a way to do so.

Our best bet was (and still is) to set the relevant setting in the ServicePointManager in our custom web service proxy, but this does not seem to have the desired effect.

From a bit of reading we've done it seems there's some element of re-use here and that the ServicePointManager may use an existing service point, in which case our newly set property will not be used.

It could be that this is created somewhere in the adapter before our custom proxy or something like that, we truly don't know at this point.

Pre-Authenticate

Fortunately this is an area we’ve had better success in, and it's good because it has a significantly bigger impact on performance than the 100-continue problem.

The web service I've mentioned before uses basic authentication on the wire and then further elements of security in the message, so on the BizTalk port we've set the credentials the SOAP adapter should use when calling this web service, and, as you'd expect, this is working just fine.

However, analysing the traffic going out of BizTalk on the wire we've noticed that for each message we send to the 3rd party, the SOAP adapter would transmit the message out without the security HTTP headers first, receive an HTTP 401 (Unauthorised) error and only then send the request with the HTTP headers baring the credentials we've set in the send port.

Obviously, and especially when relatively large messages are involved, this has a significant impact on the performance of the server (and the farm as a whole as more bandwidth is used for the message exchange).

Again, doing some reading and experimenting, we've learnt that this is pretty much down to the choices made around the DEFAULT behaviour of .net, and again, we thought, we had no control over it.

But, while we could not do anything if we were using the standard configuration of the SOAP adapter, the fact that were using a custom web proxy to call the web service (for other reasons) meant we could try and tweak the way messages were transmitted, and tweaking we did.

It didn't help us in the 100-continue case, but it definitely helped with the pre-authentication requirement - all we had to do is add a line setting the PreAuthenticate property of the HttpWebRequest to true before using it and BizTalk (well, the .net framework web service stack it uses) would quite happily provide the credentials up front avoiding the need to duplicate the round trip to the 3rd party server.

Much better now!

Labels: , , , ,

Thursday, September 06, 2007

Unnecessary limitation when using custom proxy for web services in BizTalk 2006

We're using a third party web service quite extensively; sometimes the size of messages exchanged between us can be quite big, which is not ideal from a performance point of view.

Luckily - they are quire advanced and mature and have a good support for compressed messages exchange.

To use that we needed to compress the stream going out over the wire through the adapter and set the Content-Encoding HTTP header to "gzip" to indicate the stream is compressed, on their end, if the header is set that way, they would decompress the stream before processing the message , and will compress the response in a similar way.

They were even kind enough to provide us with a class that enhances System.Web.Services.Protocols.SoapHttpClientProtocol (through inheritance) to do all of that, so really - all we needed to do to use their web service in our playground windows application was to change the proxy class generated by Visual Studio when we added the web reference to inherit from their class rather than SoapHttpClientProtocol (their class would ultimately ingerit from SoapHttpClientProtocol).

We've played with it for a while and it worked great, which meant we were ready to move it from our playground application to our BizTalk implementation; As you can guess by the fact I'm bothering with this post - it didn't go smooth…

I've played with custom web service proxies before - generated a proxy from a WSDL and configured the port to use it - so I knew that in general principle that works quite nicely;

And so I went on and generated a proxy class for that web service and changed the inheritance to the class they've provided.

When I tried to select it in the Admin Console as the proxy class to use, I could browse to the assembly I've built but, once selected, I would get the following error:



This did not make sense to me - the proxy I generated inherits from their class which, as I have the source code, I'm sure inherits from SoapHttpClientProtocol. This error did not make sense to me.

Trying to figure this one out I went on to create my own class ("TestClass") that inherits from SoapHttpClientProtocol (and had not other implementation otherwise) and changed the generated proxy to inherit from that - so basically I created the same scenario - proxy inherits from class inherits from SoapHttpClientProtocol.

When I tried to select my updated assembly in the admin console I could not select the original proxy (now inheriting from TestClass), I could, however, select TestClass itself (which inherits directly from SoapHttpClientProtocol ) but only to get another error -



This error was quickly solved by adding a public web method to TestClass; now TestClass could be selected in the UI, my proxy class, however, the one I really needed, that does not inherit directly from SoapHttpClientProtocol, could not be selected.

It has also solved the mistery, or so I I believe - it seems the UI will only display classes whose base type is SoapHttpClientProtocol, when it should havem in my view, displayed all classes whose IsSubclassOf(typeof(SoapHttpClientProtocol)) is true.

For us the solution was simple - we took all the code we were kindly given from the 3rd party and merged it into our proxy class which could then be left to inherit directly from SoapHttpClientProtocol . The problem is that this is quite limiting - there's no real scope for re-use and for no good reason.

In fact - I suspect this is just a design time problem, of the kind we spotted quite often recently where the runtime would actually work with something, but you have no way of configuring BizTalk to use it.

Labels: , ,