Tuesday, January 19, 2010

BizTalk Adapter Pack 2.0 – SAP Adapter IDoc Schema Versioning Part 1

A colleague and I have been working through some anomalies when it comes to SAP IDoc schemas that were generated using the BizTalk Adapter Pack 2.0 SAP Adapter. This is the adapter which is based upon WCF and NOT the .Net Connector based adapter.

When you go to generate an IDoc using the Consume Adapter Service wizard in Visual Studio, you will be presented with a screen like this. You will most likely have a few options when selecting an IDoc and version to use. To be sure that you have selected the right version of IDoc, you should contact your BASIS admin.image

As you select different versions of IDocs, notice that the Node ID changes. This Node ID will soon become your Target Namespace in the XSD schema that you are about to generate.

image

This Target Namespace is very important since BizTalk uses the Target Namespace and root node to determine subscriptions.

image

If you have chosen the wrong version of IDoc you will soon know. You will receive a subscription error indicating that you have received a message that BizTalk does not have a Schema for.

Event Type: Error

Event Source: BizTalk Server 2009

Event Category: BizTalk Server 2009

Event ID: 5719

Date: 1/17/2010

Time: 1:54:31 PM

User: N/A

Computer: Server

Description:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "WcfReceivePort_SAPBinding_IdocZHR_KPV3R710_Custom" URI: "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX?ListenerGwServ=SAPGWXX&ListenerGwHost=SAPSEVER& ListenerProgramId=XXXXXX&RfcSdkTrace=False&AbapDebug=False" Reason: Finding the document specification by message type "http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive" failed. Verify the schema deployed properly.

If you refer to the first image in this blog post, you will see that the Node Id has a value of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//710/Receive#Receive where as BizTalk has received an IDoc with a Namespace of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive. So how can this be? SAP by default generates an IDoc based upon the current release version. But you also have the ability to send a specific IDoc version as configured in the SAP Program ID. This provides some flexibility in the event you need to use a legacy IDoc version.

When SAP sends an IDoc out, it will populate this version information in the control record. I have turned on tracking on the receive port and as you can see this information has been populated by SAP.

image

Since I am using an XML Receive pipeline and SAP has no idea what namespace BizTalk is requiring, the SAP Adapter is generating this namespace at run-time based upon the DOCREL and IDOC_VERSION elements.

The lesson here is to make sure you know what version of the IDoc is currently configured with your Program Id within SAP. I must also caution that while this approach does work, it may not live through your next SAP upgrade. At least not without updating your BizTalk projects. With each SAP major release, you can expect this DOCREL value and potentially the IDOC_Version to increase. So the next time you upgrade your SAP environment, you may get into the same situation that I have described(subscription errors) and it may take some significant refactoring to get your BizTalk schemas and maps back in sync with your SAP version.

In the next post of this series, I will discuss how you can use a Receive Pipeline and Flat File Pipeline Disassembler to override this namespace so that it does not become dependant upon the DOCREL and IDOC_VERSION elements…stay tuned.

10 comments:

Todd Hensley said...

We've been having a lot of "fun" with the WCF SAP Adapter. Like the fact that it wants to generate a new schemas for the IDoc segements and shared types each time you run it, but gives them all the same target namespace and root node name.

For example, if you generate both a "send" and "receive" for MATMAS05, you will get these schemas:

IDoc.CREMAS05.711.3.xsd
IDoc.CREMAS05.711.31.xsd
IDocOperation.CREMAS05.711.3.Receive.xsd
IDocOperation.CREMAS05.711.3.ReceiveResponse.xsd
IDocOperation.CREMAS05.711.3.Send.xsd
IDocOperation.CREMAS05.711.3.SendResponse.xsd
IDocSharedTypes.xsd
IDocSharedTypes1.xsd
Serialization.xsd

The two "IDoc.CREMAS05*" schemas will contain identical target namespaces and root node names. So do the two IDocSharedTypes schemas. BizTalk doesn't like that too much.

So we've had to resort to deleting the 2nd "IDoc.CREMAS05*" schema, having just a *single* IDocSharedTypes.xsd in a referenced project, and editing the other schemas so they import the correct "shared" schemas.

Using the "Generate unqiue schema types" didn't help us, because the .NET type names were not the problem - the XML target namespaces are and it doesn't generate unique target namespaces.

The "Receive" and "Send" schemas use "receive" and "send" as part of the target namespace. If they would have carried that down to the other schemas it generates I suppose we would have been fine.

But we found out the hard way that the WCF *SQL* adapter doesn't like it playing around with the target namespaces it generates, so we're gun-shy about changing the target namespaces of the SAP-generated schemas. Hence our attempt at "sharing" the schemas that are in fact identical. (Today we get to test receiving IDocs from our SAP system).

Also, the adapter generates enumerations for some of the IDoc fields that do not contain values our SAP system is spitting out.

So we are having a real love-hate relationship with the SAP adapter right now.

Kent Weare said...

Thanks for the comment Todd. I haven't run into your scenario yet but that is really good to know ahead of time. None of the IDocs that we currently use are bi-directional.

I do agree with your love/hate relationship with this adapter. WCF is a great technology so we were pretty pumped to be getting off of the old adapter and onto this new one. Once you start running into these gotchas it kinda makes you question the value of making the switch(other than the old adapter currently being deprecated).

We have also had some "fun" with the Oracle and SQL WCF adapters so you aren't alone :-)

Unknown said...

Hi, Ken Weare

Thank you for your web casting and the work you have done is really great.

We have a new consultation project which integrate SAP application using Microsoft Biztalk SAP Adapter.

We do want to recommend Microsoft SAP adapter instead of third party.

Even though we are expert in Biztalk but we have no hands on experience with SAP application.

So our company is looking outside to get a help from an expert like you.

Would you contact me at michelle.lee@qlogitek.com to discuss more with project, please.

Thank you so much.
Michelle

Todd Hensley said...
This comment has been removed by the author.
hxbergen said...

Hello Kent,

great post. Im looking forword to the next post of this topic. Especialy the field DOCREL is interesting. How is it possible to receive IDOCs with XmlReceivePipeline of a specific SAP Version e.g. 46C when used SAP System has Version > 46C e.g. 700?

Kent Weare said...

Thanks for the feedback hxbergen. I don't have specfic details, but you will have to configure this on the SAP side. The Basis administrator will have to configure the Partner Profile to send a specific version of the IDOC that you are interested in receiving. The adapter is just extracting this information from the IDOC Control record so SAP will have to populate thee values to use the correct version that you are interested in.

In my next post(which I hope to releasing soon) I will go through how you can get around version dependencies but you have to use a flat file pipeline.

Stay tuned...

hxbergen said...

I have configured in the partner profile (Transaction WE20) the expected version of the idoc. With this setting a idoc with this version is send from sap, but in the control record still the current version of sap is send.

Kent Weare said...

hxbergen, I will check with our Basis team to see if they know of any other methods.

hxbergen said...

I have discussed this time ago here: http://social.msdn.microsoft.com/Forums/en-US/biztalkr2adapters/thread/28a5d89c-ad0d-4ce6-9c45-c35ecf9fe32b

but no one could help me.

Anonymous said...

Microsft has released a patch
http://hotfix.partners.extranet.microsoft.com/FixDetails.aspx?fixid=421322
[Ask for KB 2388784]
This hotfix introduces a new binding property 'ReceivedIDOCRelese'

This property confirm to the regex - DefaultRelease?(;DOCTYPE1=DOCREL1)*

This property can have the following use case scenarios -
1. When DOCREL contains the actual release of the IDOC - Leave the property blank, DOCREL of incoming IDOC will be treated as the actual release of the IDOC.
2. When all IDOC types received on the Receive location are at specific release(say 640) which is lower than the value of DOCREL in the incoming IDOC –
Set the property to a single integer value (say 640) , the value specified in the property (640) will be treated as release of all IDOC received on that Receive Location .
3. When some of the IDOCs received on the Receive Location are at a lower release (say MATMAS05 is at 620, DEBMAS05 is at 640) and all other IDOCs received on the Receive Location is at the SYSREL(say 710)-
Specify the value in following Syntax : DOCTYPE1=DOCREL1;DOCTYPE2=DOCREL2...
4. When different IDOCs received on the Receive Location are at different release (say MATMAS05 is at 620, DEBMAS05 is at 640 and all other IDOCs received on the Receive Location is at 700), but all of them are lower than the SYSREL(say 710), then you can also specify a default release -
Specify the value in following Syntax : DefaultRelease;DOCTYPE1=DOCREL1;DOCTYPE2=DOCREL2...


Note : Use the CIMTYP in place of the DOCTYP for extended IDOCs.