In order to extract the last record, I utilized the Record Count functoid and fed that value into the Index functoid. The Index functoid is also fed with the value from the "WA" element that I am really interested in. The result is then passed to a String Concatenate functoid where the data is enriched before being assigned to the destination document.
A colleague and I have been working through some anomalies when it comes to SAP IDoc schemas that were generated using the BizTalk Adapter Pack 2.0 SAP Adapter. This is the adapter which is based upon WCF and NOT the .Net Connector based adapter.
When you go to generate an IDoc using the Consume Adapter Service wizard in Visual Studio, you will be presented with a screen like this. You will most likely have a few options when selecting an IDoc and version to use. To be sure that you have selected the right version of IDoc, you should contact your BASIS admin.
As you select different versions of IDocs, notice that the Node ID changes. This Node ID will soon become your Target Namespace in the XSD schema that you are about to generate.
This Target Namespace is very important since BizTalk uses the Target Namespace and root node to determine subscriptions.
If you have chosen the wrong version of IDoc you will soon know. You will receive a subscription error indicating that you have received a message that BizTalk does not have a Schema for.
Event Type: Error
Event Source: BizTalk Server 2009
Event Category: BizTalk Server 2009
Event ID: 5719
Date: 1/17/2010
Time: 1:54:31 PM
User: N/A
Computer: Server
Description:
There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "WcfReceivePort_SAPBinding_IdocZHR_KPV3R710_Custom" URI: "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX?ListenerGwServ=SAPGWXX&ListenerGwHost=SAPSEVER& ListenerProgramId=XXXXXX&RfcSdkTrace=False&AbapDebug=False" Reason: Finding the document specification by message type "http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive" failed. Verify the schema deployed properly.
If you refer to the first image in this blog post, you will see that the Node Id has a value of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//710/Receive#Receive where as BizTalk has received an IDoc with a Namespace of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive. So how can this be? SAP by default generates an IDoc based upon the current release version. But you also have the ability to send a specific IDoc version as configured in the SAP Program ID. This provides some flexibility in the event you need to use a legacy IDoc version.
When SAP sends an IDoc out, it will populate this version information in the control record. I have turned on tracking on the receive port and as you can see this information has been populated by SAP.
Since I am using an XML Receive pipeline and SAP has no idea what namespace BizTalk is requiring, the SAP Adapter is generating this namespace at run-time based upon the DOCREL and IDOC_VERSION elements.
The lesson here is to make sure you know what version of the IDoc is currently configured with your Program Id within SAP. I must also caution that while this approach does work, it may not live through your next SAP upgrade. At least not without updating your BizTalk projects. With each SAP major release, you can expect this DOCREL value and potentially the IDOC_Version to increase. So the next time you upgrade your SAP environment, you may get into the same situation that I have described(subscription errors) and it may take some significant refactoring to get your BizTalk schemas and maps back in sync with your SAP version.
In the next post of this series, I will discuss how you can use a Receive Pipeline and Flat File Pipeline Disassembler to override this namespace so that it does not become dependant upon the DOCREL and IDOC_VERSION elements…stay tuned.
I had a requirement where I needed to upload images to a SharePoint library and use the information from the image filename to populate columns within the document library. Performing these tasks within an orchestration is pretty trivial as you can set context properties in a Message Assignment shape that will drive the behavior of the WSS Adapter. I couldn’t justify performing these tasks in an Orchestration since it would involve an extra hop to the MessageBox in order for Orchestration to be invoked.
Another option is to do all of this work in a Pipeline component. I can create a Send Port subscription that would allow me to access the context properties from the message received and update them to include the WSS properties all within in a pipeline. This would allow the solution to become a pure messaging solution and I can save an extra Message Box hop. When building this solution, I referenced Saravana Kumar’s white paper on Pipeline Components. It came in handy, especially in the area of creating the Design time pipeline properties.
Within the Pipeline Component, the first thing that I wanted to to was retrieve the source file name. I am able to do this by reading the “ReceivedFileName” property from the File Adapter’s context properties. I then wanted to clean this file name up since the value of this property includes the entire UNC path: \\Servery\RootFolder\SubFolder\filename.jpg. I have written some utility methods to parse the file name from this long string.
There are 3 parts to the image file name(1213455_NEW_20100110120000.jpg) that I am particularly interested in. They include an Asset Number, Asset State and the Date/Time that the image was taken. The scenario itself is a field worker who needs to capture an image of an asset, indicate the Asset Number and its state i.e. New/Old. This information is then captured in the name of the image. Since it is an image, there is no other reasonable way to store this meta data outside the file name. This in itself is the reason why I need to to use a pipeline component. Standard WSS adapter functionality includes the ability to use XPATH statements to extract data from the message payload and populate a SharePoint document library column.
Once I have captured this meta data and massaged it to my liking, I want to then provide this context data to the WSS Adapter. The WSS adapter is a little different than most of the other adapters in that you can populate an XML document and push that into the ConfigPropertiesXml context property. The document structure itself is a flat XML structure that uses a “key-value” convention.
<ConfigPropertiesXml><PropertyName1>Column 1 Name</PropertyName1><PropertySource1>Column 1 Value</PropertySource1><PropertyName2>Column 2 Name</PropertyName2><PropertySource2>Column 2 Value</PropertySource2><PropertyName3>Column 3 Name</PropertyName3><PropertySource3>Column 3 Value</PropertySource3></ConfigPropertiesXml>
I also want to populate the WSS Adapter’s Filename context property. I can achieve this by the following statement:
pInMsg.Context.Write("Filename",
Below is my Execute method, in my pipeline component, where all of this processing takes place. You can download the entire sample here. This code is at a proof of concept stage so you will want to evaluate your own error handling requirements. Use at your own risk.
public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
{
//Get received file name by retrieving it from Context
string FilePath = pInMsg.Context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties") as string;
//strip path from filename
string ImageFileName = GetFileNameFromPath(FilePath);
//Utility methods to parse filename
string msgAssetNumber = GetAssetNumberFromFileName (ImageFileName);
string msgAssetState = GetAssetStateFromFileName(ImageFileName);
string msgImageDateTime = GetImageDateFromFileName(ImageFileName);
//Write desired file name to context of WSS Adapter
pInMsg.Context.Write("Filename",
"http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", ImageFileName);
// Populate Document Library Columns with values from file name
string strWSSConfigPropertiesXml = string.Format("<ConfigPropertiesXml><PropertyName1>{0}</PropertyName1><PropertySource1>{1}</PropertySource1>" +
"<PropertyName2>{2}</PropertyName2><PropertySource2>{3}</PropertySource2><PropertyName3>{4}</PropertyName3><PropertySource3>{5}</PropertySource3></ConfigPropertiesXml>",
this.AssetNumber,msgAssetNumber,this.AssetState,msgAssetState,this.ImageDateTime,msgImageDateTime);
pInMsg.Context.Write("ConfigPropertiesXml", "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", strWSSConfigPropertiesXml);
return pInMsg;
}
A feature that I wanted to provide is the ability to provide the SharePoint column names at configuration time. I didn’t want to have to compile code if the SharePoint team wanted to change a column name. So this is driven from the pipeline configuration editor. The values that you provide (on the right hand side) will set the column names in the ConfigPropertiesXml property that is established at run time.
If you provide a value in this configuration that does not correspond to a column in SharePoint, you will get a warning/error on the Send Port.
Event Type: Warning
Event Source: BizTalk Server 2009
Event Category: (1)
Event ID: 5743
Date: 1/10/2010
Time: 7:38:19 PM
User: N/A
Computer:
Description:
The adapter failed to transmit message going to send port "SendDocToSharePoint" with URL "wss://SERVER/sites/BizTalk%%20Repository/Inbound%%20Documents". It will be retransmitted after the retry interval specified for this Send Port. Details:"The Windows SharePoint Services adapter Web service encountered an error accessing column "Missing Column" in document library http://SERVER/sites/BizTalk%%20Repository/Inbound%%20Documents. The column does not exist. The following error was encountered: "Value does not fall within the expected range.".
This error was triggered by the Windows SharePoint Services receive location or send port with URI wss://SERVER/sites/BizTalk Repository/Inbound Documents.
Windows SharePoint Services adapter event ID: 12295".
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
The end result is that I can use information contained in the file name to populate meta data columns in SharePoint without an Orchestration.
Also note, I have built this pipeline component so that it can be used in a Receive Pipeline or as Send Pipeline by including the CATID_Decoder and CATID_Encoder attributes.
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[ComponentCategory(CategoryTypes.CATID_Decoder)]
[ComponentCategory(CategoryTypes.CATID_Encoder)]
[System.Runtime.InteropServices.Guid("9d0e4103-4cce-4536-83fa-4a5040674ad6")]
public class AddSharePointMetaData : IBaseComponent, IComponentUI, IComponent, IPersistPropertyBag
While further investigating SharePoint-BizTalk integration capabilities I ran into a file overwrite mode called “Rename” which I thought was rather unique. I have never seen something like this in the FILE or FTP adapters so I figured that I would post something on it.
The WSS Adapter has an Overwrite property that allows you select one of the following modes:
The Yes and No values are pretty self explanatory. Orchestration allows you to determine which mode you want to set within an Orchestration at run time.
When you select the Rename value, BizTalk will use the static file name that you have configured in the Send Port configuration or the actual file name of the source document. If the document library does not contain a file with this name already, it will be added with the correct filename.
Where it gets interesting is when you try to insert a file with the same file name as one that already exists in the same document library. When this situation occurs the first portion of the file name will remain intact, however a unique identifier will be added to the end of the filename. BizTalk will not raise a warning or error indicating that this event has occurred.
So when would I use this? I think this feature could be used in one of those “exception to the rule cases” where the business has indicated that “there will never be a duplicate file uploaded into this document library” but it actually happens. It is also helpful when the view of the document library has been configured in such a way that the documents are sorted by file name. This way it makes it easier to identify one of these cases as opposed to just using a Message ID where the document could end up being displayed in a very random fashion.
First off this technique should work with any RFC connected system but since the only RFC connected system that I use is BizTalk it will therefore have some BizTalk specific information.
I have touched on this topic before in my web cast and in my blog that integrating with SAP can be tough due to the vast differences in terminology, process and technologies used. A situation that I have come across is that you often need a Subject Matter Expert (SME) or someone very familiar with a Business Process to generate an IDoc for a BizTalk developer to test or validate their portion of the integrated application. Sometimes creating an IDoc is very simple where as sometimes it may take a few transactions and screens to generate the IDoc in order to validate one of your use cases. This can lead to some frustration by both the BizTalk Developer and SAP resource as the BizTalk developer is always bothering the SAP resource to create “just one more IDoc”.
Something that a colleague brought to my attention was an SAP transaction called WE19 that allows you to re-process an existing IDoc. This allows the BizTalk developer to have the SAP resource create an initial IDoc and have the BizTalk developer re-submit that IDoc however many times they would like. Another benefit of this is process that involved deltas. We have a process where SAP will provide a “dump” of employee data that we need to update a down stream system with. We don’t need to process this data every day so we will just send the differences, or deltas, on a regular basis. So, even in a test or staging environment, once I have received the IDoc that contains all employee data I will not get that data again unless I ask for it. If we have not had any changes to our employees I will also not get the delta information either since there is no new data to feed the downstream system with.
Using this transaction is great because I don’t need to bother anyone from the SAP side to create data for me. All I need to know is when the last time that particular IDoc was sent. I can find this information in BizTalk by querying tracked message instances or by querying WE02 in SAP. Once I find this information I need to capture the IDoc number as this piece of information is required in order to resubmit the IDoc.
Here are 3 different ways to find an IDoc number. The first two involve BizTalk and the last involves using the WE02 transaction in SAP. You may want to check with your BASIS administrator if you do not have access to the WE02 transaction.
Resubmitting the IDoc
Summary
I know I have found the WE19 transaction to be very helpful so hopefully you will find it helpful as well. It can be pretty frustrating when you have a deadline and there is no one around to create IDocs for you. Using this approach will lessen your dependency on having a SAP resource around. You can also use this same technique to load up BizTalk with messages and test the robustness of the SAP adapter no matter whether it be the dot net connector version or the WCF version.
I was recently informed that my MVP status has been renewed for 2010 in the BizTalk Server track. The MVP program is an excellent opportunity to interact with product groups and other MVPs from around the world. An example of this collaboration was the BizTalk Light and Easy Viewing series that I had an opportunity to participate in this past year. It was a great community project that involved experts from all over the world building webcasts demonstrating BizTalk 2009’s capabilities which were made available to anyone who was interested in viewing/downloading them.
The highlight of the program for me is definitely the annual MVP Summit which is hosted in the greater Seattle area each year. I have met a lot of great people through this event and continue to communicate with them on a regular basis.
I have had a couple readers ask me some questions about the BizTalk Adapter Pack Migration Wizard and how well it migrates projects that leveraged the .Net connector based SAP Adapter. This is something that I have been interested in exploring for quite some time but just hadn’t gotten around to it. When we initially went live with our BizTalk 2009 environment we decided to reduce the complexity and and not migrate the SAP projects to the new WCF based adapter knowing that we had a little time to take care of this task. We have now started to migrate some of these apps to use the WCF based adapter so I figured I would re-create a use case from an existing integration scenario to see how well the wizard works.
In order to re-create this use case I decided to start the slate clean by creating a new BizTalk 2009 project and then adding an extended CONF32 IDoc called ZConf32. If you are unfamiliar with the term “extended IDoc” please see one of my previous posts. Some of this post may be redundant, as it discusses the old adapter, but it could help someone trying to integrate with SAP with the legacy adapter as well.
Note: From here on in I will refer to the .Net Connector SAP adapter as the legacy adapter and the BizTalk Adapter Pack 2.0 SAP adapter as the WCF adapter. Nothing in this post covers the BizTalk Adapter Pack 1.0 SAP adapter however I would assume that most scenarios that apply to the 2.0 adapter also apply to the 1.0 adapter.
Creating the BizTalk project
Note: This scenario could easily be accomplished as a messaging only scenario, but I have decided to provide an orchestration for illustrative purposes.
So that completes creating a fairly simple integration scenario with SAP using the legacy adapter. Next we will take a detail look into what is involved in migrating this project to use the new WCF Adapter.
Ensure the following are installed:
- Microsoft BizTalk Adapter Pack 2.0
- WCF LOB Adapter SDK SP2
- BizTalk Server 2009 or BizTalk Server 2006 R2
- Visual Studio 2008 or Visual Studio 2005
To setup the migration directory:
- Copy %Microsoft Visual Studio Install Dir% \Common7\IDE\ Microsoft.WizardFramework.dll to the same folder.
![clip_image002[5]](http://lh5.ggpht.com/_KBYzYDwIZhw/Sz0zUD4HfMI/AAAAAAAABO0/yecjjWJ3Cko/s1600-h/clip_image002%5B5%5D%5B3%5D.jpg "clip_image002[5]")
![clip_image002[7]](http://lh4.ggpht.com/_KBYzYDwIZhw/Sz0zVOEyTbI/AAAAAAAABO8/FiSNeTQms4c/s1600-h/clip_image002%5B7%5D%5B3%5D.jpg "clip_image002[7]")
So which option is better? Like most things in IT, it depends. If you have a complex solution with very “busy” maps I would opt for the first option – making changes at the edges. This option poses less risk as you are limiting the amount of change you are introducing. This option does require some updates to the core application due to the synchronous nature of the WCF based adapter. Previously IDocs were sent asynchronously (one way) where as now they are processed using a Request-Response (two way) process. So if you had an Orchestration that had a logical Send Port and Send Shape, you would now need to account for the response message coming back into the BizTalk Orchestration.
If you have a simple integration, like the one I am describing I would opt for option #2. Adding additional maps will decrease performance(most likely marginally) and also becomes a more complex application to support operationally as you are now supporting additional maps. If you have the same developer who built the BizTalk solution migrating it then they would probably also feel more comfortable in option #2. If you have brought in someone in to perform the migration they may feel more comfortable with the first option.
Having seen this error before I figured that the name of the application had probably been removed from the project file. So I just added the Application Name and then I was able to deploy.
Let’s explore what is included in the binding file that was generated by the wizard:
The adapter failed to transmit message going to send port "processVehicleTime_PortType_SendZConf32" with URL "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX". It will be retransmitted after the retry interval specified for this Send Port. Details:"Microsoft.ServiceModel.Channels.Common.UnsupportedOperationException: Incorrect Action <BtsActionMapping xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Operation Name="Operation_SendZConf32" Action="http://Microsoft.LobServices.Sap/2007/03/Idoc/3/CONF32/ZCONF32/700/Send" />
</BtsActionMapping>. Correct the specified Action, or refer to the documentation on the allowed formats for the Actions.
Check out this link for more details.
Old process
![clip_image002[9]](http://lh4.ggpht.com/_KBYzYDwIZhw/Sz0z8ZAN_gI/AAAAAAAABTI/KrfTnw88OI0/s1600-h/clip_image002%5B9%5D%5B3%5D.jpg "clip_image002[9]")
New Process – since we are using a Request-Response port type we now receive a message back from SAP(XMLReceive)
![clip_image004[5]](http://lh4.ggpht.com/_KBYzYDwIZhw/Sz0z9E5mH4I/AAAAAAAABTQ/tShNIWTtqPc/s1600-h/clip_image004%5B5%5D%5B3%5D.jpg "clip_image004[5]")
Summary
So as you can see the migration wizard had some mixed results. I am not quite sure why the Map, in either scenario, was not created/modified. The map itself was not overly complex although it did included some functoids. Another feature that I did not like was that you don’t have the ability to apply a naming convention to the schemas that were generated. If you were to add these schemas to a BizTalk solution from Visual Studio you do have the ability to provide a pre-fix using that tool so I am not sure why you could not use this same feature in the migration tool.
If you use option #2(the intrusive approach) it will go ahead and update your ports and message to use the new schema so I do think that there is some value in using the migration tool. Also, if you want to update several IDocs within one project you should be able to perform this migration by running the tool once. I definitely do not think that the tool is a silver bullet and you will need someone who is knowledgeable with BizTalk to perform these migrations. In defense of the tool, it is not an officially supported tool but is there to act as an aid and I do believe that it does accomplish this. All in all I think it took me about 20 minutes to migrate my fairly trivial application so the effort was not momentous.
If you are interested in seeing how this tool works with the new WCF based SQL Server adapter please check out Richard Seroter’s post.
