ShareTalk Integration (SharePoint/BizTalk) – Part 8 Integrating with the Cloud

First off I apologize for the length of this post, but I trust that you will find some benefits with it.  The goal of this post is to demonstrate an on-premise form being completed and having that information sent to the cloud via BizTalk 2009.  The cloud will route this request to an off-site service that will respond to this request indicating whether or not the request was valid and return a result back to BizTalk.  BizTalk will use this status to kick off some workflow. I will also dive into some of collaboration features of WSS 3.0 and demonstrate how you can still provide form exception handling and form re-submittal in an asynchronous scenario.

You may think that I am just trying to stuff as much Microsoft technology into one blog post as possible. What I am really demonstrating is giving BizTalk a user interface and allowing it to communicate with a off-premise service via the cloud.  The service itself is not hosted in the cloud, but rather in someone else’s data centre.  I am using the Microsoft cloud to relay the request using the App Fabric Service Bus(formerly known as the .Net Service Bus).

Note: I started this blog post back in early November prior to November 2009 CTP so I have had to go through and resolve the breaking changes that the new CTP introduced.  I am not planning on getting into these breaking changes in much detail, but for further details check out the App Fabric team’s blog post.

To give this post some business substance, I am going to walk through a condensed use case in the Power industry in Alberta, Canada.  The Power industry is broken down into a few segments: Generators, Transmission companies, Distribution companies and Retailers.  The scenario I am going to walk through is on that involves a customer switching their Retailer.  Distribution companies provide the “infrastructure” that allows power to be delivered to Residential and Industrial customers.  Distribution companies are also required to provide meter usage data to Retailers where they will use this information to provide the end customer with a bill.  Customers generally have an option when it comes to choosing their Retailer but not so much the Distribution company.  In this scenario, the customer would like to switch retailers.  The Retailer must send the Distribution company a “Switch Retailer Request” in order for this customer to be officially “switched”.  The Distribution company needs this information so that they can send the correct meter data to the Retailer so they can bill their customer.

 

1. Information Worker will fill out InfoPath “Switch Retailer Request”
2. InfoPath form will be saved into a WSS Document Library
3. BizTalk will consume this InfoPath form and submit the data to the Distribution Company’s “Switch Retailer" Service.  This Service is an on-premise service that is accessible via the .Net Service Bus AppFabric Service Bus.
4. The .Net Service App Fabric Service Bus will provide a “simple” SiteID validation.  A SiteID is a identifier used to designate a Customer site – kinda like an address.
5. If the SiteID validation is successful then the retailer will be switched and the InfoPath form will be sent to the “Success Switch Retailer Request” document library
6. If the SiteID validation fails then the InfoPath form will be sent to the “Failed Switch Retailer Request” document library
7. When the InfoPath form is sent to the Failed Switch Retailer Request” document library an email will be sent to the information worker informing them that the switch was not successful. The Information worker can then update the form to fix the error.  An error description is also provided in an “Error Message” column in the document library.

On Premise Switch Retailer Service

The Switch Retailer Service was written using C#, WCF and the July CTP of the .Net Services SDK November CTP of the App Fabric Service Bus SDK.  I started with a pure WCF Service(so that I could generate Schemas for BizTalk) and then added the bits required for this service to be exposed via the .Net Service bus.  This is a similar approach to the one that Richard Seroter used in his blog post.

 

I started with a couple of data contracts:

• SwitchRetailerRequest – this message will be sent from BizTalk and will contain pertinent information like the Customer’s Site ID, the Activation Date, the Retailer Id etc. 

• SwitchRetailerResponse – this message will be sent back to BizTalk from the SwitchRetailer service.  As part of this message, a status will be sent back and an ErrorMsg when the Switch Retailer fails.

 

My interface defines an operation called “CustomerSwitchRetailer” and as you can see it is expecting a SwitchRetailerRequest and will return a SwitchRetailerResponse.  I have also provide a namespace and a name for this Service contract.

The implementation of this service includes a call to the “validSite” method.  This method contains the logic that indicates whether this site is valid. 

The host for this service is a console application.  If you are familiar with the “Echo” sample then this will look very familiar.  When the CTP changed, I had to update this method due to some of the breaking changes.

 Here is the significant portion of my services’ App.config.  I started with the .Net Services SDK Echo sample and have adopted it for my use.  Changes include creating a behaviour called “SbEndpointBehavior”.  In this behavior, I have indicated that I want to use “UserNamePassword” credentials as the credentialType.  When BizTalk connects to this service, we will need to match this credentialType in the BizTalk Admin Console.  The credentials themselves are credentials that you configure in the .Net Service bus developer portal.

The UserNamePassword credential type is no longer supported in the latest version of the CTP. The prescriptive guidance is to use one of the new authentication types including Shared Secret, Simple Web Token or SAML.  For the purpose of this example I have used Shared Secret.

Also notice the binding.  I have chosen the “netTcpRelayBinding” that ships with the .Net Service SDK.  When I initially created the WCF service I was using the basicHttpBinding but since I want to take advantage of the .Net App Fabric Service Bus’ relay capabilities I want to use this binding. 

 

BizTalk

If we shift gears to the BizTalk portion of the solution, you will find the following artifacts:

• SwitchRetailer_InfoPath.xsd schema.  This schema is the message that BizTalk will receive from SharePoint and is also the underlying data structure in our InfoPath form.  The “ErrorMessage” element is not displayed on the InfoPath form but will be used later to populate a SharePoint column with the error message that is being returned from the Switch Retailer service.

• Service Schema – This schema was initially generated from consuming the WCF service that I built(before converting it to a .Net Service bus service).  A powerful feature of WCF, and therefore the .Net App Fabric Service bus, is the separation between contracts and bindings.  I am still using the same contract between the WCF service and the one exposed by the .Net Service Bus, but the binding will be different.

• I need a map that will transform the data in the InfoPath form into the Service Request method

• Once I have this request mapped, I want to send it to the service and receive the response. 

• As part of the response process, I need to determine the status of Switch Retailer.  I want to use a Distinguished field to make the retrieval of this status a simpler endeavor.  Since adding Distinguished Fields(or promoted properties) to a Service schema is not advised since you may update that schema later and lose the designation, I have created a helper message and want to map the Switch Retailer Response message to this helper message.

• I have a simple decide shape that will check the value of this distinguished field.  In the event that the status is true then I want to populate the “Success” SharePoint document library and in the event that the status is false then I want to populate the “Unsuccessful” SharePoint document library. 

Note: The WSS Adapter can be used in a dynamic fashion, so I could eliminate 1 port and use a dynamic port instead of a static port.  To keep this example simple, I have opted to use 2 static ports.

 

• In the “Else” branch I want to populate this return message with the Error Message from the service response message.  I will use this information to populate a column in the "Failures” document library

• To allow for the Error Message to flow through to SharePoint, you will need to create a column in the document library(see below), but you will also need to configure BizTalk to pass this data along to SharePoint.  So in your “Failures” send port, set both the “Namespaces Aliases” and then set the Column 01 Value with an appropriate XPath statement.

Namespace Aliases: ns0=http://BizTalk_Server_Project3.SwitchRetailer_InfoPath

Column 01 Value: %XPATH=//ns0:SwitchRetailer/ErrorMessage%

SharePoint

I am not going to go through each step in detail if I have already gone through it prevously but I will try to link back each step to a previous blog post where applicable.

• Creating the InfoPath form from an XSD (Post 6)

• Publish Info Path form (Post 6).  As part of this process a document library called “Switch Retailer Document Library” was created.

• Once you have published your form, when you click on “New” –> “New Document” your Switch Retailer Info Path Form should now appear.

• Recall in the BizTalk section that I have to route Switch Retail responses to either a Unsuccessful or Successful document libraries (Post 3).  I have now gone ahead and done this.

• In the case of the “Failures” document library I want to be able to provide error information into this document library so that when the information worker comes into fix the issue that they are aware of  what the issue is before they open the form.
• Click on “Settings” –> “Create Column”

• When a Switch Request Fails, I would like to notify the information worker of this situation.  Yes, BizTalk has an SMTP adapter, but I have decided to leverage SharePoint to provide this functionality.  In order to do this I need to set up my outgoing SMTP settings in WSS 3.0.
  • Launch the “SharePoint 3.0 Central Administration” tool

• Click on “Application Management”

• Click on “Web application outgoing e-mail settings”

• Ensure that the correct Web Application has been selected and then provide the required email settings.  For this example I am just routing emails through my ISP.

• Also ensure that the user who will be receiving these failure alerts has an email associated to their account.  To do this jump back to your “Web Site” and click on the “Site Actions” - “Site Settings” link

• Click on “People and groups”

• Click on the link of the user that you are interested in receiving email

• Ensure that you have an email provided

• Navigate back to your “Failures” document library

• Click on “Actions” –> “Alert Me”

• Provide a title for your alert, add any required users that you want to receive this email and then indicate when you want the user(s) to receive the email.  I have decided to configure these users to only receive alerts when a “new” item is added.

Configuring the demo

• Start the “Switch Retailer Cloud” service

• Since I am using the November 2009 CTP SDK and have opted to use the Shared Secret authentication type, I need to provided a Service Namespace, my Issuer Name and my Issuer Secret.  These details are available from the Azure Developer Portal.

• Next I want to ensure my BizTalk configuration is correct.  The first step is to establish my receive location to SharePoint that will look for Switch Retailer InfoPath forms that have been submitted to the “Switch Retailer Document Library”

• The next step is to establish my configuration that will allow BizTalk to connect to the App Fabric Service bus. In this two-way send port, I am going to use the WCF-Custom Adapter.  This Adapter will allow me to use the netTcpRelayBinding which allows me to communicate with the App Fabric Service Bus.  For this reason, you need to have the App Fabric Service Bus CTP installed on your BizTalk server.

I needed to create a Endpoint Behavior and since the Switch Retailer Service uses a transportClientEndpointBehaviour, the client(BizTalk) needs to match this Endpoint Behavior.  Since UserNamePassword is no longer an option, I have opted for “SharedSecret”.

• I now need to create two Send Ports; one that will route Successful Retailer switches and one that will route Unsuccessful Retailer switches.

Running the Demo

In the previous steps, we have started the Switch Retailer Service, configured BizTalk and are are ready to start calling this service via SharePoint/BizTalk.  The first InfoPath form submitted will be successful, followed by a failed request.

• InfoPath form has been populated and submitted

• Switch Retailer Request has been placed in Document Library

• BizTalk is polling this Document Library on a regular basis

• The App Fabric Service receives this request and routes it to the Switch Retailer Service.  The service will validate the request and provide the response to BizTalk.

• As you can see the response was successful and the document was routed through the SharePoint Success Send Port.

• The InfoPath form has been routed to the “Success” Document Library

• I will now send an invalid Switch Request by providing a Site ID that only has a length of 3.

• The Service has detected this invalid Site Id and logged it accordingly

• BizTalk has received the response and detected that the Switch Retailer failed.  BizTalk will therefore route this message to the “Failed” document library.

 

• Failed InfoPath form has been delivered and the Error that has occurred is populated in the “Error Message” column.

• I had previously configured SharePoint to notify me, via email, whenever a new document was placed in the “Switch Retailer Failures” document library. 

• I have fixed up the Site Id and re-submitted

• The Switch Retailer Service has received this updated request and this time it was successful

 

Generally hooking up BizTalk to a user interface can be a risky endeavor as the UI can be blocked while BizTalk is processing.  If a downstream system is down, a time out could exist on the UI client if BizTalk is working through its retry progressions.  By using the approach above, we have eliminated the potential of the UI client being locked up while BizTalk is calling the Switch Retailer service.  By leveraging some workflow capabilities in SharePoint and BizTalk we can still provide the user meaningful feedback in reasonable amount of time asynchronously.