Windows Azure BizTalk Services Preview (Part 2) –BizTalk Adapter Services SAP Integration

 

Over the past 7 years I have spent a lot of time integrating BizTalk and SAP systems.  I probably built more interfaces for SAP than any other system.  When new technology such as Windows Azure BizTalk Services surfaces I am always interested to understand what the experience looks like.

If you read my prior post in this series, you are probably wondering how it is possible to use Windows Azure BizTalk Services in the Windows Azure Cloud and connect to your On-Premise LOB Systems like SAP?  Enter BizTalk Adapter Services.

BizTalk Adapter Services allows us to use the BizTalk Adapter Pack in order to connect to LOB systems.  The BizTalk Adapter Services are hosted in IIS and leverage the Service Bus Relay (under the hood) in order to traverse Firewalls and NATs.  Using the BizTalk Adapter Service is just another tool in our toolbox when achieving Hybrid integration.

Adapter Services Installation

I am now going to quickly install the BizTalk Adapter Services and then provide a walkthrough of how we can integrate with SAP.  The scenario that we are going to walk through is part of a larger process that I won’t get into much detail within this blog but we have an SAP process were we need to feed sub-systems within the organization with Equipment data from SAP.  We will get this equipment by calling a custom SAP RFC(BAPI).

The BizTalk Adapter Service is a component that we want to install on a Server within our Network.  This machine does not require BizTalk Server to be present.

 

This is an important step as the account that we specify here will require access to SQL Server where configuration will be stored about our Relay Endpoints.

In this case I am using a local SQL Server instance.  You can use SQL Express if you so desire but I have leveraged SQL 2008 R2 (SQL Server 2012 is also supported).

This step kinda reminds me of the BizTalk SSO Service password but they are not related or linked.

The BizTalk Adapter Service installation will be installing a Management Web Service.  Since I am just using this for demo purposes I did not enable SSL. 

Once we have finished installing the BizTalk Adapter Service we will discover a new database called BAService.

We will also discover a new entry in Server Explorer within Visual Studio called BizTalk Adapter Services. 

If we right mouse click on BizTalk Adapter Services we have the ability to add a BizTalk Adapter Service by specifying the URL of the Management Service that was created as part of the installation process.

Once this Adapter Service has been added we can expand it and discover our LOB Targets.

We can add a new SAP target by right mouse clicking on SAP  and selecting Add SAP Target.

We now need to specify our SAP connection information much like we would if we were adding Generated Items within a BizTalk Server 2013 solution.  We need to specify the Application Server, the System Number, Client and Language.  I recently blogged about connecting to SAP Messaging Servers.  Not to worry these more advanced scenarios are supported by clicking on the Advanced button.

Next we need to provide our SAP credentials if we are not using Windows Credentials.

Much like the BizTalk Server experience we can search operations including IDOCs, BAPIs and RFCs.  Do note that at this point we are only able to push messages to SAP.  This means we can send an IDOC and call a BAPI/RFC and get the response back.  SAP cannot push IDOCs or call an RFC that is hosted by the BizTalk Adapter Service. 

Once we have found the artifact that we are looking for we can add it to the Selected operations list and then click the Next button.

Once again need to provide credentials that the BizTalk Adapter Service will use to connect to SAP.

This gets a little interesting.  Much like in my previous blog post where we connected to a Service Bus Topic, we will use the production Service Bus connectivity information.  So in this case I am going to leverage an existing Service Bus Namespace and credentials to enable Service Bus Relay capabilities.

We also have the ability to specify a Target sub-path that just allows us to organize our endpoints.

We now have a confirmation page that we can just click the Create button in order to finish the wizard.

With our SAP LOB Target created we can now drag that Target onto our canvas:

At this point we have just created an SAP target and added it to our canvas. but we have not actually generated our schemas that will be used within our solution.  In order to generate these schemas we need to right mouse click on our SAP LOB Target and specify Add Schemas to Project.

We will once again be prompted for SAP credentials.

Within our solution, we will now find two schemas have been added.  One is the core schema and the other is a schema for data types leveraged by the core schema.

A Web Request Schema, that we will expose externally to calling applications, is also needed.  This schema is very simple with only two fields: Plant and EquipmentName.

Next we need to add a Map that will allow us to transform our Web Request message into our SAP Request message.

The map is pretty straight forward as we only need to map a couple fields.

A Map that will allow us to transform our SAP Response into our Web Response is also required.

Next, we need to add  an Xml Request-Reply Bridge to our canvas by dragging it onto our canvas.

We now want to modify the Property page for this artifact  as it will become part of our overall URI.

By double clicking on our  Bridge, we have the ability to configure it.  Within this configuration we will want to specify the Message Type for the message that we will be receiving from the calling application and the Message Type for the type of message we will be sending back to the calling application.

We also want to specify our two Transformations.  The first one will take our Web Request and map it to our SAP Request.  The second Transformation will take our SAP Response and map it to our Web Response.

Once our Bridge has been configured, we need to connect our Request Response Bridge to our SAP LOB Target.  We can do so by dragging a Connector from our Toolbox and connecting our Bridge to our SAP LOB Target.

Routing messages is a core functionality within BizTalk Services. In order to route messages, we need to provide a filter condition that will allow us to specify messages to flow between the Bridge and the SAP LOB Target. In order to set this filter we need to click on the line that connects these two artifacts and then click on the Filter Condition.

Since we want all messages to flow to this LOB system we will choose Match All and click the OK button.

We also need to specify a SOAP Action much like we need to do with BizTalk Server.  In order to do this we need to highlight the connection between the Bridge and LOB target and then click on Route Action.

In this case we want to specify a SOAP Action and then specify the action from our Request Schema.  Much like the EAI/EDI Labs solutions we want to wrap our Expression around single quotes( ‘   ‘)

We are now almost ready to deploy our solution but before we can do that we need to specify our Deployment Endpoint.  We can do so by clicking on any blank space on our canvas and then specifying our Deployment Endpoint address in the Property page.

Deploying our solution is as simple as right mouse clicking on our Visual Studio Solution file and selecting Deploy Solution.

In order to deploy our solution we will need to provide the Deployment Endpoint, ACS Namespace(for BizTalk Service environment), Issuer Name and Issuer Shared Secret.

Within a few seconds our solution will be deployed and will be ready for testing. Once again I can  use the MessageSender application that is part of the BizTalk Services SDK. But, since looking at a black screen with white xml text isn’t very appealing, I created an ASP.Net Web application and borrowed the MessageSender class from the SDK project.  In this case I have my Web Application running on a laptop that isn’t on the same network as the machine that is hosting the BizTalk Adapter Service.

 

Within the web application, we have the ability to provide a Plant and a wild card string representing a piece of equipment in order to get more details about where that piece of equipment has been deployed, the Manufacturer and Model Number.

 

Conclusion

Hopefully this walkthrough has provided you with some valuable insight on what it takes to integrate a Line of Business system like SAP with Windows Azure BizTalk Services.  I can certainly envision scenarios where an organization uses an LOB system like SAP but may not have a dedicated integration platform to perform some of this integration. Windows Azure BizTalk Services may fill this void and enable some business scenarios that just may not have been possible for that organization.  Consider another situation where you may have a SalesForce system that requires SAP data (maybe Equipment data ) and this type of functionality in BizTalk Services really becomes a business catalyst.

I also think that the BizTalk Adapter Service may provide some interesting Mobile use cases as well as we can now expose SAP data through the cloud in a secure manner.

While the scenario is a bit dated, I also wrote a similar article on how to submit IDOCs to SAP using the Windows Azure EAI/EDI Labs CTP which has now been superseded by Windows Azure BizTalk Services.

Introducing Windows Azure BizTalk Services Preview –Part 1

 

On Monday, June 3rd Microsoft announced the Preview of Windows Azure BizTalk Services (aka WABS or just BizTalk Services).  Windows Azure BizTalk Services is a managed service provided by Microsoft.  More specifically you can think of BizTalk Services as a Platform as a Service (PaaS) offering.  Recently, I had an opportunity to work with this platform as I was involved in an NDA event in New York City.  While I was preparing for that presentation I took the opportunity to document some of my learnings so that I could share them when the content was no longer considered NDA.  With the TechEd announcement this information is now considered public hence the timing of this blog post.

The purpose of this offering is to provide customers a way to execute integration scenarios in a cloud based environment.  The idea is that you would design and build your interfaces locally within Visual Studio and then deploy your interface to the cloud where it is executed.

Conceptually we can think of a Windows Azure BizTalk Services much like any other interface that utilizes an Integration platform.  We have source systems, an intermediary where we can perform functions like Message Validation, Enrich, Transformation and Enrich (aka VETE).  We then have the ability Route our message to the appropriate destination system.  The difference with this platform is that the “heavy lifting” is occurring in the Windows Azure cloud as opposed to our On-Premise BizTalk Servers.

 

There are some subtle differences between Windows Azure BizTalk Services and BizTalk Server 2013 (On-Premise) solutions.  I say subtle because for an experienced BizTalk person these new concepts are not that big of a leap.  Below is s diagram containing some of the key artifacts and components that make up a BizTalk Service solution.

Bridges

The term that jumps out at you is probably Bridges. What is a Bridge? A Bridge is a message processing entity.  Probably the closest BizTalk construct that you can relate a Bridge to is an Orchestration but there are differences.  The reason why Orchestration is “kinda close” is that it groups a series of action or processes together within one logical container.

Bridges include Sources, Pipelines and Destinations.  Currently within the Windows Azure BizTalk Services Preview we have 3 types of Bridges:

• Xml One-Way Bridge
• Xml Request-Reply Bridge
• Pass-Through Bridge (for message transport)

Within a Pipeline we can call transformations, perform validations, encode, decode or enrich messages.

We also have 3 types Source types:

• FTP/S
• HTTP
• SFTP

and 9 Destination types:

• FTP
• One-Way External Service Endpoint (No Service Bus connectivity)
• One-Way Relay Endpoint (Service Bus connectivity)
• Service Bus Queue
• Service Bus Topic
• SFTP Destination
• Two-Way External Service Endpoint
• Two-Way Relay Endpoint (Service Bus connectivity)

Technically this Destination list is not fully complete.  We also have the ability to communicate with On-Premise LOB systems which I will go through in another Blog Post.

SDK Installation

Even though our BizTalk Services run in the cloud, we still perform our development activities within Visual Studio.  Currently Visual Studio does not ship with the appropriate templates that allow us to build BizTalk Services solutions.  However, we can get these templates from the Windows Azure BizTalk Services SDK.

The installation is pretty straight forward and I have included some screen shots below.

Note: On 64 bit machines only need to install the 64 bit SDK

 

 

 

Visual Studio Experience

When we launch Visual Studio 2012 we will discover a new C# templates under the BizTalk Services node:

If we choose BizTalk Service we see a blank diagram that allows us to drop a Bridge and Source or Destination endpoints onto it.

If we choose BizTalk Service Artifacts we will see a new project that does not include a bridge but a Transformation (map) and two sample schemas.

If we choose the BizTalk Service Project template we have more capabilities as we can drop any of the following Artifacts/Actions onto our canvas.

 

In the image below I have modeled a scenario just to visualize what building an interface in BizTalk Services may look like.  In this scenario we are a company that specializes in managing Request For Information (RFI) processes. Organizations that have projects that they would like vendor responses to will send us these requests and we will publish this information to our trusted vendor network that allow them to be informed of these opportunities. 

In this scenario I have an XML One-Way Bridge which will expose an HTTP(s) Endpoint.  Within this bridge we will perform an transformation from the message type that was sent by our customer requesting RFI information into a message type that all of our Vendor partners are expecting. Once we have transformed this message, we need to drop it on a Service Bus topic. 

Much like a BizTalk Server project we will start with our schemas.  This schema editor is very similar, if not the same, as the BizTalk Server schema editor. The first schema that we will build represents our client request.

 

Next we will create a schema that we use with our strategic vendors who subscribe to our service. You will notice that there are more fields in the partner schema.  This is where the value of our firm comes from as we have direct communication with client and have some additional insight into their project about whether the project is likely to proceed and will also expose the name of the customer.  Typically these values would be looked up from a CRM system but for now we will keep it simple and include it in the map.

Next we will add a Transformation(or Map) to our solution that will allow us to transform our CustomerRequest into a PartnerRequest message.

Getting deep into the mapper is outside the scope of this post but you will see some new “functoids” or what are now being called Message Operations.  Some of these new capabilities include ForEach, MapEach, If-Then-Else, List Operations and YES you can call custom code.

Once we have defined our Schemas and Maps we can now configure our XML Bridge.  Within this bridge we will specify our message type that we will be receiving (CustomerRequest).  We will also specify our Xml Transform (Map).  Since we are not dealing with flat files and do not have any Validation requirements we will leave those stages in their defaulted stage.

We now need to wire up the output of our Bridge to our Destination endpoint which is a Service Bus Topic.  We will do so by selecting a Connector operation and then connect the dots(kinda like in Viso) between our Bridge and Topic endpoint.

 

Now remember that we can perform message routing in BizTalk Services.  To set our Routing patch we will need to click on the line that we drew from our Bridge to our Topic.

We now need to click on the Filter Condition.  In this case we are not going to restrict any messages from continuing to our topic so we will select Match All but we do have the ability to create a filter based upon message content.

Next we will change our focus to the Service Bus Topic.  We need to configure this endpoint by clicking on it and then modifying the PartnerTopic.config (filename will vary) file for it.  Within this config file we will specify our Service Bus namespace, topic name,Issuer Name and Key.  Please note that these are production Service Bus entities and have nothing to do with the BizTalk Services Preview environment.

We also want to rename our XMLOneWayBridge so that we are not using the default name since this name will makeup part of our URI. 

Lastly we want to specify our endpoint where our Bridge can be contacted. To do this click on any empty space on the canvas and then modify the Deployment Endpoint.  (Note the endpoint below is fictitious).

At this point we can now deploy our solution to Windows Azure BizTalk Services.  We can do so by right mouse clicking on our Visual Studio Solution and selecting Deploy Solution. Here is where we want to specify our BizTalk Services environment configuration.  You cannot use your Production Windows Azure credentials here.

The deployment only takes a few seconds and when it is completed we should see a message like the following:

Testing

Part of the SDK that we just installed includes utilities that can be used when Sending and Receiving messages to/from BizTalk Services.  We are going to go ahead and leverage the MessageSender application.

The Message Sender Application is a command line tool that is expecting the following inputs:

• ACS Namespace (for BizTalk Services environment)
• Issuer Name
• Issuer Key
• Bridge Endpoint Address
• File location to message you want to send
• Content Type

The result is that our message has been sent to our bridge successfully.

If we go to the Windows Azure Production portal we will discover that we have a message within our Subscription.

Conclusion

If some of this looks familiar it is because it is!.  This is the evolution of what use to be called Windows Azure EAI/EDI Labs.  It is now being branded underneath the “BizTalk” brand which makes sense on a few levels in my opinion.  For instance the team that is working on this new service is the same team responsible for the BizTalk Server product.  One thing you will hear the product team speak to is that there is “ONE BIZTALK”. One platform with many capabilities including BizTalk Server On-Premise, BizTalk Server running as Infrastructure as a Service (IaaS) and now BizTalk Services which is running as a Platform as a Service offering.

There is another huge aspect to BizTalk Services that has been neglected from this post as I just am not familiar with it but EDI in the cloud also makes up a substantial part of this offering and arguably provides the greatest area of opportunity.

I am just getting started with blogging about BizTalk Services.  Stay tuned for my next post on how to integrate On-Premise Line of Business Applications(SAP)  with BizTalk Services.

BizTalk 2013: WCF-SAP Adapter using Messaging Servers

 

SAP supports a few different architectures when it comes to scaling out.  In some environments, mainly smaller environments, the components that facilitate sending and receiving messages (such as IDOCs, BAPIs or RFCs) may be co-hosted on an Application Server.  In some larger environments a dedicated Messaging Server(s) may be provided. We can also use a named destination that we define in our saprfc.ini file.

Up until this point all of my experience integrating with SAP has occurred through an Application Server.  This is the default Connection Type that is populated in the Send Port configuration of a WCF-SAP send port.

Recently I ran into a situation where the SAP environment was not using Application Servers.  Instead a Messaging Server was being used.  The end result was that I received the following error

 

A message sent to adapter "WCF-SAP" on send port "SendGetSAPEquipment" with URI "sap://CLIENT=010;LANG=EN;@b/*SAPSERVER*/*ENV*?RfcSdkTrace=False&AbapDebug=False" is suspended.
Error details: Microsoft.ServiceModel.Channels.Common.ConnectionException: Details: ErrorCode=RFC_OK. ErrorGroup=RFC_ERROR_COMMUNICATION. SapErrorMessage=Connect to message server failed
Connect_PM  MSHOST=*MSHOSTValue*, R3NAME=*ENV*, GROUP=PUBLIC

LOCATION    CPIC (TCP/IP) on local host with Unicode
ERROR       service *SAPSERVER* unknown
TIME        Tue Apr 23 09:38:18 201
RELEASE     700
COMPONENT   NI (network interface)
VERSION     38
RC          -3
MODULE      nixxhsl.cpp
LINE        776
DETAIL      NiHsLGetServNo: service name cached as unknown
COUNTER     1.  AdapterErrorMessage=. ---> Microsoft.Adapters.SAP.RFCException: Details: ErrorCode=RFC_OK. ErrorGroup=RFC_ERROR_COMMUNICATION. SapErrorMessage=Connect to message server failed
Connect_PM  MSHOST=*MSHOSTValue*, R3NAME=*ENV*, GROUP=PUBLIC

Microsoft.ServiceModel.Channels.Common.ConnectionException: Details:
ErrorCode=RFC_OK. ErrorGroup=RFC_ERROR_COMMUNICATION.
SapErrorMessage=Connect to message server
ailed
Connect_PM  MSHOST=*MSHOSTValue*, R3NAME=*ENV*, GROUP=*MyOrganization*

Since the SAP environment was actually using Messaging Server(s), In order to successfully make a connection to SAP, here is the correct configuration that I needed to provide:

• Application Server Group Name
• Message Server Host
• R/3 System Name
• Connection Type B

These properties will be specific to your environment and therefore I have blacked them out.  You will need to get this information from your BASIS admin.

 

However, this is not enough and this is where I got hung up.  You also need to specify TCP port where SAP will be looking for these connections.  This property is not exposed through the BizTalk GUI.  In order to specify this TCP port you need to modify the following file:

C:\Windows\System32\drivers\etc\services

Within this file you will want to add an entry like:

MSHOST            1234/tcp   # SAP ENV System Message Server Port

Once again these values will be specific to your environment.  I have added fictitious values here.  You may see something like SAPMS<ENV> but once again your mileage may vary.  In all likelihood your TCP Port will not be 1234 (that I made up).  It will likely be 4 digits though.

The most important thing to recognize is that the value in this file (that I have called MSHOST) must match the value that we see in the error message below.  This value will not match any values within your Send Port.

 

It isn’t only BizTalk that will use these values.  Your SAPGUI will also use these same values when connecting to SAP systems.  So if you are stuck, my recommendation is to find a machine that is running the SAPGUI and then copy over those values to the file that exists on your BizTalk Server(s).  I have no idea how this MSHOST value is looked up as the BizTalk Send Port has no knowledge of it. My guess is that at runtime the R3Name in combination of the Message Server Host is somehow used to lookup this MSHOST from this file.

 

Conclusion

Hopefully this blog post will save you some time.  I know that I had a tough time finding information that clearly painted the picture for me...especially in the area of the port# that is found in our services file.

Presentation from recent talks

In the past week I have had the opportunity to speak at two events.  On May 1st I spoke at the Phoenix Connected Systems User Group and on May 6th I gave two presentations at the BizTalk Bootcamp in Charlotte.

It was a lot of fun putting these demos together and I received a lot of good feedback so hopefully you will also benefit from them.  You can find the slide deck here: http://sdrv.ms/12fBU53

Note: Some of the Service Bus slides were borrowed from Clemens Vasters’ deck which he has made freely available here.

Upcoming Speaking Engagements

I am happy to announce a couple of upcoming speaking events:

May 1st – Phoenix Connected Systems User Group in Tempe, Arizona

Introducing BizTalk 2013 and the new Cloud adapters

Microsoft recently released their 8th version of BizTalk called BizTalk Server 2013. One of the new capabilities in this version is the ability to use out of the box adapters when communicating with Windows Azure Service Bus endpoints and other Software as a Service (SaaS) solutions. In this session Kent will focus on exposing Line of Business services to the cloud using the new WCF-BasicHttpRelay, SB-Messaging and WCF-WebHttp adapters. This session will include some slides and lots of demos including exposing SAP information to a mobile device using these technologies.

May 6th/7th  - BizTalk Boot Camp in Charlotte, North Carolina

BizTalk 2013 - New cloud-based BizTalk Server adapters

This session will also focus on the new Cloud based adapters for BizTalk Server 2013.  I will be demonstrating some of the new capabilities found in the BasicHttpRelay, SB-Messaging and WCF-WebHttp adapters.

BizTalk 2013 CU1 has been released

The product team recently announced that BizTalk 2013 CU1 has been released. Within the CU there are 3 issues that are fixed:

1. User cannot perform certain database-related operations in BizTalk Server 2013
2. BAM tools cannot be configured in a multi-node BizTalk Server 2013 environment
3. The vertical scroll bar on the target schema does not work correctly when
   you use Visual Studio to design a BizTalk Server 2013 map

Personally, the only issue that I have noticed was item number 2.  It is nice to see this fixed as I was having a horrible time configuring BAM in a multi-server environment.  I definitely recommend installing the update.  This time around, Microsoft is delivering these updates via the Windows Update process.  Overall, the process is painless and rather convenient.  Windows Update will not automatically install these updates, they are something that you need to “opt in” for.

To install the update:

• Launch Windows Update

• Check for updates

• You should discover a BizTalk 2013 section as part of the Important Updates.  If you want additional info, click on the More Info link on right hand side.

 

Note: The KB article indicates A reboot is not required, however I was prompted for a reboot but that must be related to the Windows Server Updates.

BizTalk Monitoring Whitepaper is now available

For the past month and half, in my spare time, I have been working on a whitepaper that compares and contrasts System Center Operations Manager (SCOM) and BizTalk360. 

I have used both products extensively in my professional life and in order to remain objective within the paper I have driven my evaluation off of requirements.  These requirements are based upon my experience of managing the 24x7 operations of a BizTalk environment at a large Power Distribution company in Canada.

The paper can be downloaded from here and feel free to leave any comments that you may have below.

I would also like to thank Steef-Jan Wiggers for being a technical reviewer on this paper.  He provided me with some valuable feedback that increased the quality of the paper.