European Trip Recap

 

I recently returned back from Europe where I had a chance to participate in two extraordinary events: Bouvet BizTalk Innovation Day (s) in Stavanger, Norway and the 40th running of the Berlin Marathon.

Bouvet BizTalk Innovation Day–Norway Recap

This was a two day event hosted by Bouvet. For those of you who are not familiar with Bouvet, Bouvet provides services in the fields of information technology, digital communication and enterprise management. Bouvet has about 900 employees divided between 14 offices in Norway and Sweden. - See more at: http://www.bouvet.no/en/About-Bouvet/

On day one each of the speakers had the opportunity to present their topic to a crowd of around 70 BizTalk professionals from all over Scandinavia.  The topics ranged from newer technologies like Windows Azure BizTalk Services, Windows Azure Mobile Services, Windows Azure Service Bus and some more universal topics like being proactive when monitoring the health of BizTalk solutions, BizTalk Mapping Patterns, identifying and rescuing a BizTalk Hostage Project and seeing a preview of the next version of BizTalk360.  There was also a special keynote by Paolo Salvatori who works for Microsoft Italy and is recognized world wide for his abilities. All presentations were very well received as indicated by the attendee surveys.

My presentation focused on Enterprise Mobility.  This is a topic that I have been dealing with at my day job so I had an opportunity to demonstrate some of the areas of enterprise mobility that I have been thinking about lately.  It was also an opportunity to demonstrate a ‘reference’ application that I have been collaborating on with Mikael Hakansson.

Some of the core principles that I have taken into consideration when dealing with Enterprise Mobility include:

• Active Directory Federation: When a person leaves the company and their AD account has been disabled, this “tap” should be turned off for other mobile/cloud based services.
• Leverage a Mobility platform to reduce the diversity required in supporting multiple platforms.  Windows Azure Mobile Services helps us address this by providing APIs for the popular platforms that allow us to centralize activities like Data access, Authentication, Identity Providers, Custom APIs and Scheduled tasks.
• Most, if not all, Enterprise Mobile apps need to consume Line of Business (LOB) System data.  Windows Azure BizTalk Services (and the BizTalk Adapter Service) allow us a secure way in and out of our enterprise without poking holes in firewalls.  I should note that these capabilities are also available with BizTalk Server 2013.
• Accessing On-Premise LOB systems isn’t possible (in my scenarios) without the underpinnings of the Windows Azure Service Bus.  Using this technology to span network layers never gets old. The BizTalk Adapter Service has a strong dependency on these services.
• Data Storage:  Even though I am leveraging SAP master data in this scenario, I do need to maintain the state of the business process.  In this case I am using SQL Azure to host our data.  We can leverage Windows Azure Mobile Services’ APIs that make getting data in and out of the database a breeze.
• Finally, we can’t forget about Toast Notifications.  We want the ability to send notifications out to users (in this case approvers) and Windows Azure Mobile Services helps us deal with sending Toast Notifications to a variety of platforms. 

Here is one of the scenarios from my demo that illustrates many of the principles that were previously mentioned.

     

A few screenshots of the application running in the Windows Phone Emulator:

This was one of the more challenging demos that I have ever been involved in.  I had a lot of fun working on this reference app with Mikael and learned a lot in the process.  My complete slide deck can be found here.

Conclusion

Many people work on bringing events alive, but two people who I would like to recognize are Tord Glad Nordahl and Anders Stensland.  They, in addition to the support Bouvet provided, pulled off a fantastic event.  I have had the opportunity to present in Sweden in 2010 and 2011 and I continue to be amazed by the amount of BizTalk interest in Scandinavia.  If you do have the opportunity to attend the Bouvet BizTalk Innovation conference in the future, I highly recommend it.  They did an amazing job.

40th Berlin Marathon

One of my hobbies is running.  I am a pretty average runner but I enjoy the challenges of running and also try to reap the health benefits of staying active.  I have run over 12 half marathons over the past 6 years and finished my first marathon last year in Chicago.  Whenever I have gone to Europe to speak in the past I have always tried to make a side trip within Europe to experience another culture.  In speaking with one of the other presenters (Steef-Jan Wiggers) we had decided that we would head to Berlin after the conference in Norway.  He recommended going to Berlin to experience its rich history.  Having never been to Germany, myself and my wife made plans to join him in Berlin.

I knew that the Berlin Marathon was held in late September.  The Berlin Marathon is one of the 6 major Marathons in the world.  The others include New York, Boston, Chicago, London and Tokyo.  So when I found out that I would be in Berlin on the same day of this historic event, I couldn’t resist the temptation of participating in this event.

The registration deadline had passed but I was able to find a travel agent from Boston who would sell us packages.  With this information, I presented the opportunity to Steef-Jan and he obliged.  He has recently gotten back into running and this would provide a great opportunity to run his first marathon.

The event itself was rather amazing.  Over 42 000 runners participated in the event with an estimated 1 million spectators.  It was an awesome experience and one that I will never forget.  I finished the marathon in 4 hours 34 minutes and 56 seconds which was 4 minutes faster than my Chicago time.

 

A few pictures:

The prize

 

Before the race.  The garbage bags helped keep us warm while we waited for our turn.

Steef-Jan before the race

 

After the run

 

Celebrating – German style

 

After the race the Adidas store would engrave your time into a running band that was provided as part of your registration.

 

 

One of the best parts of the MVP program is the people you meet and the friendships that you develop.  Without being in the MVP program, this trip would have never happened.  Being part of the program is truly an honor.

Thanks Tord for your hospitality in Norway.  It was a great opportunity to experience my Norwegian heritage and I thoroughly enjoyed your beautiful country. 

Thanks Steef for being an amazing tour guide while in Germany.  Your German came in handy many times and I learned a lot about German history while I was there.  Running the marathon with you was also a great experience.  Next time we won’t do as much sightseeing the day before the race .

I also would like to thank the other MVPs (Sandro, Nino, Saravana) and Paolo for a great experience as well.  Talking shop whenever we get together is a lot of fun and always interesting. 

BizTalk360 - Monitoring Service High Availability Feature

 

I recently went through my second implementation of BizTalk360 and ran into a feature that I wasn’t previously aware of. Typically I have installed BizTalk360 on a BizTalk Server itself which posses a bit of a risk if you only install it on one BizTalk Server and that BizTalk Server happens to be offline. 

My current environment consists of a multi-node cluster (an actual cluster with Failover Cluster Services).  I recently asked the question to Saravana Kumar if this was the way to go when looking for a redundant monitoring solution.  He indicated that my idea would work and is completely supported however I may want to look into a new feature called Monitoring Service High Availability.  When using this feature, BizTalk360 itself is maintaining its state by storing it in the database.  In my case, One node will be active and the second node will be passive – much like a service being managed by Windows Failover clustering.

To access this feature click on the Settings link in the top right hand corner of the screen.

Next, click on the Monitoring Service High Availability link.

Even though the BizTalk360 Service is actively running on both Servers (in my case), BizTalk360 is designating one of the servers as being the primary.

We have the ability to change the primary server by selecting it and then clicking on the Bring Server Active button.

Instantly our primary will switch to becoming a secondary and vice-versa.  This was very quick.  Much quicker than I have experienced failing over a service using Windows Failover Clustering.

The next test is to take our primary Service (or Server Offline).  To do this I will just stop the BizTalk360 service.  By doing so I am simulating what would occur if our service stopped or we lost our entire primary server.  To make this test even more real I am going to enable a test alert, make sure I receive the first alert and then stop the BizTalk360 Service.  My expectation is that my second node will become primary and I should receive another test alert.  This time the alert will be generated from the newly activated node.

 

Below I have configured an existing alarm to function in  TEST MODE.

I have received my alert as expected.

I will now stop the BizTalk360 Service on Node 1.

If I navigate back to the Monitoring Services High Availability screen I find that my “Node 2” is now the active server and my “Node 1” is no longer participating as it is offline.

If I check my inbox, I find that I continue to receive these “TEST Alerts” from BizTalk360.  This time the alerts are coming from my 2nd Node.

If we now go back to my 1st Node and start the BizTalk360 Service, we will discover that BizTalk360 has recognized that the service is back online but is in a passive state.

Conclusion

I have been around Windows Fail-over Clustering for quite some time and am comfortable working within that environment.  The BizTalk environments that I have used in the past also tend to leverage Windows Failover Clustering in order to support Cluster Host Instances for adapters such as S/FTP, POP3 and Database Polling.  Using Windows Failover Clustering is an option for ensuring BizTalk 360 is online and redundant, but it is not a pre-requisite.  As I have demonstrated in this post; BizTalk360 provides this functionality out of the box.  This is great news, especially for those who have multi-node BizTalk environments but do not have (or need) Windows Failover Clustering.  This allows you piece of mind in the event one of your BizTalk Servers goes offline, that you can have BizTalk360 installed on another node your coverage will not be interrupted.  Kudos to the BizTalk360 team for building such an important feature and making it very easy to use!

Dynamics AX 2012 R2 File based integration with BizTalk Server 2013

 

I am currently involved in a project where I need to integrate Dynamics AX with a 3rd party payroll system.  Dynamics AX 2012 provides a rich integration framework called Application Integration Framework or (AIF).  One of the core strengths of this framework is the ability to generate a typed schema through a configuration wizard.  Combine that with a AX’s ability to create inbound and outbound ports and you now have the ability to generate export(and import if interested) files rather quickly.

When I mentioned in the previous paragraph “typed messages” I meant that AX will generate XSD schemas that we can include in our BizTalk projects.  This is a breath of fresh air compared to some other ERP systems where you get handed a CSV flat file that you have to build a flat file schema for.

In my scenario I was receiving a list of Work Orders so a colleague working on the AX side was able to provide me with the Work Order Schema and an imported schema that includes AX types.  At this point I add the schemas into my solution, build my map and wire everything up.  Go to run an initial test and was presented with the following error:

Details:The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.

This is a pretty standard error message that basically comes down to BizTalk received a message that it was not expecting.  The reason why BizTalk was not expecting it was because AX wraps outbound messages with a SOAP Envelope as you can see in the image below.

 

SOAP Envelopes are certainly nothing new but I didn’t expect AX to use them when writing a file to the file system.  When receiving Web/WCF Services BizTalk automatically takes care of extracting the message body from the incoming SOAP message for us.  With the FILE Adapter that facility just does not exist.

You will notice in screenshot below that there is a namespace that is specific to AX.  This got me thinking that AX probably has an XSD for this message type as well.

After digging around a bit I did find the location of the AX schemas to be in the Program files\Microsoft Dynamics AX\60\Server\MicrosoftDynamicsAX\bin\Application\Share\Include folder.  The schema that I was looking for was called Message.xsd

Just adding this the BizTalk project was not enough.  I needed to make a few small tweaks to the schema:

• Click the “Schema Icon” of the schema and then set the Envelope property to be True.  This instructs BizTalk that it is an envelope schema and when BizTalk sees this message that it needs to strip out the Envelope which in this case is a SOAP Envelope.

• Set the Body XPath property by selecting the Root Node of the schema and then populating the appropriate value which in this case is

/*[local-name()='Envelope' and namespace-uri()='http://schemas.microsoft.com/dynamics/2011/01/documents/Message']/*[local-name()='Body' and namespace-uri()='http://schemas.microsoft.com/dynamics/2011/01/documents/Message']/*[local-name()='MessageParts' and namespace-uri()='http://schemas.microsoft.com/dynamics/2011/01/documents/Message']

We can now deploy our application. When it comes to our Receive Location that will be picking up this message, we want to ensure that we are using the XMLReceive Pipeline.  Within this Pipeline the XML Disassembler stage will take care of the SOAP envelope so that when the message body is presented to the MessageBox that  any subscribers will receive the expected message body.

Conclusion

When I first discovered that I was receiving a SOAP wrapped message my instincts said maybe AX could just use a WCF port instead of a FILE port.  This just wasn’t the case, there are only two options when it comes to configuring an outbound port: FILE and MSMQ.  Using MSMQ would not of helped me in this case as the same issue would have existed. 

AX certainly does provide the ability to call a WCF service but it is a more custom based approach.  I would have had to expose this schema as a WCF service but then my AX colleagues would have had to write code against the proxy to populate all of the different data elements.  This would have defeated the purpose of using the AIF framework in order to expedite the ability to delver a solution under very tight timelines.  Luckily with a little tinkering we were able to come up with a reasonable solution without writing custom code.

I have to think that AX is wrapping these messages in a SOAP Envelope for a reason.  Perhaps a WCF outbound port is coming in an upcoming release?

Windows Azure BizTalk Services Preview (Part 2) –BizTalk Adapter Services SAP Integration

 

Over the past 7 years I have spent a lot of time integrating BizTalk and SAP systems.  I probably built more interfaces for SAP than any other system.  When new technology such as Windows Azure BizTalk Services surfaces I am always interested to understand what the experience looks like.

If you read my prior post in this series, you are probably wondering how it is possible to use Windows Azure BizTalk Services in the Windows Azure Cloud and connect to your On-Premise LOB Systems like SAP?  Enter BizTalk Adapter Services.

BizTalk Adapter Services allows us to use the BizTalk Adapter Pack in order to connect to LOB systems.  The BizTalk Adapter Services are hosted in IIS and leverage the Service Bus Relay (under the hood) in order to traverse Firewalls and NATs.  Using the BizTalk Adapter Service is just another tool in our toolbox when achieving Hybrid integration.

Adapter Services Installation

I am now going to quickly install the BizTalk Adapter Services and then provide a walkthrough of how we can integrate with SAP.  The scenario that we are going to walk through is part of a larger process that I won’t get into much detail within this blog but we have an SAP process were we need to feed sub-systems within the organization with Equipment data from SAP.  We will get this equipment by calling a custom SAP RFC(BAPI).

The BizTalk Adapter Service is a component that we want to install on a Server within our Network.  This machine does not require BizTalk Server to be present.

 

This is an important step as the account that we specify here will require access to SQL Server where configuration will be stored about our Relay Endpoints.

In this case I am using a local SQL Server instance.  You can use SQL Express if you so desire but I have leveraged SQL 2008 R2 (SQL Server 2012 is also supported).

This step kinda reminds me of the BizTalk SSO Service password but they are not related or linked.

The BizTalk Adapter Service installation will be installing a Management Web Service.  Since I am just using this for demo purposes I did not enable SSL. 

Once we have finished installing the BizTalk Adapter Service we will discover a new database called BAService.

We will also discover a new entry in Server Explorer within Visual Studio called BizTalk Adapter Services. 

If we right mouse click on BizTalk Adapter Services we have the ability to add a BizTalk Adapter Service by specifying the URL of the Management Service that was created as part of the installation process.

Once this Adapter Service has been added we can expand it and discover our LOB Targets.

We can add a new SAP target by right mouse clicking on SAP  and selecting Add SAP Target.

We now need to specify our SAP connection information much like we would if we were adding Generated Items within a BizTalk Server 2013 solution.  We need to specify the Application Server, the System Number, Client and Language.  I recently blogged about connecting to SAP Messaging Servers.  Not to worry these more advanced scenarios are supported by clicking on the Advanced button.

Next we need to provide our SAP credentials if we are not using Windows Credentials.

Much like the BizTalk Server experience we can search operations including IDOCs, BAPIs and RFCs.  Do note that at this point we are only able to push messages to SAP.  This means we can send an IDOC and call a BAPI/RFC and get the response back.  SAP cannot push IDOCs or call an RFC that is hosted by the BizTalk Adapter Service. 

Once we have found the artifact that we are looking for we can add it to the Selected operations list and then click the Next button.

Once again need to provide credentials that the BizTalk Adapter Service will use to connect to SAP.

This gets a little interesting.  Much like in my previous blog post where we connected to a Service Bus Topic, we will use the production Service Bus connectivity information.  So in this case I am going to leverage an existing Service Bus Namespace and credentials to enable Service Bus Relay capabilities.

We also have the ability to specify a Target sub-path that just allows us to organize our endpoints.

We now have a confirmation page that we can just click the Create button in order to finish the wizard.

With our SAP LOB Target created we can now drag that Target onto our canvas:

At this point we have just created an SAP target and added it to our canvas. but we have not actually generated our schemas that will be used within our solution.  In order to generate these schemas we need to right mouse click on our SAP LOB Target and specify Add Schemas to Project.

We will once again be prompted for SAP credentials.

Within our solution, we will now find two schemas have been added.  One is the core schema and the other is a schema for data types leveraged by the core schema.

A Web Request Schema, that we will expose externally to calling applications, is also needed.  This schema is very simple with only two fields: Plant and EquipmentName.

Next we need to add a Map that will allow us to transform our Web Request message into our SAP Request message.

The map is pretty straight forward as we only need to map a couple fields.

A Map that will allow us to transform our SAP Response into our Web Response is also required.

Next, we need to add  an Xml Request-Reply Bridge to our canvas by dragging it onto our canvas.

We now want to modify the Property page for this artifact  as it will become part of our overall URI.

By double clicking on our  Bridge, we have the ability to configure it.  Within this configuration we will want to specify the Message Type for the message that we will be receiving from the calling application and the Message Type for the type of message we will be sending back to the calling application.

We also want to specify our two Transformations.  The first one will take our Web Request and map it to our SAP Request.  The second Transformation will take our SAP Response and map it to our Web Response.

Once our Bridge has been configured, we need to connect our Request Response Bridge to our SAP LOB Target.  We can do so by dragging a Connector from our Toolbox and connecting our Bridge to our SAP LOB Target.

Routing messages is a core functionality within BizTalk Services. In order to route messages, we need to provide a filter condition that will allow us to specify messages to flow between the Bridge and the SAP LOB Target. In order to set this filter we need to click on the line that connects these two artifacts and then click on the Filter Condition.

Since we want all messages to flow to this LOB system we will choose Match All and click the OK button.

We also need to specify a SOAP Action much like we need to do with BizTalk Server.  In order to do this we need to highlight the connection between the Bridge and LOB target and then click on Route Action.

In this case we want to specify a SOAP Action and then specify the action from our Request Schema.  Much like the EAI/EDI Labs solutions we want to wrap our Expression around single quotes( ‘   ‘)

We are now almost ready to deploy our solution but before we can do that we need to specify our Deployment Endpoint.  We can do so by clicking on any blank space on our canvas and then specifying our Deployment Endpoint address in the Property page.

Deploying our solution is as simple as right mouse clicking on our Visual Studio Solution file and selecting Deploy Solution.

In order to deploy our solution we will need to provide the Deployment Endpoint, ACS Namespace(for BizTalk Service environment), Issuer Name and Issuer Shared Secret.

Within a few seconds our solution will be deployed and will be ready for testing. Once again I can  use the MessageSender application that is part of the BizTalk Services SDK. But, since looking at a black screen with white xml text isn’t very appealing, I created an ASP.Net Web application and borrowed the MessageSender class from the SDK project.  In this case I have my Web Application running on a laptop that isn’t on the same network as the machine that is hosting the BizTalk Adapter Service.

 

Within the web application, we have the ability to provide a Plant and a wild card string representing a piece of equipment in order to get more details about where that piece of equipment has been deployed, the Manufacturer and Model Number.

 

Conclusion

Hopefully this walkthrough has provided you with some valuable insight on what it takes to integrate a Line of Business system like SAP with Windows Azure BizTalk Services.  I can certainly envision scenarios where an organization uses an LOB system like SAP but may not have a dedicated integration platform to perform some of this integration. Windows Azure BizTalk Services may fill this void and enable some business scenarios that just may not have been possible for that organization.  Consider another situation where you may have a SalesForce system that requires SAP data (maybe Equipment data ) and this type of functionality in BizTalk Services really becomes a business catalyst.

I also think that the BizTalk Adapter Service may provide some interesting Mobile use cases as well as we can now expose SAP data through the cloud in a secure manner.

While the scenario is a bit dated, I also wrote a similar article on how to submit IDOCs to SAP using the Windows Azure EAI/EDI Labs CTP which has now been superseded by Windows Azure BizTalk Services.