Monday, May 11, 2015

/nSoftware Powershell Adapter for BizTalk Server

 

In the past I have blogged about /n Software and their SFTP Adapter here and here.  Hard to believe one of those posts goes back to 2007. One thing that /nSoftware continues to do is add new adapters to their suite.  In this case it is the Powershell Adapter.

Can’t say that a Powershell Adapter previously was on my radar until a scenario was brought to me.  We have a very specialized piece of software that does “analysis” (I will leave at that for now).  This software is essentially a library that has been wrapped around an exe.  This exe will receive a series of parameters including a path to a file that it will use to perform its analysis on.

A suggestion was brought up about calling this exe using Powershell.  While I am sure we could call this from .Net the Powershell warranted some investigation.  So sure enough in a web search, /nSoftware comes up with an offering and sure enough we had it installed in all of our environments.

Since BizTalk is going to deliver the flat file used as an input to this process, I decided to check out the Powershell Adapter and allow BizTalk to orchestrate the entire process.  For the purpose of this blog post I will over-simplify the process and focus more on a POC than the original use case.

As part of the POC I am going to receive an xml file that represents our Parameter data.  We will then send this same message out through a Send Port that is bound to the /nSoftware Powershell adapter.

In order to help illustrate this POC, I have a console application that will simply receive 3 parameters and then write the contents to a file in my c:temp folder.  The reason why I am writing to a file is that when I call this exe from Powershell I don’t see the console window being displayed.  Perhaps there is a way to do that but I didn’t look for a solution for that.

namespace PowerShellPOC
{
    class Program
    {
        static void Main(string[] args)
        {

            string[] lines = { args[0], args[1], args[2] };
            // WriteAllLines creates a file, writes a collection of strings to the file,
            // and then closes the file.
            string filename = DateTime.Now.ToString("ddMMyyyymmhhss") + ".txt";
            System.IO.File.WriteAllLines(@"C:\temp\" + filename, lines);

           
        }
    }
}

 

In hindsight, I should have just built a send port subscription but here is my orchestration.

image

Using a standard FILE – receive location

image

On the Send things start to get a little more interesting. We will create a Static One-Way port and select the nSoftware.PowerShell.v4 Adapter.

image

Within our configuration we need to provide a Port Name (which can be anything) and our script.

image

If we click on the Script ellipses we can write our PowerShell script.  In this case we are able to retrieve our message that is moving through our Send Port and pass it into our executable.

image

If we only wanted some data elements we can also use $param3 = $xml.arguments.ReturnType

In this case “arguments” is our root node of our BizTalk Message and “ReturnType” is a child node in our XML Document.

When we go to process a file we will find that our text file has been created and it contains our 3 parameters; 2 that are hard coded and our BizTalk Messsage Payload.

image

Conclusion

When I think of BizTalk, I don’t necessarily think of Powershell.  But there will always be times when you need to perform some function that is a little bit off mainstream.  What I do like about this approach that there was no additional custom dev required to support the solution and we can use the actual BizTalk message in our Powershell script.

I am still exploring the capabilities of the adapter but after a dialog with the /nSoftware team I understand that remote Powershell scripts can be run and we can also use Dynamic ports and Solicit Response ports if we want to return messages from our PowerShell script to BizTalk.

For more details please check out the /nSoftware website.

Saturday, May 9, 2015

BizTalk 2013 SharePoint Adapter not respecting SharePoint 2013 View Name

 

I have done a lot of BizTalk-SharePoint Integration in the past and ran into a situation recently that surprised me. There wasn’t an easily identifiable resolution online so I have decided to document this for the benefit of others.

Background

We have a process that requires a user to approve a financial summary document in SharePoint.  Once the document has been approved, BizTalk will then fetch the details behind those financial transactions, from another source system, and send them to SAP.

In the past I have leveraged SharePoint views as a way for BizTalk to pickup messages from a SharePoint document library. The way to achieve this is to rely upon meta data that can be populated within a SharePoint document library column.

Adding a custom column to a document is very simple.  Under the library tab we will discover the Create Column label. We can simply click this button and then add a column and related properties as required. 

image

With our custom column created, we can now create a view for BizTalk to “watch”.  In our example we were dealing with an approval workflow.  We can create our custom column called Status and then when BizTalk initially publishes this financial summary document(for users to approve), we can use the SharePoint adapter to populate this column with a value of Pending.  After a user has reviewed the document, that Status value can be changed to Approved.

Since we don’t want BizTalk to move Pending documents we will create a view that will only show Approved documents.  To create a custom View we can once again click on the Library tab and then click on Create View

 

image

For our purposes a Standard View will be sufficient.

image

We need to provide a View Name and can also indicate where we want this column to be positioned.

Tip – In my experience I have experienced odd behavior with spaces in the name of SharePoint entities.  My advice is to avoid spaces in names where possible.

image

Lastly, since we only want Approved documents to show up in this field we need to add a filter.

Within our filter we want to Show items only when the following is true:

Status is equal to Approved

image

We can now save our view and test it. To test it we will upload two documents.  One will have the Status of Approved and the other will have a Status of Pending.  When we click on All Documents we should see both documents.

image

When we click on our view for BizTalk, which in this case is called BizTalkMoveView we will only see our Approved document.

image

From a SharePoint perspective we are good and we can now create our SharePoint Receive Location in BizTalk.  For the purposes of this blog post I am using a Send Port Subscription; I will receive the message from SharePoint and then send it to a File folder.

In our BizTalk Receive Location configuration we are going to use the Client OM which in this case is the SharePoint Client Object API.  This allows us to communicate with SharePoint without having to install any agents on a SharePoint Server.

We also need to configure our SharePoint Site URL, Source Document Library URL and View Name

image

When we enable our Send Port and Receive Location we should receive 1 file in our File Folder right? WRONG! Both files were picked up and moved to our file folder even though we have a View enabled.

image

If we go back to SharePoint we will discover both documents are gone.

image

Issue

The issue is that for some reason, BizTalk 2013 is not using/respecting the View Name property that is available in the Receive Location Configuration.

Resolution

The resolution is to install BizTalk 2013 CU 2. The download and more details about CU2 can be found here.

Before you install, the recommended approach from Microsoft is:

  • Stop all host instances
  • Stop SQL Server Agent which is responsible for running the internal BizTalk jobs
  • Perform a Database Backup

Running the CU2 exe is pretty straight forward and only takes a couple minutes.  I wasn't prompted for a reboot but decided to go that route regardless.

After applying the CU, I uploaded two documents again.  One had a Status  of Approved while the other had a Status  of Pending.

image

Our BizTalkMoveView is also displaying our file correctly

image

When we enable our Receive Location we will discover that only our Approved file has moved.

image

image

Our document that was in a Pending state remains in SharePoint as expected.

image

Conclusion

BizTalk 2013 was the first version that had support for the SharePoint Client Object model.  So I am not sure if this bug only impacts when you are using the Client OM within the BizTalk Receive Location.  I do know that in previous versions of BizTalk that this was not an issue.  However those versions of BizTalk relied upon the SharePoint Adapter Service being installed on the remote SharePoint Server.  Using the Client OM is the way to go as it also allows you to communicate with SharePoint Online/Office365.

Tuesday, March 24, 2015

Introducing Azure App Service

If you have been keeping up with cloud based platforms, I am sure that you have realized it has become a very competitive landscape.  Some cloud platforms focus on websites ,infrastructure, and database platforms, while others focus on integration or API management.  We even have Mobile backend as a service platforms that we can choose from. While organizations can choose to cherry pick from each of these different platforms, they can also get first class capabilities, that encompasses all of these domains, from Microsoft’s Azure platform.


At the Integrate 2014 conference in December, 2014 Microsoft gave us a preview of its next generation ‘App’ platform.  At the time, the term Microservice was being used to describe the “architecture style” of the applications that can be built using this new platform.  While the Microservice name did not make its way into the product, the concepts that were discussed at that event have not changed. The day has finally arrived when this Next Generation has emerged in the form of a Preview Service.  The name of this service is called Azure App Service.

For Microsoft this is a PLATFORM PLAY.  No longer do we see disparate product lines in the areas of Integration, API Management, Web Development, Mobility and Workflow.  In addition to these characteristics, we also have the rest of Azure that we can leverage including Service Bus, Analytic Engines(both Microsoft and 3rd party), Databases and Infrastructure.  When you think of all of these different capabilities, using a common platform, common authentication schemes, common tooling, common billing and a global reach for providing these services, Azure is more than just compelling.
Using the Azure App Service we can now create the following types of Apps:

  • Web Apps
    • Build and deploy mission critical Web Apps that can leverage Azure’s hyper-scale capabilities.
  • Mobile Apps
    • The evolution of Azure Mobile Services that allows developers to build engaging Mobile Applications.
  • API Apps
    • Expose your own WebAPIs, consume APIs including 3rd party SaaS based connectors for many popular SaaS platforms.
  • Logic Apps
    • Build workflow application in a declarative manner using a Web browser.

What are some use cases of an API App?
An example of an API app may be a SalesForce connector.  Consider a scenario where you have a mobile application that requires data from SalesForce.  There may be good reasons to place this request on a messaging bus (and don’t worry you still can!) but there may also be good reasons to plug into a connector directly.  Consider a low latency situation where you want to by-pass a monolithic ESB in favor of something lighter.  You could use an API Client library that allows you to consume this SalesForce connector without having a complicated auth scenario.   If you have ever looked at a Mobile Service client, you will feel very comfortable working with API App Client.  When I saw an insider’s demo of this functionality, I was really happy to see the similarities to a paradigm that I am used to.
Similarly, what if you have built an ASP.Net Web API and you want to consume it from a Web Application.  While it can be fun constructing a json message using a String Builder (not really), wouldn’t it be nice to leverage a strongly typed API Client instead?  Yes it would!  So once again much like Azure Mobile Services client we can use something similar in the form of an API App Client.
Another aspect of API apps is that Swagger is a first class citizen of Azure App Service.  We have the ability to generate contracts between the client and API Server which aids in having strongly typed client libraries.

A point worth making is that when building or consuming API apps you are in Visual Studio and there is an accompanying Visual Studio SDK to leverage in order to build or call these API Apps.
Another interesting aspect of this platform is the Cross Platform capabilities.  You can choose to build your APIs in the language of your choice (Java, PHP, Node.js, Python + .Net) and expose them as an HTTP endpoint which you can then consume from other applications.

What are some use cases of a Logic App?
A Logic App is really a workflow application.  Personally, I am not a big fan of Logic App name but it is preview so maybe it will change.  One of the first questions that comes to mind is this workflow based upon Windows Workflow and the answer is no.  Perhaps this is why Microsoft stayed away from using “Workflow App”.

What is interesting about a Logic App is that this is a Workflow that you would construct in a Web Browser without writing code. While some people have made some references to IFTTT (If This Then That), Microsoft feels that this is a much more robust, Enterprise grade, platform.
Below is a screenshot of a “work in progress” Logic App. In this example we can retrieve records from SQL Server, create contacts in Salesforce and then send a welcome tweet all without writing a line of code.

In order to add a connector we simply drag and drop it onto our Logic App canvas and then configure it.
image
A question you may be asking is well if I am composing a workflow in the cloud, how can I add some version control around this?  Behind the scenes, the Logic App Workflow itself is based upon JSON and we have the ability to save/export this JSON data and subsequently store in TFS/GIT etc.


image

So where does BizTalk(Integration) fit into this platform?
The BizTalk brand continues to exist both on-premise in its BizTalk Server sku but also in the form of an API App or what is being called a BizTalk API App.  Since building integration applications requires more than just connectivity we can expect to still perform valuable functions such as:

  • Rules Engine
  • Batching/Debatching
  • Encoders/Decoders
  • VETR messaging patterns (Validate, Extract, Transform, Route)
  • EDI
    • X12
    • EDIFACT
    • AS2
  • Hybrid Connectivity

We can drag these BizTalk API Apps onto our Logic App Canvas much like any other connector. I have highlighted a few of them.


image


It is important to remember that this is a Platform play.  BizTalk, within Azure, now participates in the broader platform.  You can think of the platform as a series of building blocks.  As a developer you now have the ability to select the different building blocks that you need in order to build a compelling solution.  As an example of the breadth of this platform, you do not see Workflow currently listed in the BizTalk section, but that doesn’t mean that the capability doesn’t exist.  This is a capability that is part of Logic Apps. As a BizTalk person we need to start thinking more about Azure and that is a good thing due to the vast amount of capabilities that exist.  Let’s be honest, in the past BizTalk seemed to be a distant cousin of Azure, but it clearly has a place at the family table in this new paradigm.

This is good news for developers.  In the past BizTalk has been accused of being too bulky of a platform with too much ‘lift’ required for developers to be productive.  With this new platform Microsoft is democratizing integration and bringing the toolset that BizTalk Developers are used to using to a broader audience.  This will enable more use cases that involve BizTalk and as a result we will see more integrated Web and Mobile applications leveraging BizTalk capabilities.
In addition to these core BizTalk API capabilities we will also see Connectors in the form of SaaS Connectors, Premium Connectors and Protocol Connectors.
SaaS Connectors

  • Office 365
  • SalesForce
  • Sugar CRM
  • OneDrive
  • DropBox
  • Marketo
  • Facebook
  • Box
  • QuickBooks
We will also see Premium Connectors in the form of:
  • SAP
  • Seibel
  • Oracle
  • DB2
and Protocol Connectors in the form of:
  • FTP
  • SFTP
  • POP3/IMAP
  • SMTP
  • HTTP
  • SOAP

image
Conclusion
I have heard many people refer to this release as the “Back to BizTalk 2004” release.  What I mean by this is that back in 2004, Microsoft released BizTalk which at the time was the most modern Integration platform available.  While we are still in preview and there is a lot to learn, there are a lot of expectations about this release.  The current version of BizTalk is extremely robust, but it could benefit by having some very modern capabilities and being able to efficiently run in the cloud.  What we are seeing today are the initial steps of BizTalk once again becoming the most modern Integration platform available. 


One particular pattern that I am going to be interested in exploring is the hybrid connectivity scenarios that exist.  As you may have noticed above, we do have an Azure Service Bus Connector.  We also have an Azure Service Bus Adapter in BizTalk Server.  So suppose you have an async messaging requirement that involves on-premises systems but you want to tap into one of these cloud based systems.  Well now you can leverage the Service Bus to ‘bridge’ your on-prem envionment with the Azure App Platform Service where we can then leverage one of these fancy new SaaS connectors to talk to SaaS system like Box, Marketo, Dropbox etc without having to do any custom work in BizTalk Server.  It allows you to take advantage of robust on-premise messaging while tapping into modern, SaaS base connectivity and allows you to migrate these workflows to the cloud at a cadence that you are comfortable with and that make business sense.

image


It is also worth mentioning that I am just scratching the surface on these new capabilities and we are still in Preview mode.  I think being a Microsoft platform developer is going to get even more exciting than it already is.  I think developers will be able to build some compelling, mobile, connected apps using a vast array of building blocks.
Expect much more to come on this blog regarding this release.

Saturday, March 7, 2015

Upcoming Presentation: BizTalk Boot Camp

 

BizTalk Boot Camp 2015 – Microsoft Campus, Charlotte North Carolina


In addition to me speaking at the BizTalk Summit in London,  I will also be speaking about Azure API Management at the BizTalk Boot camp at the Microsoft campus in Charlotte on April 29th/30th.

I had the opportunity to speak at this event two years ago and was happy to come back.  The event is being organized by Mandi Ohlinger who works for Microsoft and is responsible for a lot of the technical content that you find on Azure BizTalk Services and BizTalk Server.

This is a free event and registration is required.

My topic will be an introduction to Azure API Management and how you can leverage this new Azure capability with your existing BizTalk Services.

Upcoming Presentation: BizTalk Summit

 

BizTalk Summit 2015 -  April 13/14 London, England

The BizTalk Summit has become an annual, must attend, event in London.  This event is once again being organized by BizTalk360 in conjunction with Microsoft and the BizTalk Product group.  BizTalk360 has organized several large events and I expect this one to be even bigger than some of the other events they have hosted including the very successfully Integrate 2014 event they put on in Redmond in December 2014.

There have been over 250 attendees registered and you can still take advantage of some discounted tickets being available.  This event provides a great opportunity to learn and network with Integration experts from all over the world including Microsoft BizTalk and Service Bus product team members.  Please visit this link for registration information.

My buddy Sandro Pereira has put together a blog post called Top 7 reasons to attend BizTalk Summit 2015 which I thought was an interesting read.  You can take a look at that article here.

Recently, some of the BizTalk Summit session abstracts have been posted, including mine, so I thought I would update my blog as my the sessions were not public when I previously announced I would be speaking.

My topic is on API Management and more specifically Azure API Management.  You can read the details below.

Session Abstract

API Management Part 1 – An Introduction to Azure API Management

Building APIs is not just about technology. APIs enable many new business opportunities, but only if done correctly. As a result of lucrative opportunities, many Software vendors have emerged or pivoted from their SOA management roots to provide API Management capabilities. These API Management platforms provide the building blocks behind a successful API program.

In this session, Kent will introduce you to Microsoft’s Azure API Management platform by providing an overview that highlights its capabilities and the opportunities that emerge for organizations. As part of this presentation, Kent will demonstrate how developers can create their first API and discuss strategies for transforming existing services to leverage Azure API Management.

This presentation will consist of general guidance on API Management, an Azure API Management portal walk-through and demos that re-enforce the concepts that were introduced.

You may be asking yourself – “Part 1” well what is “Part 2”?  There will be another session that includes Azure API Management by Tomasso Groenendijk. We have been working together to ensure there is no overlap and he will pick up where I leave off from an Azure API Management capability perspective.  

image

Saturday, February 7, 2015

BizTalk Summit 2015 – London

Last year I had the opportunity to participate in the BizTalk Summit in London.  It was a great event with more than 200 attendees from 20 different countries present.  For a recap of this trip please check out this blog post.

I was recently asked if I was interested in coming back to speak at this year’s event which I gladly accepted.

More details will be available in coming weeks but the agenda is shaping up to be excellent with sessions from the BizTalk Product group, other Microsoft employees and several Microsoft Integration MVPs. I think this event will be even bigger than the recent Integrate event in Redmond which was a big success.  If you are in Europe, you do not want to miss this event.

My session abstract has been put forward and should be up on the event website in the coming days so I don’t want to ruin that surprise, but you can expect it to be focused on Azure API Management and how it plays a role in a BizTalk developer’s toolset.

LondonSummit -speakers-badges

Tuesday, January 27, 2015

Upcoming Presentation – February 2nd, 2015

Recently Integration MVPs Michael Stephenson and Saravana Kumar have launched a series of community-focused integration events.  What is unique about these events is that they have been designed to cater to an International audience.  Live Webinars have been held every Monday evening (UK time) for the past couple weeks and for several more in the near future.  They have created a website, appropriately called http://www.integrationusergroup.com/ where you can find all the details for these events.

When Michael asked if I was willing to present the talk I gave at the Integrate 2014 event I was happy to oblige.  On Monday, February 2nd I will be discussing “What to look for in an Integration Platform”.  I have included the overview for the session below and want to take this opportunity to thank Michael, Saravana and the BizTalk360 team for hosting these events and providing a great service to the Integration community.

Overview

The landscape for integration platforms continues to evolve. Cloud, Hybrid, Mobility, SaaS connectivity, API Management and MicroServices are introducing new architectural patterns and requirements. These new capabilities are also introducing new entrants in the integration platform industry. For customers who are looking to upgrade, or adopt a new integration platform, this can be an overwhelming and confusing situation. In this talk, Kent will take a requirements driven approach when selecting an integration platform.

http://www.integrationusergroup.com/?event=what-to-look-for-in-an-integration-platform&event_date=2015-02-02