Introducing Azure App Service

If you have been keeping up with cloud based platforms, I am sure that you have realized it has become a very competitive landscape.  Some cloud platforms focus on websites ,infrastructure, and database platforms, while others focus on integration or API management.  We even have Mobile backend as a service platforms that we can choose from. While organizations can choose to cherry pick from each of these different platforms, they can also get first class capabilities, that encompasses all of these domains, from Microsoft’s Azure platform.

At the Integrate 2014 conference in December, 2014 Microsoft gave us a preview of its next generation ‘App’ platform.  At the time, the term Microservice was being used to describe the “architecture style” of the applications that can be built using this new platform.  While the Microservice name did not make its way into the product, the concepts that were discussed at that event have not changed. The day has finally arrived when this Next Generation has emerged in the form of a Preview Service.  The name of this service is called Azure App Service.

For Microsoft this is a PLATFORM PLAY.  No longer do we see disparate product lines in the areas of Integration, API Management, Web Development, Mobility and Workflow.  In addition to these characteristics, we also have the rest of Azure that we can leverage including Service Bus, Analytic Engines(both Microsoft and 3rd party), Databases and Infrastructure.  When you think of all of these different capabilities, using a common platform, common authentication schemes, common tooling, common billing and a global reach for providing these services, Azure is more than just compelling.
Using the Azure App Service we can now create the following types of Apps:

• Web Apps
  • Build and deploy mission critical Web Apps that can leverage Azure’s hyper-scale capabilities.
• Mobile Apps
  • The evolution of Azure Mobile Services that allows developers to build engaging Mobile Applications.
• API Apps
  • Expose your own WebAPIs, consume APIs including 3rd party SaaS based connectors for many popular SaaS platforms.
• Logic Apps
  • Build workflow application in a declarative manner using a Web browser.

What are some use cases of an API App?
An example of an API app may be a SalesForce connector.  Consider a scenario where you have a mobile application that requires data from SalesForce.  There may be good reasons to place this request on a messaging bus (and don’t worry you still can!) but there may also be good reasons to plug into a connector directly.  Consider a low latency situation where you want to by-pass a monolithic ESB in favor of something lighter.  You could use an API Client library that allows you to consume this SalesForce connector without having a complicated auth scenario.   If you have ever looked at a Mobile Service client, you will feel very comfortable working with API App Client.  When I saw an insider’s demo of this functionality, I was really happy to see the similarities to a paradigm that I am used to.
Similarly, what if you have built an ASP.Net Web API and you want to consume it from a Web Application.  While it can be fun constructing a json message using a String Builder (not really), wouldn’t it be nice to leverage a strongly typed API Client instead?  Yes it would!  So once again much like Azure Mobile Services client we can use something similar in the form of an API App Client.
Another aspect of API apps is that Swagger is a first class citizen of Azure App Service.  We have the ability to generate contracts between the client and API Server which aids in having strongly typed client libraries.

A point worth making is that when building or consuming API apps you are in Visual Studio and there is an accompanying Visual Studio SDK to leverage in order to build or call these API Apps.
Another interesting aspect of this platform is the Cross Platform capabilities.  You can choose to build your APIs in the language of your choice (Java, PHP, Node.js, Python + .Net) and expose them as an HTTP endpoint which you can then consume from other applications.

What are some use cases of a Logic App?
A Logic App is really a workflow application.  Personally, I am not a big fan of Logic App name but it is preview so maybe it will change.  One of the first questions that comes to mind is this workflow based upon Windows Workflow and the answer is no.  Perhaps this is why Microsoft stayed away from using “Workflow App”.

What is interesting about a Logic App is that this is a Workflow that you would construct in a Web Browser without writing code. While some people have made some references to IFTTT (If This Then That), Microsoft feels that this is a much more robust, Enterprise grade, platform.
Below is a screenshot of a “work in progress” Logic App. In this example we can retrieve records from SQL Server, create contacts in Salesforce and then send a welcome tweet all without writing a line of code.

In order to add a connector we simply drag and drop it onto our Logic App canvas and then configure it.

A question you may be asking is well if I am composing a workflow in the cloud, how can I add some version control around this?  Behind the scenes, the Logic App Workflow itself is based upon JSON and we have the ability to save/export this JSON data and subsequently store in TFS/GIT etc.

So where does BizTalk(Integration) fit into this platform?
The BizTalk brand continues to exist both on-premise in its BizTalk Server sku but also in the form of an API App or what is being called a BizTalk API App.  Since building integration applications requires more than just connectivity we can expect to still perform valuable functions such as:

• Rules Engine
• Batching/Debatching
• Encoders/Decoders
• VETR messaging patterns (Validate, Extract, Transform, Route)
• EDI
  • X12
  • EDIFACT
  • AS2
• Hybrid Connectivity

We can drag these BizTalk API Apps onto our Logic App Canvas much like any other connector. I have highlighted a few of them.

It is important to remember that this is a Platform play.  BizTalk, within Azure, now participates in the broader platform.  You can think of the platform as a series of building blocks.  As a developer you now have the ability to select the different building blocks that you need in order to build a compelling solution.  As an example of the breadth of this platform, you do not see Workflow currently listed in the BizTalk section, but that doesn’t mean that the capability doesn’t exist.  This is a capability that is part of Logic Apps. As a BizTalk person we need to start thinking more about Azure and that is a good thing due to the vast amount of capabilities that exist.  Let’s be honest, in the past BizTalk seemed to be a distant cousin of Azure, but it clearly has a place at the family table in this new paradigm.

This is good news for developers.  In the past BizTalk has been accused of being too bulky of a platform with too much ‘lift’ required for developers to be productive.  With this new platform Microsoft is democratizing integration and bringing the toolset that BizTalk Developers are used to using to a broader audience.  This will enable more use cases that involve BizTalk and as a result we will see more integrated Web and Mobile applications leveraging BizTalk capabilities.
In addition to these core BizTalk API capabilities we will also see Connectors in the form of SaaS Connectors, Premium Connectors and Protocol Connectors.
SaaS Connectors

• Office 365
• SalesForce
• Sugar CRM
• OneDrive
• DropBox
• Marketo
• Facebook
• Box
• QuickBooks

We will also see Premium Connectors in the form of:

• SAP
• Seibel
• Oracle
• DB2

and Protocol Connectors in the form of:

• FTP
• SFTP
• POP3/IMAP
• SMTP
• HTTP
• SOAP

Conclusion
I have heard many people refer to this release as the “Back to BizTalk 2004” release.  What I mean by this is that back in 2004, Microsoft released BizTalk which at the time was the most modern Integration platform available.  While we are still in preview and there is a lot to learn, there are a lot of expectations about this release.  The current version of BizTalk is extremely robust, but it could benefit by having some very modern capabilities and being able to efficiently run in the cloud.  What we are seeing today are the initial steps of BizTalk once again becoming the most modern Integration platform available. 

One particular pattern that I am going to be interested in exploring is the hybrid connectivity scenarios that exist.  As you may have noticed above, we do have an Azure Service Bus Connector.  We also have an Azure Service Bus Adapter in BizTalk Server.  So suppose you have an async messaging requirement that involves on-premises systems but you want to tap into one of these cloud based systems.  Well now you can leverage the Service Bus to ‘bridge’ your on-prem envionment with the Azure App Platform Service where we can then leverage one of these fancy new SaaS connectors to talk to SaaS system like Box, Marketo, Dropbox etc without having to do any custom work in BizTalk Server.  It allows you to take advantage of robust on-premise messaging while tapping into modern, SaaS base connectivity and allows you to migrate these workflows to the cloud at a cadence that you are comfortable with and that make business sense.

It is also worth mentioning that I am just scratching the surface on these new capabilities and we are still in Preview mode.  I think being a Microsoft platform developer is going to get even more exciting than it already is.  I think developers will be able to build some compelling, mobile, connected apps using a vast array of building blocks.
Expect much more to come on this blog regarding this release.

Upcoming Presentation: BizTalk Summit

 

BizTalk Summit 2015 -  April 13/14 London, England

The BizTalk Summit has become an annual, must attend, event in London.  This event is once again being organized by BizTalk360 in conjunction with Microsoft and the BizTalk Product group.  BizTalk360 has organized several large events and I expect this one to be even bigger than some of the other events they have hosted including the very successfully Integrate 2014 event they put on in Redmond in December 2014.

There have been over 250 attendees registered and you can still take advantage of some discounted tickets being available.  This event provides a great opportunity to learn and network with Integration experts from all over the world including Microsoft BizTalk and Service Bus product team members.  Please visit this link for registration information.

My buddy Sandro Pereira has put together a blog post called Top 7 reasons to attend BizTalk Summit 2015 which I thought was an interesting read.  You can take a look at that article here.

Recently, some of the BizTalk Summit session abstracts have been posted, including mine, so I thought I would update my blog as my the sessions were not public when I previously announced I would be speaking.

My topic is on API Management and more specifically Azure API Management.  You can read the details below.

Session Abstract

API Management Part 1 – An Introduction to Azure API Management

Building APIs is not just about technology. APIs enable many new business opportunities, but only if done correctly. As a result of lucrative opportunities, many Software vendors have emerged or pivoted from their SOA management roots to provide API Management capabilities. These API Management platforms provide the building blocks behind a successful API program.

In this session, Kent will introduce you to Microsoft’s Azure API Management platform by providing an overview that highlights its capabilities and the opportunities that emerge for organizations. As part of this presentation, Kent will demonstrate how developers can create their first API and discuss strategies for transforming existing services to leverage Azure API Management.

This presentation will consist of general guidance on API Management, an Azure API Management portal walk-through and demos that re-enforce the concepts that were introduced.

You may be asking yourself – “Part 1” well what is “Part 2”?  There will be another session that includes Azure API Management by Tomasso Groenendijk. We have been working together to ensure there is no overlap and he will pick up where I leave off from an Azure API Management capability perspective.  

Slides from Calgary Azure Dev Camp

On January 10, 2015 I had the opportunity to speak about the Azure Service Bus.  This was an introductory presentation on the subject, but I  was able to sneak in a more complex demo at the end that includes Azure Mobile Services and BizTalk.  I have uploaded the slides to slideshare below.  If you want to see a video of the the final demo you can view that on Channel 9.

Calgary azure dev camp - Service Bus from wearsy

2014 Year in Review and looking ahead to 2015

It is that time of year where I take a step back and reflect on the previous year and then start thinking about the upcoming year.  2014 was an incredible year, one that I will not forget any time soon.  I did a lot of travelling in 2014 and met a lot of great people, for this I am grateful for the opportunities that allowed this to happen.

So let’s get into it, here are some of the events that took place over the past year.

Released another book

Johan Hedberg, Morten la Cour and myself released a second edition of the (MCTS) Microsoft BizTalk Server (70-595) certification book to include new BizTalk 2013 content. We released the book in March, 2014 as it also co-incided with the launch of the Microsoft Partner Assessment exam that Microsoft uses to measure its Silver or Gold Application Integration partners.

If your are interested in getting certified, either personally or as part of the Microsoft Partner network, I still feel that it is a great resource that will aid you in achieving your goal.

The Book is available from both Amazon and Packt. 

European Adventure

In March, I had the opportunity to head to Europe to present at a few events.  These events included the BizTalk Summit in London on March 3rd/4th, Swedish BizTalk User group in Stockholm, Sweden on March 5th and the Netherlands BizTalk User group on March 6th.

Special thanks to Saravana Kumar, Johan Hedberg and Steef-Jan Wiggers for allowing me to participate in these different events in your countries.  The BizTalk community is second to none in Europe so it is always an honour to head over there.

Even though it was an extremely busy week, we did have some time to see the sights in London. But otherwise a lot of time was spent at airports and in airplanes.

MuleSoft

After my return from Europe, I made a career change and joined MuleSoft. Event though MuleSoft has its headquarters in San Francisco I was able to sign on as a Remote Solutions Architect.

In 2014, MuleSoft made a lot of investments in supporting the Microsoft eco-system.  Most of the activities I was involved with included Microsoft at some level. The highlight of my time there would have to be getting involved in the release of a couple of investments related to calling .NET code (.Net Connector) from a Mule workflow and the MSMQ connector. I had the opportunity to demonstrate these components at MuleSoft’s flagship event called Connect in San Francisco in May 2014.

I decided to leave MuleSoft at the end of September 2014.  The logistics of me being a remote worker in Canada created some challenges and as a result I decided to move on.   Overall, I value the time I spent at MuleSoft.  I got to learn about a new eco-system, met a lot of great, smart people and got exposure to API Management which I hadn’t been exposed to previously.

After leaving Mulesoft, I traded my eclipse IDE for Visual Studio.  I have shifted my attention back to the Microsoft Integration stack including BizTalk, Azure Service Bus and Azure API Management. 

MVP Summit

I was able to attend the Annual MVP Summit in Redmond.  It was another great opportunity to talk shop with my integration colleagues from around the globe. Since the Summit has moved to the fall instead of Spring, we have developed a bit of a tradition by heading to Lowell’s, in the Pike Market, for breakfast before heading to a Seattle Seahawks game.  Even though it rained pretty much the whole game it was still a fun way to to spend a Sunday ffternoon.

 

Microsoft had a bit of a surprise for us once the sessions started.  It was at this point that we started to hear more about the next generation platform that they were building.  This more than kept us occupied in many discussions through-out the week. 

 

Channel 9

The amount of brain-power that shows up for MVP Summit is flat-out impressive.  Some of the brightest minds in the industry are present and eager to share some of their experiences.  Mark Mortimore, who runs marketing for BizTalk, decided to tap into some of this knowledge and have people record some short demos in advance of the Integrate 2014 event.  I took this opportunity to discuss a scenario I had built in the spring involving capturing data from a Power Plant and making it available to mobile devices via BizTalk, Azure Service Bus and Azure Mobile Services. You can view my clip and others here.

Integrate 2014

Microsoft held its annual Integration event in North America in December 2014.  I had the opportunity to speak at this event. My topic involved considerations when choosing an Integration Platform.  The Integration Platform landscape has been very crowded lately and there are a lot of mixed messages flying around.  The point of my session was to take a step back an look at a requirements based process to selecting a platform.  I discussed some of the top requirements that I would be looking for in a platform.

I also took this opportunity to announce a Whitepaper that myself, Michael Stephenson and Steef -Jan Wiggers had been working on called Choosing the right Integration Platform. This whitepaper is available for free on the BizTalk360 site.

The Integrate 2014 event had an incredible amount of announcements and gave attendees a lot of insight into where Microsoft is headed with Integration.  You can find a more detailed recap of the event here.

ASU

During all of these activities, I was also pursuing a Master’s Degree.  In December 2014, I graduated from Arizona State University with a 4.0 GPA.  The program that I was enrolled in was called MSIM (Masters of Science in Information Management).  The program is delivered on-campus and online through the W.P. Carey School of Business.  For the past 16 months I have spent approximately 20 hours a week in the program.

It was a great experience and I am glad that I had the opportunity to go through it.  I had some great teammates in my group and we worked really well together even though both of them are in Phoenix and I am in Calgary.

2015

In 2015, I am definitely looking for a slower pace and being able to spend more time with my family. While I will continue to focus on technology with my spare cycles, I definitely plan to spend more time skiing and running this year.  Between school and doing a lot of travel last year these are some of the activities that I had to concede.

From a technology perspective, I certainly plan to dive deeper into the upcoming Microsoft Microservices offering, Azure API Management, Azure Event Hubs and Stream Analytics.

MVP Award

I did receive word that my MVP status has been renewed for 2015.  I want to thank both my Canadian MVP leads Sim and Joel and the Product Group leadership team  Vivek, Guru and Mark for keeping me in the program even though I spent some time away in another platform.  2015 is going to be a really big year for Microsoft in the Integration space and I am thrilled to get involved.

I also want to take this opportunity to thank all my twitter followers and readers of this blog for their continued interest in what I have to say.  The Microsoft Integration community is really amazing and I am inspired to be part of it.  Here’s to a great 2015!

Book Review - Getting Started with BizTalk Services

 

I recently read the Getting Started with BizTalk Services book and decided to blog about my experience with it.  I have test drove earlier versions of BizTalk Services (blog, blog,  and blog) and decided that I wanted to catch up on some of the more recent developments in this space.  Reading this book was a great way to get this type of information from one source.

Let’s first start off with the authors; Jon Fancey and Karthik Bharathy.  Both Jon and Karthik are BizTalk veterans and are both very well respected in the community.  I knew picking up the book that the quality would not be a concern.

The book does not require a reader to possess extensive BizTalk Server experience.  It does help when drawing comparisons between features in Server vs Services.  Even without a lot of Microsoft Integration experience, a reader can be very productive working through the examples in this book.

Within the book, you will discover 8 chapters covering 156 pages.  Each chapter contains some contextual background information followed by easy to follow examples that include: Introduction to BizTalk Services Messages and Transforms Bridges Enterprise Application Integration Business to business Integration Management APIs Tracking and Troubleshooting Moving to BizTalk Services

Even though I have some experience with the BizTalk Services beta and its predecessors I did learn some things from this book.  Probably most valuable chapter for me was Chapter 2 – Messages and Transforms.  While I have used the new “mapper” in BizTalk Services, there were certainly some operations that I haven’t used before including:

• List Operations
• Get Context Properties inside a map
• If then else operation
• Transform Exception Settings.

Another useful chapter for me was Chapter 5 – Business to business Integration.  Even though the BizTalk Services platform is rather young, the BizTalk Services’ EDI capabilities are known to be one of the strengths of the platform.  Since I have not done much in the EDI space, this chapter acted as an EDI Primer and then was able to relate these EDI concepts to the BizTalk Services solution.

Lastly, the Management API chapter was interesting as well.  I have seen Steef-Jan present on the topic, but it was nice to be able to read through some examples on how you can manage your BizTalk Services application.

Conclusion

In the final chapter; Chapter 8 – Moving BizTalk Services discusses some of the current gaps between BizTalk Server and BizTalk Services. The authors drop some subtle hints around what features are coming that will close the feature parity gap that currently exists.  Based upon the agenda for the upcoming Integrate 2014 event, I suspect this gap will be closed rather quickly.  Which means, the timing to pick up this book, to ensure you understand the fundamentals of the BizTalk Services, before some of these other features are announced provides a great opportunity to ensure you can hit the ground running.

The book can be sourced from both Amazon and PacktPub in both e-book and traditional paperback format.   The Kindle version of the book is a mere $8.96 USD which is a incredible bargain.

BizTalk 2010 R2 CTP: Azure Service Bus Integration–Part 5 Sending messages to Service Bus Queues using Sessions

 

What are Service Bus Sessions?

Service Bus Sessions are actually a rather broad subject as there are a few different scenarios in which they can be used.  At its simplest description I consider Service Bus Sessions to be a way to relate messages together.  More specifically here are a few ways in which they can be used:

• To address the Maximum message size constraint.  Service Bus Queues can support messages that have a size of 256 KB or smaller.  Using Sessions allow us to break a larger message down into smaller messages and then send them over the wire.  A consumer, or receiver, can then receive all of these message “chunks” and aggregate them together.
• To support receiving a related set of messages in First In First Out (FIFO) fashion
• Allows for affinity between a consumer and a Service Bus Queue in competing consumer scenarios.  Imagine having 3 consuming clients all trying to receive messages from the same Service Bus Queue.  Under normal circumstances you cannot be assured that one receiver will receive all messages within a message set.  One can expect the messages to be distributed amongst the clients as each consumer “competes” for the right to process a particular message.  In this scenario, once a receiver has started to process a message within a session, that consumer will process all messages within that session barring some sort of application crash.
• In some scenarios, using a Service Bus Session allows for routing of messages.  Within a receiver, you can specify an explicit Session that you are interested in.  So in some ways a Session can be used almost like a filter.  I am not suggesting that this approach be used instead of Service Bus Topics/Subscriptions, but there may be a specific business requirement to do this.

Why are Service Bus Sessions important in BizTalk processing?

BizTalk deals with a variety of different messaging scenarios in many different industry verticals.  Supporting Service Bus Sessions is just another tool in the the BizTalk toolbox for supporting new requirements.  A scenario that I came up with is dispatching messages.  For instance if we wanted to load up a field worker with all of his orders, wouldn’t it be nice to have all of his orders sent to him as a batch?  As opposed to him receiving some of his orders only to receive more orders later on.  For instance he may have driven by one of his customers already because the messages that he receive were out of order and other field workers were also receiving their orders which delayed him in receiving all of his.

Putting it together – Modifying Service Bus Queue

A pre-requisite for this type of messaging scenario to work is configuring our Service Bus Queue to support Sessions.  This can be enabled in a couple different ways:

• When creating a queue from within the Azure Portal, we can check the Enable sessions checkbox.

• When using the QueueDescription class we can set the RequiresSession property to true.

NamespaceManager namespaceClient = new NamespaceManager(serviceUri, credentials);

if (!namespaceClient.QueueExists(Sender.QueueName))
        {

            QueueDescription queueDescription = new QueueDescription(Sender.QueueName)
            {
                RequiresSession = true
            };
            namespaceClient.CreateQueue(queueDescription);
          
        }

BizTalk Configuration

In order to keep the solution very simple, we will create:

• Two Receive Ports
• A Receive Location for each Receive Port.  The purpose of these Receive Locations is to simply inject messages into BizTalk so that we can set the SessionID property on the Send Ports.
• 2 Send Ports
• One for Mike
• One for Joe
• Each Send Port will have a filter for the corresponding Receive Port.  Within each Send Port we will configure a SessionID .

The other Send Port will use the same URI, however it will have a different SessionID value which will be Joe.

 

Service Bus Queue Client

The code below will make a connection to our session-ful Service Bus Queue and retrieve all messages that have a SessionId of Mike.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using System.Runtime.Serialization;
using BrokeredMessageToBizTalk;

namespace RetrieveServiceBusSession
{
    class Receiver
    {

        const string QueueName = "<your_sessionful_queue>";
        static string ServiceNamespace = "<your_namespace>";
        static string IssuerName = "<your_issuerName>";
        static string IssuerKey = "<your_IssuerKey>";
        static void Main(string[] args)
        {
            TokenProvider credentials = TokenProvider.CreateSharedSecretTokenProvider(Receiver.IssuerName, Receiver.IssuerKey);
            Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", Receiver.ServiceNamespace, string.Empty);

            MessagingFactory factory = null;

            factory = MessagingFactory.Create(serviceUri, credentials);

            QueueClient sessionQueueClient = factory.CreateQueueClient(Receiver.QueueName);

            //Create sessionQueueClient and subscribe to SessionIDs that have a value of "Mike"
            MessageSession sessionReceiver = sessionQueueClient.AcceptMessageSession("Mike", TimeSpan.FromSeconds(60));
            BrokeredMessage receivedMessage;

                while ((receivedMessage = sessionReceiver.Receive(TimeSpan.FromSeconds(60))) != null)
                {
                    var data = receivedMessage.GetBody<PowerOut>(new DataContractSerializer(typeof(PowerOut)));
                    Console.WriteLine(String.Format("Customer Name: {0}", data.CustomerName));
                    Console.WriteLine("SessionID: {0}", receivedMessage.SessionId);
                    //remove message from Topic
                    receivedMessage.Complete();
                }
          
            Console.WriteLine("All received on this session...press enter to exit");
            Console.Read();
        }
    }
}

 

The code itself is very similar to that of some of my previous blog posts on ServiceBus integration.  The main difference is instantiating a MessionSession object.

MessageSession sessionReceiver = sessionQueueClient.AcceptMessageSession("Mike", TimeSpan.FromSeconds(60));

Within this line of we are indicating that we want to receive messages that belong to the Mike Session.  We can also provide a TimeSpan as an argument to specify the duration in which we want to receive from this Session.  Setting this value is more useful when we are looking for any available Session as it allows all messages within a Session to be processed before moving onto the next Session.

Testing

I have two sets of messages here.  Two of the messages will be routed through the Joe Send Port and subsequently the SessionID for these two messages will be set to Joe. The other two messages will be routed through the Mike Send Port and subsequently will have its SessionID property set to  Mike.

As mentioned previously, both Send Ports are configured to send to the same Service Bus Queue.  When we do run our client application, the expectation is that messages belonging to the Mike Session will be retrieved.  The Joe Messages will remain in the Queue until another receiver pulls them down or the the Time To Live (TTL) threshold has been exceeded.

When we start our Consumer application we do discover that the “Mike” messages are processed.

So what happened to the other messages?

The “Joe” messages are still in our Queue.  If we navigate to our Windows Azure Portal, we will discover that our Queue Length is set to 2.

So how do we get these messages out?

We have a couple options, we can create another MessageSession instance and retrieve all Messages belonging to the Mike Session or we can not specify a Session and our client will look for the next available Session which in this case will be the Mike Session.

Let’s go with the second option and retrieve the next available session.  In order to do so we need to change the following line of code from

  MessageSession sessionReceiver = sessionQueueClient.AcceptMessageSession("Mike", TimeSpan.FromSeconds(60));

to

  MessageSession sessionReceiver = sessionQueueClient.AcceptMessageSession(TimeSpan.FromSeconds(60));

We essentially are no longer specifying a specific Session that we are interested in and are now interested in any Session.

I will now process another 4 files; 2 will belong to the Joe Session and 2 will belong to the Mike Session.  What we expect to happen is that all 4 Joe messages will be processed since it is the next available Session.

So this proves that we have In Order Delivery occurring at the Session level.  Initially our Mike Session was processed which left our Joe messages outstanding.  We then loaded 4 more messages to the Queue and since the Joe messages were first in, they were processed first.  Our remaining 2 messages that now belong to Mike can be retrieved by starting up our Client Application once again.

Note:

Within my BizTalk Send Ports I statically configured my SessionID.  This isn’t very practical in the “Real World” but it was easy to demonstrate for the purpose of this blog post.  Much like other BizTalk context properties the SessionID property is available and can be set dynamically within an Orchestration Message Assignment shape or a Pipeline Component.

Conclusion

Overall I found this functionality pretty neat. I do think that it is another capability that we can leverage to support more granular control over message processing.  I do like the opportunity to group messages together and treat them as a batch.  This also works when dealing with message size limitations as we can stich a bunch of smaller messages together that collective make up a large message.

BizTalk 2010 R2 CTP: Azure Service Bus Integration–Part 4 Sending Messages to Service Bus Topics

 

Back again with Part 4 in this series.  This time around we are going use BizTalk to send a message to an Azure Service Bus Topic using the new SB-Messaging Adapter.

What is the difference between a Service Bus Queue and Topic?

Conceptually they are very similar in the sense that they both provide a durable message store within Azure where Publishers can publish messages and Consumers can consume messages.  Queues store messages in a First In First Out (FIFO) manner and provide a competing consumer experience.  What this means is that if we have two queue clients that are polling the same queue then only one client will receive a copy of any given message.

Topics provide some additional features that really support a Publish/Subscribe (Pub/Sub) architecture.  Topics allow for multiple clients to subscribe to the same Topic through subscriptions.  Subscriptions also support SQL92 expressions and allow a consumer to filter messages out based upon BrokeredMessage Properties.

Scenario

In Part 3 I  discussed how our Work Order Management system can notify our Major Account System when an Estimated Time of Restore is available.  This allows Major Account Representatives the ability to reach out to customers to share the good/bad news about when their power will be restored.

We are going to build upon this scenario but instead of sending all messages to a Queue we are going to send it to a Azure Service Bus Topic instead.  Due to the , fictional, growth of our company there are now two distinct groups responsible for Major accounts.  One for the city of Edmonton and another for the city of Calgary. (Both of these cities exist within Alberta, Canada)  Each queue Subscription client will now subscribe to events for their city.  BizTalk will however just send messages to one Topic and let the Service Bus work out which message needs to be delivered to each client.

Modifying Client Application(s)

Calgary Application

• Once again we are going to leverage the work that we have done in previous posts(In this case Part 3).  The first thing we will do is rename our previous C# Console project from BrokeredMessageFromBizTalk to BrokeredMessageFromBizTalkCalgary.

• Next we will modify the namespace for the Program.cs file so that it reflects the name of the project (BrokeredMessageFromBizTalkCalgary).  Note we will leave the namespace of our EstimatedTimeToRestore.cs as is.
• Below is the entire code listing for our Calgary Client.  Within this code we will use our connectivity parameters to establish a connection.  Once we have a connection we will see if the Calgary subscription currently exists.  If it does not, we will create it.  We will also create it and add a SQLFilter.  This particular filter is interested in messages where the Address Brokered Message Property equals “Calgary”.

using System.IO;
using System.Runtime.Serialization;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using BrokeredMessageFromBizTalk;

 

namespace BrokeredMessageFromBizTalkCalgary
{
    class Receiver
    {
        const string TopicName = "<your_topic>";
        static string ServiceNamespace = "<your_namespace>";
        static string IssuerName = "<your_owner>";
        static string IssuerKey = "<your_key>";
        static string connectionString = String.Format("Endpoint=sb://{0}.servicebus.windows.net/;SharedSecretIssuer={1};SharedSecretValue={2}",
             ServiceNamespace, IssuerName, IssuerKey); 

        static void Main(string[] args)
        {
           
            TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
                Receiver.IssuerName, Receiver.IssuerKey);
            Uri uri = ServiceBusEnvironment.CreateServiceUri("sb", Receiver.ServiceNamespace, string.Empty);
            MessagingFactory messagingFactory = MessagingFactory.Create(uri, tokenProvider);

 

            // Create a "Calgary" SQLFilter
           SqlFilter calgaryFilter = new SqlFilter("Address = 'Calgary'");

            //Check to see if Calgary Subscription exists, if it does not then create it
            NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(Receiver.connectionString);
            if (!namespaceManager.SubscriptionExists(Receiver.TopicName, "Calgary"))
            {

                //Create Calgary Subscription with our calgaryFilter
                namespaceManager.CreateSubscription(Receiver.TopicName, "Calgary", calgaryFilter);

            }
            //Create Subscription Client using Peek/Lock Mode
            SubscriptionClient sc = messagingFactory.CreateSubscriptionClient(Receiver.TopicName , "Calgary", ReceiveMode.PeekLock);
            BrokeredMessage bm;
            while ((bm = sc.Receive(new TimeSpan(hours: 0, minutes: 0, seconds: 20))) != null)
            {
                var data = bm.GetBody<EstimatedTimeToRestore>(new DataContractSerializer(typeof(EstimatedTimeToRestore)));
                Console.WriteLine(String.Format("An estimated time of restore {0} has been received for {1}", data.RestoreTime, data.CustomerName));
                Console.WriteLine("Brokered Message Property Address has a value of {0}", bm.Properties["Address"]);
                //Remove message from Topic
                bm.Complete();
            }

        }
    }
}

Edmonton Application

• Within our existing Visual Studio solution we are going to add another C# Console Application called BrokeredMessageFromBizTalkEdmonton.

• Within this application we will create a reference to our “Calgary” project.  We need to do this so that we have access to the EstimatedTimeToRestore class. This is accomplished by including the following statement:
  • using BrokeredMessageFromBizTalk;
• Otherwise we can simply copy and past the “Calgary” Program.cs code and adapt it for our Edmonton scenario.The main areas that we need to focus on are creating a unique Edmonton Subscription and ensuring that our Edmonton filter matches Edmonton messages and not Calgary messages.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Runtime.Serialization;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using BrokeredMessageFromBizTalk;

 

namespace BrokeredMessageFromBizTalkEdmonton
{
    class Receiver
    {
        const string TopicName = "<your_topic>";
        static string ServiceNamespace = "<your_namespace>";
        static string IssuerName = "<your_owner>";
        static string IssuerKey = "<your_key>";
        static string connectionString = String.Format("Endpoint=sb://{0}.servicebus.windows.net/;SharedSecretIssuer={1};SharedSecretValue={2}",
             ServiceNamespace, IssuerName, IssuerKey);

        static void Main(string[] args)
        {

            TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
                Receiver.IssuerName, Receiver.IssuerKey);
            Uri uri = ServiceBusEnvironment.CreateServiceUri("sb", Receiver.ServiceNamespace, string.Empty);
            MessagingFactory messagingFactory = MessagingFactory.Create(uri, tokenProvider);

 

            // Create a "Edmonton" filtered subscription
            SqlFilter edmontonFilter = new SqlFilter("Address = 'Edmonton'");

            //Create NamespaceManager object and see if Edmonton Subscription exists for our Topic
            NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(Receiver.connectionString);
            if (!namespaceManager.SubscriptionExists(Receiver.TopicName, "Edmonton"))
            {
                //If the Subscription does not exist, create it
                namespaceManager.CreateSubscription(Receiver.TopicName, "Edmonton", edmontonFilter);

            }

            //Create Subscription client and use Peek/Lock mechanism for delivery
            SubscriptionClient sc = messagingFactory.CreateSubscriptionClient(Receiver.TopicName, "Edmonton", ReceiveMode.PeekLock);
            BrokeredMessage bm;
            while ((bm = sc.Receive(new TimeSpan(hours: 0, minutes: 0, seconds: 20))) != null)
            {
                var data = bm.GetBody<EstimatedTimeToRestore>(new DataContractSerializer(typeof(EstimatedTimeToRestore)));
                Console.WriteLine(String.Format("An estimated time of restore {0} has been received for {1}", data.RestoreTime, data.CustomerName));
                Console.WriteLine("Brokered Message Property Address has a value of {0}", bm.Properties["Address"]);
                //remove message from Topic
                bm.Complete();
            }

        }
    }
}

Creating Topic

Much like we did in the previous blog post where we created a Service Bus Queue, we can also create a Service Bus Topic from the http://windowsazure.com portal

• To create a new Topic simply click on the New Topic button.

• Provide a Topic Name.  For the purpose of this post I am using PowerRestoreTopic.  We can then leave the default settings as is.

Note: We can also create Topics via code by using the NamespaceManager class much like we did for Subscriptions.  I decided to use the portal just to demonstrate that we have a few options when creating Service Bus artifacts.

Modifying BizTalk Solution

Once again we are going to keep the BizTalk Solution changes to a minimum and will continue to use a Messaging Only scenario.

• The first change that we are going to make is to our BrokeredMessagePropertySchema by adding our Address field.  Note if you recall from our C# projects that we created SQLFilters based upon the Address property being equal to either Calgary or Edmonton(depending upon the application).  By adding this property to our PropertySchema we can now promote this value in our XSD.
• Next we will modify the ETRfile.xsd and promote the Address field.  We will then map this field to the property we just created in our PropertySchema.

• We can now deploy our BizTalk Application and bounce any related Host Instances.
• While inside the BizTalk Administration Console, we need to modify our SB-Messaging Send Port called SendEstimatedTimeToRestoreToAzure.  Previously we were using this Send Port to send messages to a Service Bus Queue.  We will now be sending messages to a Topic which uses the same convention within the URI.  Instead of including our Queue Name in the URI we need to specify the Topic Name.  In our scenario I am going to specify PowerRestoreTopic.

• Our Authentication tab remains unchanged. We will continue to use the same Issuer Name and Issuer Key as in our previous example.

• Finally, we need to click on the Properties tab so that we can enable our custom Brokered Message Properties. To enable these properties we need to provide the namespace for our Property Schema that we previously modified.  As you may recall, we added a property called Address that we will use to route messages within our Azure Topic Subscription.

Testing our Solution

• We are going to leverage the Receive Location that we created in Part 3 of this blog series.  Our solution is expecting messages that conform to the ETRfile.xsd schema.  Below, I have included two sample files.  One contains an Address that belongs in Edmonton and the other one has an Address that belongs in Calgary.  We will drop both of these files within the same receive location and they will be sent to the same PowerRestoreTopic.

• Once the messages have been sent we will launch our two client applications (Calgary and Edmonton).  The expectation is that the Calgary application will retrieve the Calgary Message and the Edmonton application will retrieve the Edmonton Message.
• As expected, our Edmonton client retrieved the Edmonton message and the Calgary client retrieved the Calgary message.

Conclusion

In this post we used existing BizTalk skills to send typed messages to an Azure Service Bus Topic.  We promoted a property within BizTalk that was converted to a Brokered Message Property that can be used within Subscription Filters in order to route messages between different consumers.

Once again the experience is very seamless. I really do like the .Net API for Service Bus.  I think it is intuitive and is very logical.  As simple as it is to use this API it is hard to imagine anything being much simpler. But it has been done. The BizTalk team has made interfacing with the Service Bus even easier than using the API.  From a BizTalk perspective the Service Bus is just another endpoint and requires some additional configuration.  I do find that this aligns with some of the early goals of BizTalk.

BizTalk 2010 R2 CTP: Azure Service Bus Integration–Part 2 Brokered Message Properties

In my last post I provided a walkthrough that allows you to send a typed Brokered Message from a Console application to a Service Bus Queue,  have BizTalk retrieve this message and then write it to disk.  I am now going to expand upon that scenario and describe how we can leverage Brokered Message properties within BizTalk to route the message to different locations using BizTalk’s promoted properties.
What is a Brokered Message Property?
In many ways a Brokered Message Property is very similar to a Promoted Property within BizTalk.  These properties can be used to capture meta-data outside the body of the message.  We can then use these properties for routing within the Service Bus when delivering messages to different Service Bus Topics. It is important to note that we don’t have to use these properties for just routing.  We can also use them as part of business logic in downstream systems if we so desire.
Why is this important for BizTalk?
As I mentioned in the previous paragraph we can use Promoted Properties within BizTalk to route messages and we can also use it to capture meta data if we want (although you should look at distinguished fields instead if that is your intent).  In the BizTalk 2010 R2 CTP there is now support for transitioning Brokered Messages Properties from Service Bus Queue clients to BizTalk promoted properties.  BizTalk applications themselves do not understand a Brokered Message property, but BizTalk will convert these Brokered Message Properties into BizTalk Promoted Properties where they can be used to route messages.
Scenario Walkthrough
In my previous blog post I used a Power Outage scenario.  My client application would pass along customer information to a Service Bus Queue and then BizTalk would pick that message up and write it to disk.  In a ‘real life’ scenario I would have routed that message to a Customer Information System (CIS) or a Work Order Management (WOM) system so that a field operations team could address the power outage.  In this walkthrough I am going to build upon that scenario.  This difference this time around is that I am going to introduce a Brokered Message Property called isCriticalCustomer.  I hate to publicly admit it but not all customers are treated equally when it comes to delivering power.  An example of a Critical Customer may be a hospital.  It is more important for a Power company to get their power on before yours.  A patient’s respirator is more important that someone watching the latest American Idol episode. 
Within my Console application this isCriticalCustomer property will be set as a Brokered Message Property.  When this message is retrieved by BizTalk this property will be converted into a Promoted Property and BizTalk will then use that Promoted Property to route the message to a different destination.
Note: A person with a lot of Service Bus experience may say why don’t you just use Topics?  I could have a Topic for regular customers and a Topic for Critical Customers.  This is also a valid pattern but for the purposes of demonstrating BizTalk capabilities I will leave the routing to BizTalk.
Modifying Queue Client
I am not going to display all of the code required for this client to work.  I am going to be adopting the code I listed in my previous post.  So please refer back to that post for the starting point.  I will include any areas within this post where I have made changes.
In the code below I am going to create and send two messages.  In red you will discover that I am setting a Brokered Message Property called isCriticalCustomer. In the first message I am indicating that this is not a critical customer (aka a regular customer).  In the second message I am saying that it will be a Critical Customer.  Once we get to the BizTalk section you will see how we can use this property to route the message within BizTalk.
              //Create new instance of PowerOut object
              //This customer will not be a Critical Customer
              PowerOut po = new PowerOut();
              po.CustomerName = "Stephen Harper";
              po.PhoneNumber = "613-123-4567";
              po.Address = "24 Sussex Drive";
              BrokeredMessage message = new BrokeredMessage(po, new DataContractSerializer(typeof(PowerOut)));
              message.Properties.Add("isCriticalCustomer", false);
              myQueueClient.Send(message);
              //Create new instance of PowerOut object
              //This customer will  be a Critical Customer
              po = new PowerOut();
              po.CustomerName = "Calgary General Hospital";
              po.PhoneNumber = "403-123-4567";
              po.Address = "1 Red Mile Drive";
              message = new BrokeredMessage(po, new DataContractSerializer(typeof(PowerOut)));
              message.Properties.Add("isCriticalCustomer", true);
              myQueueClient.Send(message);
BizTalk Modifications
You may recall from my previous post that my BizTalk solution was very simple as I only had a schema that represented this Customer message being sent from my client application.  So in order to support our new scenario I only need to add one more artifact to my solution and that is a Property Schema.  The reason why I need to add this schema is that I need an artifact within my BizTalk application to “hold” these values as they are being populated when BizTalk receives the message.  This is no different than when you want take a value from one of your “regular” BizTalk schemas and turn it into a Promoted Property.
Within our BizTalk solution we need to do the following:

• Add a PropertySchema to our project.  Once it has been added there will be a default property that we will rename to isCriticalCustomer and change the data type to be a boolean.

• We now need to re-deploy our application.
• Open up the ReceiveBMfromServiceBus Receive Location, click the Configure button.  Now click on the Properties tab.  Within this tab we are going to specify our namespace for our PropertySchema.  If you are unsure where you can get this namespace from, look in the image above and notice the value of the targetNamespace matches the value that I have put in this text box.  We also need to ensure that the Promote Brokered Message Properties checkbox is checked on.

• Next we are going to remove our previous Send Port and create two new Send Ports.  One send port will be for Regular Customers and the other will be created for Critical Customers. 
• Below is the Send Port for regular customers.  Notice that a separate sub-folder called RegularCustomers has been created for these files.

• Click on the Filters label and then add a new Property.  You will notice that within the dropdown list you will find the property that we created in our PropertySchema called isCriticalCustomer.  We need to select this value and then set the value to false.

Note: When you pull down the Property drop down you will also discover the Out of the Box Service Bus Brokered Message properties.  These properties are out of the scope of this post but it is something that may be beneficial down the road. 

• We now want to perform similar actions to our other send Port that will be use to send our CriticalCustomer messages.

• Once again we are going to click on the Filters label.  We will use the isCriticalCustomer property again but this time we will set the Value to true.

• We can now bounce any affected Host Instance(s) and start our application.

Testing our Application
As you my recall, we modified our Console application so that it will send two messages to the same PowerOutage queue.  In the first message, we set the isCriticalCustomer Brokered Message property to false.  In the second message, for the hospital, we set it to true.  The end result is that we should receive one message in our Regular Customers folder and one in our Critical Customers folder.

• As promised when I run the application I will find one message in each corresponding folder:

• If I open the files I will discover that the right message was delivered to the correct folder:

Conclusion
Overall it is a pretty slick, and seamless, experience.  I think the BizTalk product team has done a great job in bridging the Service Bus Brokered Messaging Property with BizTalk’s Promoted Property.  In my opinion, the Azure Service Bus and BizTalk Server really complement each other by providing robust Hybrid solutions.  It is great to see smooth interoperability between these two technology sets.
This concludes this blog post.  I am not done yet with this series as I have still just scratched the surface.  I plan on writing more about my experience with Sending messages to Service Bus Queues/Topics from BizTalk and perhaps dive into some different messaging patterns.