Service Bus Notification Hubs–Part 3 My Solution Overview

In my previous post I introduced the concept of Customer Power Outages and, at a high level, introduced  how Outage Management Systems work.  The key take away from that blog post is that important, event driven data is passing through BizTalk and there is an opportunity use this information to provide better Customer and Employee engagement through Service Bus Notification Hubs.  Think about, conceptually how different is it from BAM Alerts?  The idea behind BAM Alerts is data is moving through BizTalk, Tracking Profiles pick it up and send it to subscribers who are interested in it via email (oversimplified…I know).  The process of collecting this information for Notification Hubs is different but now we can reach an audience that we may never had access to before.

In this post we will take a closer look into how we could actually implement a Power Outage system and include notifications to both customers and employees by using Service Bus Notification Hubs. 

Note: In some areas I have over-simplified the process in order to focus on some of the key technical aspects of Notification Hubs.

In the diagram below I have laid out the series of events that make up the architecture:

1. Customer determines their power is out and launches the Windows 8 application on their SurfaceRT tablet in order to report their Power Outage.  The mobile client registers itself for Toast Notifications.  In this case the Customer is only interested in events pertaining to its SiteID (Customer ID) so a Tag of 0090123456789 is included as part of the Toast Notification registration. Next, the customer clicks the submit button to notify the power company of their outage. The message is sent to a Customer Outage Service Bus Queue.
2. BizTalk is using the new SB-Messaging Adapter to connect to this Service Bus Queue and pulls down the Customer message.
3. BizTalk will perform a transformation and send the message to the Work Order Create Queue.  Once again, the new SB-Messaging Adapter will be used when communicating with the Service Bus.
4. Once the Work Order message has been sent to the Service Bus Work Order Create Queue, BizTalk will send a message using the Service Bus .Net Preview SDK to the Service Bus Notification Hub.  Included in this message is the tag “Airdrie” which happens to be the City were the customer, who submitted the trouble ticket, lives.  It also happens to be the area that the Power Line Technician (PLT)  is responsible for.  Only he will receive this toast notification because he is the only employee that is registered for this tag. 
5. The PLT will now receive a Toast Notification indicating that he has a new Work Order that he needs to complete.
6. When the PLT clicks on the Toast Notification, the PLT App is launched and he can click the Retrieve Next Order button to download the order. 
7. Once the PLT has had a chance to assess the situation he can provide an Estimated Time of Restore (ETR) and send this information to a Work Order Update Queue that exists in the Service Bus.
8. BizTalk will pickup this message from the Work Order Update Queue.
9. The updated Work Order information will now be sent to the Outage Management System using the FILE Adapter.
10. BizTalk will use the information contained in the Work Order Update message to push a Notification message up to the Service Bus Notification Hub.  As part of this message, a tag for the customer’s Site ID is populated.  In this case it is 0090123456789 which happens to be the same Site ID as the customer who initially logged the Power Outage ticket.
11. The customer will now receive a Toast Notification indicating their Estimated Time of Restore (ETR).
12. Steps 7 – 11 will be repeated once the PLT has restored power and a notification can be sent to the customer, when the work order has been closed in the Outage Management System, indicating that their power has been restored and give them a duration of the outage.

Conclusion
Hopefully this post has described in more detail how Service Bus Notifications can improve customer and employee engagement.  I promise that the next post in the series will have some code.  I just felt that if I could build a story, it would provide some worthwhile context that truly demonstrates why this Notification Hub technology is important.

Service Bus Notification Hubs–Part 2 Improving Customer Engagement

 

In my previous post I introduced Service Bus Notifications and explained why you may want to use it when sending Notifications to mobile devices.  In this post I want to explore its use in a corporate environment and how introducing this technology can improve customer engagement as well as employee engagement.

Use Case

Back in the Summer of 2012, I wrote a couple blog posts on BizTalk and SignalR.  The idea was that BizTalk could provide real time notifications to on premise users.  The notifications provided the current  state of particular events that were flowing through BizTalk. In this post there is a similar theme but this time we will extend the experience onto customers and mobile field workers.

The scenario that we are going dive into is a typical Customer Power Outage scenario.  We have all been there, we get home from work (or wherever) and discover that our power is out.  If you have never worked in the utility sector, you probably have no idea what goes on behind the scenes.  I have worked in the sector for close to 7 years and I can assure you that these scenarios are taken seriously.  At least the places that I have worked have treated them that way.

Each utility will vary greatly when it comes to dealing with Power Outage scenarios.  Some utilities will have very sophisticated SCADA and Outage Management Systems (OMS) others may simply rely upon customers calling their utility to notify them that their power is out.  Sometimes the power outage may be widespread other times it may be only you.  Much like any other company, or industry, the utility companies are looking for ways to provide Self Service capabilities to customers.  Just like you can check your bank account online, why can’t you see your power consumption or inform your power company that your power is off via a mobile application?

In the diagram below, you will see many different key data flows:

• Notifying the OMS system of devices that have failed via SCADA interfaces
• Customers calling and talking to a Contact Centre agent.  The Contact Centre agent in turn will leverage a Customer Information System (CIS) to log a ticket.
• Customers who use an Integrated Voice Recognition (IVR) service when live agents are currently servicing other customers
• Self Service inputs such as mobile applications and  Websites

If an organization is using a Middleware product, such as BizTalk to bridge all of these different inputs with the Outage Management System then BizTalk has access to a lot of relevant, event driven information.

Another interesting fact is when orders are created and dispatched to Field workers aka Power Line Technicians (PLTS) information is often passed back to an ERP system.  This information is often used for Financial or Reporting purposes.  For instance if there is a power outage, then resources will be used to address the power outage.  This will include Employee time and materials such as replacement parts and equipment used such as vehicles to fix the problem. This information is usually passed back once the outage has been completed or maybe as the outage is being resolved in the form of updates. Since ERP systems are usually not part of Outage Management Systems, this information needs to be moved via  Middleware platform such as BizTalk.

Hopefully it has become apparent that BizTalk has access to a lot of important event driven information that is passing through it that may be of interest to other parties such as customers.  For instance if your power was out, would you like to know when the Estimated Time to Restore (ETR) is?  Or if you were the owner of a small business, who relied upon power to run its business, would you like to be notified the minute the outage is over? I am pretty sure you would.  The interesting part is that any decent Outage Management System will have this information.  The problem is that utilities just do not have the people resources to start calling people to let them know. Here is where the opportunity lies to use Service Bus Notification Hubs.

The Solution

Enter Service Bus Notification Hubs.  I think this is a great complimentary technology to BizTalk.  In this case BizTalk is performing traditional On-Premise integration and can easily hook into a progressive service like Service Bus Notification Hubs to deliver better customer engagement with very little effort.

I have built out a fairly comprehensive solution that dives deeper into this problem.  I will be breaking down this problem into a series of different posts.  But, as a bit of a teaser you can see part of the solution in the screen shot below.

 

Service Bus Notification Hubs – Part 1

 

This past week the Service Bus team at Microsoft rolled out a preview of their latest offering called Service Bus Notification Hubs.  For those of those of you who keep an eye open for all things Middleware from Microsoft, you may recognize the code name of the project: Iguazu.  You may  have even seen Iguazu presented by Clemens Vasters at one of the recent Microsoft conferences/summits.

What are Service Bus Notification Hubs?

So what are Service Bus Notification Hubs?  Probably the simplest way to explain it is an internet-scale Push Notification Service.  Push Notifications are nothing new.  The major mobile platforms all have them and you have inevitably used them if you use social media apps like Facebook, Twitter or even games.  These notifications are often called “Toast Notifications” in the sense that they pop up and inform you of some relevant event such as someone commenting on a post of yours or informing you that you have been mentioned on Twitter.

If they have been around for a while, why do I care about them now?

Just because Notification Services from Apple, Google and Microsoft have been around for a long time, doesn’t mean that it has been easy to use them.  Often times developers need to manage all of the subscriptions and channels within their own data stores. Developers were also required to deal with the nuances of the  different platforms and managing which subscriptions were running on a particular platform.  This results in a lot of fragmentation.  Finally, dealing with large scale has also been challenging for some organizations.  Imagine if you were a large media outlet with hundreds of thousands or millions of users.  Providing timely notifications to users of this magnitude becomes extremely important.  For example receiving a notification for a touchdown that happened in the 2nd Quarter when the game is already over is not a good user experience. Service Bus Notification Hubs can take care of this by delivering to a wide audience immediately.

Benefits of using Service Bus Notification Hubs

While I am relatively new to this technical domain(mobile notifications), I found building solutions off of Service Bus Notification Hubs was pretty straight forward.  I find it very compelling that I can choose to use Native Notifications that are specific to a respective platform or I can just use one of the many generic templates that ship and can address devices on multiple platforms.  Since the service is currently in “preview mode”, only Windows 8 and IOS notifications are supported but Microsoft is targeting Android and Windows Phone when the service becomes Generally Available.  Another key benefit, and perhaps this is why it is in the Service Bus family, is the service is true Pub-Sub.  I don’t have to worry about specific subscriptions.  I just have to throw the message at the Service Bus and let it figure out who is interested in it.  It could be 1 person/device or it could be 100 000.  It doesn’t matter to my Server code.  This features alone makes the service worth it.

What’s next?

There is quite a bit of content that is already available for this topic and I recommend checking out the following links:

• Service Bus Notification Announcement by Scott Guthrie
• Service Bus Notification Hubs – Code Walkthrough (Windows 8 Edition)
• Service Bus Notification Hubs – Code Walkthrough (IOS)
• Service Bus Notification Hubs – Elio Damaggio

Also stay tuned to this blog where I will be walking through a corporate scenario from the Energy sector where I will combine Service Bus Notification Hubs, BizTalk and Windows 8 Store Apps.

MSDTC Adventures: 'System.EnterpriseServices.TransactionProxyException'

I was configuring a new Development environment that consisted of a BizTalk Application Server and a Remote SQL Server.  The BizTalk Installation went smoothly but the Configuration did not.  I tend to configure SSO service on its own as it can be problematic. There is no point in configuring the remaining databases until a person knows that the SSO database is created successfully.  Otherwise, there will be a lot of wasted key strokes.

In this particular case I was able to configure the SSODB but as soon as I would try to configure the Group and Runtime, the System.EnterpriseServices.Transaction Proxy exception started to surface.

If you perform a Google/Bing search for 'System.EnterpriseServices.TransactionProxyException' you will probably get quite a few results.  Most of the results will point you in the direction of a MSDTC not being enabled or configured correctly.

In my case MSDTC was configured correctly based upon Microsoft’s recommendations. I was a little puzzled at this point but in some of the support forums people were discussing the DTC Tester tool.  Having been through a lot of frustration and a few failed attempts at trying to configure the Group and Runtime features I figured I would give it a try.  I am glad I did.

The purpose of DTC Tester is to enlist a remote transaction against the Temp Database which is hosted in SQL Server.  If you are able to successfully enlist and commit this transaction then there is a very high probability that your BizTalk configuration (as it relates to MSDTC) will be successful.

The tool is available here.  Download it and run the self extracting executable.  At this point we will discover a ‘dtctester’ executable and an End User License Agreement.

The dtctester executable needs to be run from a command prompt but requires an ODBC connection is configured before we can run the tool.  The ODBC connection should be pointed at the Database Instance where we want our BizTalk Databases to be created.

ODBC connections can be created via the Control Panel and a System DSN will work.

From a command prompt we can now run the tool.  When I ran it for the first time I encountered errors (highlighted in red)

 

C:\Downloads>dtctester <ODBCConnectionName> <UserName> <Password>
Executed: dtctester
DSN:  <ODBCConnectionName>
User Name: <UserName>
Password: <Password>
tablename= #dtc7063
Creating Temp Table for Testing: #dtc7063
Warning: No Columns in Result Set From Executing: 'create table #dtc7063 (ival int)'
Initializing DTC
Beginning DTC Transaction
Enlisting Connection in Transaction
Error:
SQLSTATE=25S12,Native error=-2147168242,msg='[Microsoft][ODBC SQL Server Driver]
Distributed transaction error'
Error:
SQLSTATE=24000,Native error=0,msg=[Microsoft][ODBC SQL Server Driver]Invalid cur
sor state
Typical Errors in DTC Output When
a.  Firewall Has Ports Closed
-OR-
b.  Bad WINS/DNS entries
-OR-
c.  Misconfigured network
-OR-
d.  Misconfigured SQL Server machine that has multiple netcards.
Aborting DTC Transaction
Releasing DTC Interface Pointers
Successfully Released pTransaction Pointer.

In my situation the issue ended up being the Windows Firewall on the BizTalk Server was not allowing Outbound DTC connections which was blocking MSDTC from participating in Transactions with SQL Server.  You can modify the MSDTC Firewall rules in the Control Panel.

After allowing MSDTC to communicate through the firewall I re-ran the dtctester tool and had success.

C:\Downloads>dtctester <ODBCConnectionName> <UserName> <Password>
Executed: dtctester
DSN:  <ODBCConnectionName>
User Name: <UserName>
Password: <Password>
tablename= #dtc9418
Creating Temp Table for Testing: #dtc9418
Warning: No Columns in Result Set From Executing: 'create table #dtc9418 (ival int)'
Initializing DTC
Beginning DTC Transaction
Enlisting Connection in Transaction
Executing SQL Statement in DTC Transaction
Inserting into Temp...insert into #dtc9418 values (1)
Warning: No Columns in Result Set From Executing: 'insert into #dtc9418 values (
1) '
Verifying Insert into Temp...select * from #dtc9418 (should be 1): 1
Press enter to commit transaction.

Commiting DTC Transaction
Releasing DTC Interface Pointers
Successfully Released pTransaction Pointer.
Disconnecting from Database and Cleaning up Handles

After completing this successful test with the dtctester tool I could successfully configure my BizTalk Group and Runtime.

WCF-SAP User account not in validity date

Recently, I have been working with a new organization and have been been building some interfaces that communicate with SAP. Over the Christmas break I continued to work on some of these interfaces.  On December 31 my interfaces were working just fine.  But, on January 1st, I made a few changes and then ran my BizUnit regression tests just to make sure everything was ok.  To my surprise, my tests were failing and the issue was related to the following error.

It just seemed like too much of a co-incidence that the errors started occurring on the first day of the new year. My gut told me that my account must have expired. At my previous organization the SAP-BizTalk system accounts never expired but they do at this one(which is probably not a bad thing).  The resolution in this case was for the SAP Basis team to update the validity date for the account.  Once this attribute was updated I could re-connect to SAP through my interfaces.  I have no idea why the SAP error message doesn’t just say “your account has expired”.