Part 1: BizTalk + SignalR

For those unfamiliar with BizTalk, it is Microsoft’s premiere Enterprise Application Integration (EAI) platform.  It is used to integrate disparate systems over a variety of protocols using a durable pub-sub mechanism.

SignalR does have some similarities to BizTalk in that it is a messaging system that also supports the notion of pub-sub.  However, SignalR’s sweet spot is really lightweight messaging across Web clients.  SignalR itself is a scalable, asynchronous .Net library authored by David Fowler and Damian Edwards.  If you are new to SignalR, I recommend checking out this post by Scott Hanselman who describes many of the underlying technical details that I will not be going into in this post.

Why is SignalR important?

One of the true benefits of SignalR is it is Asynchronous by nature.  I don’t profess to be an expert web developer.  I have done some in an earlier life prior to my BizTalk days but I know enough to understand that locking up a user’s browser during a request-response interaction can be a really bad thing.  Yes, technologies have been introduced like AJAX and JQuery to provide a more asynchronous experience and they both have their strengths and weaknesses. But, overall they take many steps to solve this Request-Response locking problem.  The question remains, what happens when you have events occurring in other systems that you want raised within your current system that you are interacting with?  This is where I feel the true “magic” of SignalR comes into place.

Scenario

I work in the Electricity/Power industry and we are implementing an Outage Management System (OMS).  OMS systems are used to calculate or anticipate the particular device(s) that are the underlying problem that is causing a Power Outage.  OMS systems may have many different types of input including Customer Calls, IVR messages or even SCADA events.  In this case we are only going to focus on Customer Calls.

This OMS system is a commercial off the shelf (COTS) product that we have purchased from a vendor.  This product has defined, XML based, asynchronous interfaces that require the use of  WebSphere MQ queues.  Using BizTalk to integrate with the OMS system makes a lot of sense and plays well to BizTalk’s strengths that include:

• Support for MQ Series
• Durable Messaging
• Tracking
• Configuration
• Correlation (Async messaging)
• XML Schemas
• etc..

But the question remains, we need to capture information that is coming from our Customer’s calls in our Call Centre.  One option that we are currently exploring is a light weight Web Based application that will allow our Call Centre to quickly capture customer’s outage information and then pass this information to BizTalk and have BizTalk deal with calling the OMS’s semi-complex interfaces.

Much earlier in my career I may have been tempted to do the following:

• Expose a WCF/Web Service that a Web Application can consume
• Accept the request from the Web App and then proceed to call the asynchronous interfaces that exist in the OMS system.
• In the meantime, the Web Application that called the BizTalk Service is being blocked as BizTalk is still processing messages Asynchronously.
• Once BizTalk is done interacting with the OMS system, BizTalk will provide a response back to the Calling Web Application.

The reality is that his is a bad pattern.  You don’t want to lock up your users in a Web Application if you don’t have to especially when you have asynchronous messaging happening in the backend.

An alternative approach, that I like much better, is outlined below:

• Expose a WCF/Web Service that a Web Application can consume.
• Once BizTalk has received the Web Request from the Web Application, simply provide an acknowledgement that the message has been received and will be processed.
• At this point the Web Browser has been posted back.  If our Web Application is built around technologies like JQuery and/or AJAX our users can continue to perform some work.
• In the meantime as BizTalk is calling the OMS' related interfaces, BizTalk can provide status updates back to the Web Application using SignalR.  There is actual information that the OMS system will pass back that our end users are interested in.  More specifically, it will include information as to when the Customer can expect their power to be restored (Estimated Time of Restore).  If you have ever experienced a power outage, I am sure you would like to know if it is going to last 30 minutes or 10 hours.

The benefits to this approach include:

• User’s browser is not locked up
• Users are continuing to be updated as to the status of their request
• No need to continue to refresh your page (Just say NO to F5) in order to get a status update.

Conclusion

I am sure at the beginning of this post you were thinking what could BizTalk and SignalR possible have in common?  I hope that I have provided a good example of how these two technologies complement themselves.

In Part 2 of this series, I will actually implement this pattern that I have shown above.  I have split this information into two parts due to the total length of the content.  Stay tuned!

Exposing common service(s) to SAP and WCF clients

I have a scenario I am dealing with at work that involves exposing some common data to two different systems: SAP and a Custom ASP.Net Web App.  Both of these applications will request BizTalk to fetch some data from a variety of database views, aggregate it and package it up nicely for these calling systems.  Both Systems will be requesting this information on demand – i.e. Synchronously.  SAP will be calling an RFC hosted in BizTalk,via the SAP Adapter, using the method that I identified in a previous post.  The Custom Web Application will be consuming a WCF Service hosted in IIS.

Conceptually, my solution looks like this:

Whenever you have multiple systems trying to consume the same data, you generally  try to utilize a Canonical schema approach.  Canonical schemas allow you to take the different external data formats and transform them into a common internal format before handing them off for processing like in an Orchestration. 

You then perform all of your processing using this internal format to reduce the amount of code/configuration that you require to solve the problem.  Additionally, when you need to make a change, you do so in one place as opposed to two(or multiple) locations.

In order to keep things simple for this POC, I decided to reuse my RFC Add solution where you can have a client pass two numbers to BizTalk, BizTalk will then sum them and provide the answer back to the calling application.

For the Web Client, I will simply expose custom “Web” Schemas as WCF Services using the BizTalk wizard provided within Visual Studio.  Note that I did not want to expose my SAP schemas to my Web Application.  I could have done that but it is not a good practice as any changes to the SAP schemas would impose a change on my Web Application whether it was required or not.  Also, SAP schemas tend to be complex and we don’t want to unnecessarily propagate that complexities onto other applications if we don’t have to.

Initially I thought my solution would be pretty straight forward:

• Generate my SAP Schemas
• Create my Schemas that will be used for the Web Application and expose them via Wizard
• Create my Canonical Schemas
• Create related maps

I then created my logical port and set the Request and Response message types to my Canonical schemas. I deployed my application and configured my Physical Port within the BizTalk Admin Console.  I decided that I was going to re-use the port that was created as part of the BizTalk WCF Publishing Wizard.  I would simply add a Receive Location for SAP and set the appropriate inbound and outbound port mappings. 

Using inbound port mapping is very simple, I can specify multiple maps and BizTalk will detect which Map to use based upon the message type that is being passed.   So if we receive a request from SAP, BizTalk will detect this and use the SAP_to_Canonical.btm map.

 

It then hit me…how will BizTalk determine which Map to use on the Outbound (Response) message? The message being passed to the port will always be the same as it will be my canonical format.  I soon found out.  As you can see in the screenshot below, my SAP response was sent down to my Web Client(which in this case was the WCFTest tool).  Not the desired result that I was looking for.

While chatting with a colleague he mentioned why don’t you try a Direct Bound port.  I have used Direct Bound ports in the past but only in asynchronous scenarios.

So to fix this, I changed:

• My logical Request-Response port to be a Direct bound port and to be Self Correlating.

• Created an additional Receive Port.  I now have a Receive Port for my Web App and for SAP.

• Made the appropriate Inbound and Outbound Port Mappings.  Now each port only has 1 Inbound and 1 Outbound port mapping.

 

• My orchestration will no longer have a Physical Port to bind to since it will be Direct Bound to the MessageBox

• Now when I execute my test from the WCF Test Client, I get the correct result in the WebAddResponse message type that I am expecting

• I am also getting the correct response from SAP

Conclusion

The magic in this solution is really the Request-Response direct bound port.  The idea is that our Orchestration will place a subscription on our Canonical Request message.  It doesn’t really matter how that message gets into the MessageBox as long as it is there.  In this case we have exposed two end points, one for SAP and one for our Web App.  In both scenarios they will take their specific Request message and transform it into our Canonical message and therefore our Orchestration will pick it up.

Request-Response ports always use a form of Correlation so that it can provide the correct Response back to the calling client.  We can take advantage of this mechanism to ensure we get the correct Canonical Response message which in turn can use Outbound Port mapping and send our response in the correct format to the calling application.

Microsoft TechEd North America 2012-Day 4

So this post is a little delayed due to all of the excitement around the BizTalk sessions.  However the sessions were that good that I wanted to still publish the post.

Azure Service Bus Solution Patterns  -Clemens Vasters and Abhishek Lal

Another session by Clemens and Abhishek.  This time around it was a very practical session based upon some Customer Use Cases and how to implement some popular integration design patterns based upon the “Integration Bible” - Enterprise Integration Patterns.  To view the actual session on Channel9, click here.

Some of the Use Cases included:

• Web Services For Retailers
  • Company from Italy
  • Provide SAAS solution for Retail Stores
  • Seed local retail outlets with Catalogue and Pricing information
  • Push out to retail stores
    • Use Topics to distribute information to each retail store
• SaaS with Dynamic Compute Workload
  • High Performance Computing (HPC) scenario
  • Command and Control messages sent in from Service Bus
  • ISV specialized dynamic compute capacity provider
• Consumer Web Site
  • Web site that searches for data about people – credit check, criminal check etc.
  • Their challenge was back end data co-ordination
  • Different profiles for users who have different access to to back end services
  • Queues for decoupling the web layer from middle-tier services

 

Scaling things out

Next Clemens walked us through a scenario that Microsoft has been working on with a particular customer.  The solution was related to remote controlling air conditioners.  The idea is that a consumer would have the ability to manually control it but also power providers could *potentially* control it to prevent rolling brown-outs from occurring.  Instead of instituting  wide spread rolling brown-outs, each customer could alter their consumption. Collectively these savings add up and prevent demand from exceeding supply.  I am a little skeptical about a power company(I work for one) controlling someone’s air conditioner but in theory it makes a lot of sense.

The requirements for this solution includes:

• Pair devices, such as air conditioner, to local Wi-Fi connection
• Users need the ability control the device
• Control requests could be made from back yard or across the world
• Service Bus makes these control requests possible from anywhere that has an internet connection.
• Devices will then send consumption data to Azure where the data can be viewed on a mobile device. This data will make its way to Azure via Service Bus.  The premise behind this is if customers are more aware of their consumption patterns, then they may try to alter them.  This is something that my organization has also been investigating.

So a question remains, these types of consumer devices will not have the .Net Service Bus bindings installed so how will they actually communicate?  The answer is really HTTP.  You can send HTTP requests to the Service Bus and in this case Clemens introduced a concept that he likes to call “N-HTTP”.  It is a similar to the “NoSQL” movement but in this case is related to HTTP.  HTTP in many cases includes HTTP Headers but also an entity body.  The entity body could include JSON content, XML content etc.  The challenge with entity bodies is that you need a parser to package the information up in requests or un-package it when receiving responses.  This would further complicate things as these parsers would need to be loaded into these consumer devices.  What’s interesting is HTTP Headers is they are well understood, across devices, systems, technology stacks etc,  and do not require parsers.  So if you can get away with sending key/value pairs when sending or receiving messages then this solution should work for you.

Receiving messages from Service Bus generally includes using ‘long polling’ when waiting for messages.  Using  long polling sockets isn’t a great use of power resources for devices that do not have permanent power sources (devices that rely on batteries).  With this in mind, Microsoft has been working with other industry leaders on the AMQP (Advanced Message Queuing Protocol).  AMQP is a popular queuing technology that is used in financial brokerage settings.  Another benefit of using AMQP is that it has a quieter socket profile which results in lower battery consumption.  So this is an area that Microsoft is investing in that will have wide spread benefits….Cool Stuff!!!

 

Message Channel Patterns

Abhishek was back on point and walked us through some popular messaging patterns including:

• Pub-Sub
  • Accomplished via Topics
• Content Based Router
  • Using Topics based upon a Subscription Rule
• Recipient List
  • Sender wants to send the message to a list of recipients
  • Common use-cases
    • Order processing systems – route to specific vendors/departments
    • “Address LIKE ‘%First%’
• Message Routing
  • Session re-sequencer – receiving messages out of order and then using the defer method to postpone processing the next message until you receive the next message that is “in order”

I must admit, when I learn more about the Service Bus I do get a little giddy.  I just see it as such an enabling technology.  It facilities building applications that just wouldn’t be possible or cost prohibitive in years gone by.  Whether it is submitting SAP  timesheets remotely or reporting customer power outages it is an amazing technology the opportunities are endless when it comes to bridging data center boundaries.

 

Mobile + Cloud: Building Mobile Applications with Windows Azure – Wade Wagner

Wade Wagner, a former Microsoft Azure Evangelist, put together a pretty interesting session related to Windows Phone and Azure.  To watch this session on Channel 9 click here.  In the past I have followed some of the work he did with the Mobile toolkits for the different mobile platforms, but just haven’t had the time to take a closer look.

This session focused primarily on Windows Phone 7 and how it interacts with some of the Azure services (Storage, SQL Azure, Tables, ACS).  Personally, I think these technologies complement each other very well.  Especially in the area of bridging mobile devices with on-premise LOB solutions and leveraging the Access Control Service (ACS) for authentication.

Three reasons for Device + Cloud

• Allows for new application scenarios
• The cloud levels the playing field
• The cloud provides a way to reach across device platforms and a larger pool of resources from which to pull

Why Azure?

• PaaS you build it, Windows Azure runs it
• Automatic O/S patching
• Elasticity and Scale
• Utility Billing
• Higher-level services
• ACS, Caching, CDN (cache static content), Traffic Manager (route traffic across Azure datacenters based on locale)

Wade then demonstrated a scenario really lends itself well to this technology.  A mobile application that will take advantage of Social Identity providers(Windows Live, Google, Yahoo) for authentication via the Access Control Service.  Wade demonstrated that this isn’t as complicated as it sounds.  With the help of a Nuget package and adding a STS reference we can get this working in the matter of a minutes.  Wade then added some additional functionality to consume a ASP.Net Web API.  Most presenters would have left their demo there.  Giving people the information to build the services but then leaving out some “real world” gaps around security.  Wade did take his demo one step further and then showed we can use the ACS service to authorize user requests as well.  Before the ASP.Net WebAPI method is called, we can intercept this request and validate that the token that has been included as part of the HTTP Request is a valid ACS token.  Provide the token is valid, the appropriate data will be returned.

Wade then wrapped up his session demonstrating how we can use the Azure Push Notification service to serve up “toast notifications”.  Another set of useful information that I hope to play with soon. 

If you are into mobile apps, you definitely owe it to you to watch this session so you can learn about all of the Azure services that you and your customers can benefit from.

Building Integration Solutions Using Microsoft BizTalk On-Premises and on Windows Azure - Javed Sikander and Rajesh Ramamirtham

Update:  This session has now been posted to Channel 9 and you can view the video here.  Feel free to post any comments at the bottom of this post.

 

This was a follow up session to the Application Integration Futures – The Road Map and what's next on Windows Azure  session that was discussed here.  The primary focus of this session was to demonstrate some of the new capabilities of BizTalk On-Premises, BizTalk IaaS and BizTalk PaaS. 

During the presentation there were many questions as to what the differences between the On-Premises version and the IaaS version would exist.  After many questions about a particular feature (BAM, ESB Portal etc) Bala  stepped in and declared that all features that exist in the On-Premises version will exist in the IaaS version.  After a further discussion after the session, it looks like there is a little more work to do in the area of clustered host instances but otherwise we can expect symmetry between these two versions.

Since BizTalk Next (aka “R2”) will be released as part of the latest Microsoft platform offering (Windows Server, SQL Server, Visual Studio), all BizTalk projects will target the .Net 4.5 platform.

The primary purpose of this session was to demonstrate some of these new features lets get into some of the scenarios/demos that were discussed.

BizTalk Consuming REST services

In the first example, the team demonstrated BizTalk consuming a REST feed from the Azure Data Market.  Specifically, the feed was related to flight delays.  BizTalk connected using the new WCF-WebHttpBinding and performed a GET operation against this particular feed.  Since the foundation for authentication when communicating with Azure is the Access Control Service (ACS), Rajesh demonstrated the out of box ACS authentication configuration.

BizTalk consuming SalesForce.com over REST API

Once again BizTalk was configured to consume a REST service.  In this case it was a SalesForce customer feed.  Within the Send Port, the “SOAP Action Header” was populated and once again included the GET operation.  A custom transport behavior was used to provide the appropriate credentials. When executed, a list of customers was returned from SalesForce.

Next, the URI in the SOAP Action header was modified and a hard coded id was provided for a particular customer.  In this case only this particular customer was returned.  Both myself and Bill Chestnut were thinking “great, but how do we inject a dynamic customer id to this GET request”?  Once again the BizTalk team had an answer for this and it came in the form of a new Variable Mapping button.  When clicked an interface that will allow us to specify the name of a context/promoted property.  Bottom line is that we can drive this dynamic value from message payload or context.

Finally, the last SalesForce demo included a POST, where they were able how to demonstrate how to update a customer record in SalesForce.com. 

 

BizTalk PaaS: Azure EAI Services

The team then switched gears and started talking about BizTalk PaaS: Azure EAI Services.  I have no idea as to whether this will be the official name.  This is what the title of their slide included so I am using it here.  I do like it.  I do like that BizTalk is still associated with this type of functionality.  I must caution that the product team did indicate not to look too much into naming/branding at this point.

Some of the functionality(some new, some old) that we can expect in the PaaS solution includes:

• Sequence of activities to perform impedance mismatch
• Flat file disassembly
• Message validation
• Transforms
• Content based routing
  • XPath, FTP properties, Lookup (against SQL Azure), Http properties, SOAP
• Hosting custom code
• Scripting functoid to host .Net Code
• XSLT support
• New Service Bus Connect Wizard
• BizTalk connectivity to Azure Artifacts (Service Bus Queues, Topics, XML bridges)

EDI Portal

• Metro UI for managing trading partners
• Manage and monitor AS2, X12 agreements
• View resources like Transforms, Schemas, Certificates

EDI Bridge

• Archiving
• Batching
• Tracking

Other

• IaaS will be a public TAP
• Other BizTalk releases(On-Premises/PaaS) will be “regular” TAP
• On a lighter side, I did ask if we can expect a Metro version of the BizTalk Admin Console.  Don’t expect it any time soon .  Basically any new UIs that need to be created will follow the Metro styling but other than that don’t expect many updates to previous tools.

Conclusion

This was a great session that included many demos and really proved that what the Product team was speaking to in the previous session wasn’t just lip service.  Having been at the MVP Summit, I must say I was pleasantly surprised at the amount of functionality that they have been working on.  Once again, I love the direction that they are heading in.  It has an updated feature set that should please customers no matter what their ideal deployment model is (On-Premises, IaaS, or PaaS).  You can also tell that they are serious about symmetry although it may take a while for PaaS to be closer aligned to On-Premises/IaaS but I think they are headed in the right direction.

Application Integration Futures – The Road Map and what's next on Windows Azure

This presentation has hosted by Bala Sriram and Rajesh Ramamirtham

 

Blog Update: I have added a conclusion where I have provided a summary and some of my thoughts on this release.  Also, this session has now been posted so feel free to take a look at it here.  You can also add any comments that you have regarding this session at the bottom of this post.

Key takeaway from Bala: We are innovating in BizTalk!

General Update

• BizTalk Server “R2” release will be available around 6 months after Windows 8
• CTP expect this summer
• Commitment to releasing server for years to come. Publicly indicating there will be at least another release beyond “R2”
• 12k+ BizTalk customers
• 81% of Fortune Global 100 use BizTalk
• 79% of customers are using BizTalk 2010
• CU delivered every quarter with product enhancements
• Best NSAT in the industry
• 6 of 8 largest US Pharmaceutical Companies use BizTalk
• Continue to bet on BizTalk – We will take your investments forward!
• Enabling new Azure based BizTalk scenarios for EAI & EDI
  • Bringing together BizTalk on-premises and in Azure

What customers are telling us?

• Keep me current with platform, standards and LOB changes
• Reduce time and cost of developing of Integration solutions
• Let me focus on business challenges, not technology infrastructure
• Cloud advantages
  • Cost-effective, scalable infrastructure for easy deployment
  • Some scenarios like b2b are amenable to cloud
• Cloud Challenges
  • Data privacy, isolation , control more integration
  • LOB assets will continue to be on-premise
• Phased cloud adoption on my terms
  • One size does not fit all

How BizTalk will meet these requirements?

• Upgrade to latest MS platform
• Improved reach for B2B customers
• Better performance and manageability
• BizTalk on Azure IaaS
  • Eliminate HW procurement lead times
  • Reduce time and cost to setup and maintain BizTalk environments
• BizTalk on Azure PaaS for EAI and EDI
  • Reduce partner onboarding and management cost
  • Leverage existing BizTalk artifacts
  • Rapid configuration-driven development for common integration patterns
• All of these working together seamlessly as one BizTalk
  • Trying to work under “one umbrella” but no naming can be implied at this time

BizTalk Server On-Premise Update

• Platform Update
  • Support for:
    • VS 2012,
    • Window 8 Server
    • SQL Server 2012
    • Office 15
    • System Center 2012
• B2B enhancements:
  • EDI
  • HL7 2.5.1, 2.6
  • SWIFT 2012 Message Pack
• Better Performance
  • In Order Delivery process
    • Serialization created delays
  • Improved dynamic send ports and ESB via host handler association of Send ports
    • Can configure a dynamic send port host handler in Admin Console
  • MLLP adapter performance
  • HIS DB2 client transaction load balancing, client bulk insert (15 times faster)
• Better manageability
  • Visualize BizTalk artifact dependencies in BizTalk admin console
  • ESB toolkit as core part of BizTalk setup and product
  • HIS Administration using Config files with application metadata stored in XML
• Improved connectivity
  • Consume REST Services directly in BizTalk
    • WebHttpBinding will be used when calling REST Services
    • ACS support
  • Simplified SharePoint integration experience
    • No more adapter web service installs on SharePoint
  • Improvements to existing adapters (HIS, SMTP)
    • improved macros
  • Easy connectivity to Service Bus Relay, Queues and Topics
  • CICS http client connectivity to Windows

BizTalk running in Azure (IaaS)

• Use case :

  • First step in the cloud adoption
    • Eliminate hardware procurement lead times
    • Reduce time and cost to setup and maintain BizTalk environments
    • Move applications from on premise and back
  • Create a virtual network in Azure and enable connectivity to on-premise network
    • User logs into Azure Portal
    • User creates a new VM and selects BizTalk stock image
    • User specifics BizTalk environment topology and adds them to existing virtual network
    • New VMs are provisioned for user in Azure IaaS
    • User logs into the provisioned VM which has BizTalk installed and configured and starts using it.
• Targeting same Windows 8 timeframe
• Microsoft will provide guidance on performance
• MSDTC support in Azure?
  • It is supported now in IaaS and was brought in to support BizTalk
• All features that work on premise will work in IaaS

Goals

• Seamlessly connect with Azure artifacts
• Enable hybrid applications that span Azure and on-premises
• Expose LOB services both on Premise and to the cloud

Conclusion

Wow…that was a lot of content in a short 1:15 h session.  There was actually more information released related to EDI support in the EDI Services (PaaS) but I just couldn’t keep up between writing this blog and tweeting with the European BizTalk community.

What I liked:

• REST support event if it is only Send
• Cleaner integration with SharePoint.  A similar statement was made with BizTalk 2010 but talking with the product team members after the presentation I know that this is not lip service.  The Adapter Web Service is gone.  No more installs on SharePoint servers.  Also, no more consuming SharePoint’s legacy “Lists.asmx” web services.  Yay!
• Ordered Delivery performance.  It will be nice to have some improved performance while maintaining sequential integrity.
• First class ACS support in selected “cloud enabled” Adapters
• BrokeredMessage property/BizTalk Context property support
• BizTalk IaaS – should open new capabilities
• I can see the symmetry between on-premises and PaaS starting to materialize

What I would love to see:

• Exposing REST end points
• Single Mapper/Transformation experience between On-Premises and PaaS offering
• Support for other sterilizers than XML (JSON, C#)  - stay tuned?
• Service Bus Connect – Receiving requests from LOB systems (SAP IDOCS)

 

Overall the tone was extremely encouraging.  Personally, I haven’t seen this much innovation come from the BizTalk team since BizTalk 2006 R2 when support WCF/WCF LOB adapters was introduced.  Yep..I said it.  The next release of BizTalk is no longer “just a platform update” .  In my opinion, this is a full release and should be named accordingly.  For those that think BizTalk is dead – better think again.  The operation was successful and the patient is still alive.

Microsoft TechEd North America 2012-Day 3

 

An Overview of Managing Applications, Services, and Virtual Machines in Windows Azure - Karandeep Anand

In this session Karandeep walked us through the new portal. The new portal does not include Service Bus endpoints like queues or topics and also does not include caching… yet.  I am told that Microsoft is hoping to have this functionality in the portal by end of the year.  However, they have enabled single sign on so you should be able to toggle back and forth between the old and new portals quite easily.

Within the portal we can get the state of:

• Virtual Machine
• Websites
• Cloud Services (Web roles/Worker roles etc)
• SQL Databases
• Storage
• Networks

Below I have taken a screenshot of the Azure portal.  I only have one SQL Database in the production portal but you can get a sense of the new look and feel.

You also have the ability to add new assets by clicking on the “+ New” link.

You can then specify the type of service that you would like to provision.  In the case of Virtual Machines, you need to sign up for the preview before you can actually provision a VM.

 

Scripting Management Support

Using the VM portal is not the only way to manipulate services in Azure.  There is first class support for scripting in Windows, MAC and Linux.  In the case of Windows admins will find comfort knowing that there is first class support for PowerShell.  These PowerShell command lets will take advantage of the same underlying REST APIs that the portal is using

 

High Availability and Service Level Agreements

Having a third party, such as Microsoft, host IT services for your organization may create some concerns within your organization.  What if your services go down?  What “skin” does Microsoft have in the game.  To put it bluntly, they have some skin in the game.  Perhaps not as much as some would like but Microsoft will be reimbursing organizations for their usage should they fail to live up to their commitments.

Here is a, very, loose break down of Microsoft’s SLA policy:

• 99.95 uptime – monthly SLA
• 4.38 hours of downtime per year for multiple role instances
• 99.9 for single role instances
• 8.75 hours per year
• What’s included?
• Compute Hardware failure (disk, CPU, memory)
• Datacenter failures – network failure, power failure
• Hardware upgrades, software maintenance – HOST OS updates
• Planned downtime – 6 day notice, 6 hour window, 25 minute downtime

What is not included?

• VM container crashes, Guest OS failures

Monitoring and Auto-Scaling applications

Now this was cool!  A company called AppDynamics demonstrated their monitoring solution for Azure.  Some of the features included:

• Application performance management dashboard.  This included a graphical representation of your distributed solution and provided the latency that exists between each component.
• You also had the ability to interrogate the stack level trace to get very granular
• The tool also supported the ability to auto scale your application based upon different criteria sets including
• CPU
• Message through-put
• errors
• specific business hours
• critical conditions

Since Azure supports a “Pay as you go” model I found this tool to be extremely intriguing.  Not only did it look nice, but it provides functionality can can allow you to reduce costs when your app is not very busy and also auto scale to ensure of a good user experience when your site is busy.  To read more about this company and their product for Azure, please read the following press release.

 

Building HTTP Services with ASP.Net Web API – Daniel Roth

The other session that I wanted to talk about was the Building HTTP Services with ASP.Net Web API.  For the past couple months I have been playing with MVC3, JQuery and AJAX so this session was rather timely.

What is Web API?

• An Http Service
• Designed for broad reach
• browsers
• phones
• devices
• set top boxes
• Uses HTTP as an application protocol, not a transport protocol.  So what this really means is it takes advantage of existing verbs GET, POST, PUT, DELETE

Why build Web APIs?

• Reach More Clients
• Scale with Cloud
• Embrace HTTP – simplify things
• use existing verbs
• Web API Requirements
• Need a first class HTTP Programming model
• Easily map resources to URIs and implement the uniform interface
• Rich Support for formats and HTTP Content negotiation
• separate out cross cutting concerns
• Light weight
• ASP.NET WebAPI is the end result

WebAPI description

If you like self documenting APIs, then WebAPI has some built in features to support this type of functionality.

• Use the IApiExplorer Service to expose “contract”
• Provides runtime description of WEB API
• Renders content in a useful way
• Shows request and response formats
• XML, JSON, url-encoded

Hosting

• Many options – self host(console), IIS, Azure roles, other web servers
• MSDN code gallery and NuGet Code packages are available

Other

• ASP.NET Web API is available as part of MVC4
• Is part of the recent open source movement that Microsoft has been involved in
• Product team accepts 3rd party contributions
• Unprecedented transparency
• When Microsoft  devs check in code, you have access to code through  GIT repository
• Asp.net mvc4 and web api is included in Visual Studio 2012 RC
• WebAPI is now a Visual Studio project template
• Can also create a unit test project
• New  MVC like map route for WebAPI
• api/{controller}/{id}
• JSON, XML and form-url-encoded supported out of the box for HTTP Request
• JSON and XML natively supported for HTTP Response
• Validation is run on the data from every request
• Check ModelState.IsValid to see if you have a valid request
• Support for ODATA queries
• return IQueryable<type> Get()
• decorate method with QueryAble

Conclusion

I realize that the release of this technology has been highly contested.  There have been people using the WCF stack that are now in a tough spot to migrate away from this technology to WebAPI.  For me, as someone new to this space I really liked how you organize your project and have clean separation from controller to controller.  You can quickly expose services without the need for heavy WSDL type contracts.

I also like how most of Daniel’s presentation was run from Fiddler.  Like he mentioned several times, WebAPI at its root is really just HTTP.  So what better tool to craft requests than Fiddler.

In closing, I do a lot of System Integration.  Primarily with BizTalk and must admit, I like contract based development where you are defining a firm contract upfront.  I have never been a big fan of loosely based lightweight services as things can quickly to to hell when doing this type of stuff for EAI.  However, I have woken up and seen the light.  I really do feel there are good use cases for this type of technology for light weight application based services.  I don’t necessarily think that this technology is a great fit for EAI, but for applications that may be surfaced using a variety of clients (mobile, web) I think this is a great way to expose back end services to front end client.

 

Stay tuned for Day 4 as I expect to have some encouraging BizTalk news to report!

Microsoft TechEd North America–Day 2

 

Day 2 Keynote

Day 2 opened with another Keynote this time led by Antoine Leblond.  It did address some of my concerns with Day 1’s keynote.  This time the focus was on Windows 8 and they did demonstrate the OS running on a few different devices.  The demos were nice and relevant, however they did not demonstrate it on new hardware.  They used the Samsung Series 7 slate with the Samsung Series 9 laptop.  They also demonstrated Windows 8 on a Lenovo laptop(sorry nothing cool about a Lenovo).  If they wanted to generate some excitement, and didn’t want to pull out a shiny new beta Samsung,  they should have just bought a new MacBook Pro and run Windows 8 on it.  I think Microsoft missed another big opportunity but I digress.

Line of Business Apps

They did demonstrate some good looking Metro applications on the devices.  They did provide a quick overview of the Beer Ranger app that I mentioned in a previous post.  They also demonstrated a native SAP application that is part of their Sales Automation Pipeline lineup.  The application was a nice looking app, but much like many other SAP applications, I had no idea what it was suppose to do (opps did I say that out loud).

Trackpad

My primary personal (non-work) machine is a MacBook Air that runs Windows 7.  Many people ask why I do this and how I am missing out on all of these touch gestures.  For me I love my Microsoft Arc Touch mouse and quite frankly could care less about the gestures that you can use on a trackpad.  But for those of you who like using gestures on a trackpad there is some good news for you.  Windows 8 will support “Apple like” gestures including Semantic zoom and access to the new “App bar” to name a few.

 

Developing for Windows 8

Antoine provided us with a rundown of what is involved in building applications for Windows 8.  Here are some of the highlights.

• Windows Runtime (WinRT)
  • New API set that allows you to build apps and games
  • Supports touch, keyboard and mouse
  • Support for “contracts” so that apps can leverage OS functions like “share”.  Much like you can Share “data’  on a Windows Phone 7, you will be able to perform a similar function with your own application but hooking into these contracts.
• Tool Support
  • new release of Visual Studio (2012)
  • C++, C#, JavaScript, css, html are all supported technologies
• Language and platform support for inline async calls. 
  • I really like this feature.  I was never a big fan of all of the delegate spaghetti code a person previously had to write to support async methods.  In a previous post, I had to write async REST methods to support calling REST based services from a Windows Phone 7 app.  I expect it to get much  simpler now.
• Visual Studio Simulators
  • Can simulate different types of hardware(slates, desktops, large displays, small displays etc)
  • Rotate screen
  • Higher/lower resolutions
  • Touch gestures

ARM Support

This was interesting to see as I have heard a little bit about ARM support but just have not seen it in action. 

Some of the benefits of leveraging ARM based devices include:

• Low power consumption
• Long Battery life
• Trusted boot
  • Validate all code in boot path before it runs
  • Device encryption is always on
• App model
  • Geared at making apps that don’t alter the state of the machine (Security benefit)
• Can use same Management infrastructure to manage devices
• Metro apps work on WinRT as well
  • An RDP client does exist so that you can log into other Windows PCs/Servers
  • Apps must be signed by known trusted authority or have appropriate cert
  • Apps must must have been run through “WAC” approval application
    • used to honor design principals about not altering the state of the machine
• Key office applications are available

Essential Tips for the Windows Azure Startup

This was a really interesting session.  Michele Leroux Bustamante is well known in the WCF and MVP community as a person with deep technical skills.  I have seen her speak before at a previous TechEd so I thought this would be a good session to attend.  Something that I appreciate about Michele’s presentation style is that she remains composed through out the entire presentation.  Even when she runs into some issues, such as a demo not quite working out, she is able to recover with a tremendous amount of poise. I believe she is Canadian, maybe this has something to do with it. 

This time around she was giving guidance on developing a Startup based upon Windows Azure.  It was a very enlightening session and it was quite evident that she “gets it”.  She has acted as a consultant to many start-ups and provided the following tips when building applications for Start up companies.  Something to keep in mind is that these principles, while applicable to Startups, also just good practices to follow even if you are a well established brick and mortar company.

• Startups need to show some traction early on
• Go fast, maintain quality
• Monitor status, analytics and adjust accordingly
• watch for conversion rates
  • do visitors create accounts

10 Essential Tips

Here they are as they were presented to the audience:

1. Design for Role Scale out

• Needs to happen up front.  By the time you need to scale it will probably be too late or more difficult to
• Need to design for scale, may want to segregate or isolate controllers to allow to further scale out functions that may be more popular or have more access patterns (Mobile, API)
• Domain Mapping
• Create a CNAME or A Record for the IP address of your production deployment

2. Use an SMTP relay service

• Most applications require some form of email communication
• Can use System.Net.Mail.SMTPClient
• Write email “messages” to a queue and then dequeue and send
• Need to use a relay service so that your “From Email” address does not get blocked/spam
• smtp4Dev is a great tool for use in Development
• authsmtp is a production ready email relay service that may be beneficial

3. Configuration Profiles

• Avoid web.config for
• settings that vary between staging, production
• settings that require experimentation for performance
• settings that support diagnostics and test
• Use the Azure Service config files instead
• ServiceConfig.Local.cscfg
• ServiceConfig.Cloud.cscfg

4. Don’t forget to Cache

• You don’t realize how much latency that accessing frequent data creates
• Co-locate Cache with across roles
• Together produce distributed cache total
• Any role can access
• Be careful, Cache is not durable, may not live forever
• Use for optimization
• Performance increases are phenomenal

5. Watch your Queuing costs

• Costs may escalate due to the amount of polling
• If you are polling, you are paying
• Understand the differences between Service Bus and Azure Storage Queues 
  • Message lifetime
  • Max message size
  • Max total storage
  • Duplicate detection
  • Order guarantees
  • Dead letter queue
  • Storage metrics
  • Purge capability
  • Long polling/manual back-off polling
• Initial decisions are about cost and agility
• consider Storage Queues due to back-off polling

6. Collect Diagnostics

• When writing new project, there is a lot of hacking going on because you are trying to be fast
• Difference between getting it done and getting it done properly
• Create a diagnostic helper and establish patterns

7. Monitor from outside

• Azure Ping free monitoring tools
• Sends SMS or email when monitoring
• Storage
• SQL
• Queues
• Is site running?
• Azure Watch
• Monitoring and alerts

8. Don’t drink the no-SQL Kool-Aid

• VCs love “NoSQL”
  • Can be pressure from VCs to use it
  • VCs think it is cheaper to manage because you don’t need a DBA
    • Asking for trouble if you don’t understand your relational data model
• Need people who understand SQL to look into NOSQL and report back to the group on what the pitfalls are
• Go to NoSQL for obvious stuff
  • search indexes
  • GEO data
  • profile data (coming from social media)
• Keep core competencies in RDBMS
• Then reach out to NoSQL  experts to help bridge the two worlds

9. Enable Social Logins and Simplify Sign Up

• You want conversion rates – make it dead simple then!
• ACS facilitates this – simple to use
• Dirt cheap per transaction
• The more you ask from a user to register, the less likely they are to sign up
• Keep it simple you will get conversions
• Pinterest – only email address to sign up?

10. Estimate your costs

• Layers of cost
  • Storage
  • Storage Transactions
  • Bandwidth (# 1 thing if you have  a lot of media)
  • Cache
  • ServiceBus
  • SQL Azure
  • Bandwidth
• Need to run estimates
• scenario based
• BizSpark may offer some cost savings for new startups

Service Bus Overview – Clemens Vasters and Abhishek Lal

As usual Clemens put on a good show. This time Abhishek joined him in this presentation and provided some solid demos.  The first half of the presentation was largely a review for me as Service Bus is an area that I try to stay up to speed on.  The second half of the presentation introduced some new tooling and features as part of the June 2012 release.  Selfishly, I don’t want to go into too many details here as I would like to actually play with some of these features and then provide a more complete blog post(s) in order to provide these subjects with some additional context.   Stay tuned!