Kent Weare's BizTalk Blog

BizTalk Server 2010 R2 Announced

2011-12-08T21:06:00.001-08:00

In a previous post, I discussed Microsoft’s integration road map particularly related to Microsoft BizTalk Server. The foundation of that post was based upon Tony Meleg’s presentation from the World Partner Conference. The good news is that today, Microsoft has given us a glimpse into what is to come with respect to BizTalk Server. Previously, Tony cautioned us that Microsoft will continue to support BizTalk and will introduce incremental changes to the platform. Where as most of the innovation will occur in the Azure AppFabric space.

The next version of BizTalk Server is currently called BizTalk Server 2010 R2. I suspect they are calling this an “R2” release since it is an evolutionary release that particularly focuses on platform alignment with Windows Server 8, SQL Server 12 and Visual Studio 11. For those of you who do follow the BizTalk scene closely you may recall that BizTalk Server 2010 was originally called BizTalk Server 2009 R2 when it was in beta. It was renamed to BizTalk Server 2010 prior to RTM. So who knows if the BizTalk 2010 R2 name will stick but for now that is the name we will go with.

In addition to the platform alignment that I have already mentioned, we can look forward to some new features in the areas of:

Adapter connectivity to new data sources including IBM Informix V11 and IBM IMS/DB V11
Updates to industry schemas like HIPPA, HL7, SWIFT and SWIFTNet
Improved Performance and Scalability
Tighter integration with the Azure Service Bus
Adjustment to licensing geared towards cloud hosting

Of the features mentioned, I certainly will not complain about improved performance and am happy to see tighter integration with Azure AppFabric. I really hope that they introduce a smoother way of integrating with Azure AppFabric Queues and Subscriptions.

I am a little disappointed that there was no mention of fully supported REST endpoints, but it is early so I am hoping that Microsoft will surprise us with that feature.

I have only touched on some of the new features that will be available. You can find the complete post from Microsoft here.

BizTalk: Adding BizTalk 360 to your Environment

2011-10-01T09:58:00.001-07:00

For those that are not familiar with BizTalk 360, it is a Rich Internet Application (RIA) that allows you to monitor and maintain your BizTalk environment via a Web browser. From what I can see, from a high level, there are a few components that you install in your BizTalk environment that make up the solution:

Silverlight Application which can be accessed via a remote web browser
WCF Service(s) that expose the operations to the Silverlight application
A Windows Service that handles some of the monitoring and notification features
A BizTalk 360 database that will store your configuration and audit/governance

My organization recently purchased a license and support from BizTalk 360. In this blog post I plan on explaining some of the rational and features that influenced our decision.

Requirements

Over the past 5 years our BizTalk environment continues to grow. With this growth comes more responsibility maintaining it. Our current middleware team consists of both developers and administrators and therefore there is an on-call rotation that is used to handle the after hours support. After hours support requirements are very binary; BizTalk’s infrastructure is either on or off. This would include the core components like SQL Server, SSO, Host instances and applications being online. We are not concerned if a single message is suspended. The reason? There is no one from the business available to consult with in order to correctly determine a resolution to the problem.

Also, in parts of Canada the IT Market is getting hot. Let’s face it there is nothing fun, or cool about being on-call. Even though our environment is very stable, people felt that they were still tied down to their cell phone since we have Service Availability metrics to meet and when an issue occurs response time is very important. As all BizTalk resources know, BizTalk doesn’t have to be down to get roped into “fire fighting” triage sessions. Sometimes your large ERP or engineering systems go down and since BizTalk connects into them you are impacted as well.

The company that I work for is very committed to employee retention and reducing the amount of on-call is a step in the right direction for reducing turnover and keeping employee morale high.

Solution

I know Saravana from the MVP program and at the last MVP Summit Saravana was showing MVPs and some of the BizTalk Product Group an early preview of what is now BizTalk 360. My initial thoughts included what a nice looking application and some of the features fill a void left by the BizTalk Admin Console but I wasn’t convinced that I had an immediate use for it.

While I was in Sweden in June, with Richard Seroter, promoting the book I contributed to, I ran into Saravana again and checked out the progress his start-up made on the product. Once again I was impressed with the tool but wasn’t sure how I would use it.

Come July and some more concerns about the on-call situation arose and this turned on a light bulb in my head about how we could use BizTalk 360 to help support our environment after hours. We already have contracted resources that focus on after-hours support for our core infrastructure: Servers, SAN, Network etc. I thought what if I could piggy back on this existing agreement? Knowing that I didn’t need BizTalk Experts to keep an eye on my servers after hours, remember the type of service I am looking for is binary: on or off. BizTalk 360 provides this visibility in a couple ways:

Dashboard – By looking at this type of screen if is very obvious which artifacts are up and which are down. Enabling these artifacts is a pretty easy action to perform so I don’t need to pay to have BizTalk Experts monitoring after hours.

Notifications – My company also uses SCOM for monitoring BizTalk and we will continue to do so even with BizTalk 360. But the feature that I like about BizTalk 360 is the Health Summary email notification that you can configure. I can now subscribe this after hours team to periodic notifications where they can quickly, and easily, determine the health of the environment. I don’t want these people logged onto my BizTalk servers hitting F5 all night inside of the BizTalk Admin Console. Using these email notifications frees them up to work on other things but also provides the visibility that we need.

(Sample email notification – note I have added the black marks to retain some privacy)

Another area that is important to me is the Audit and Governance features that exist. When trusting a 3rd party to provide support it is very nice to have an audit of what actions took place during support periods. It protects both the client and the service provider and is a win-win all around. This is a feature that has been missing from the BizTalk Admin Console that could benefit many organizations.

Overall the process of procuring the BizTalk 360 application has been very positive. When I first tested out the product there were a few features missing that I really wanted. Saravana took the time to review them over a Live Meeting, took my high priority needs and was able to provide me with a build on a committed timeline. Some of my feature requests involve clustered host instances so if you use clustered host instances in your environment – you are welcome!

The BizTalk 360 team is very committed to the product and their customers. It has been refreshing to work with a company that is so motivated.

Conclusion

Something that I have learned through this process is to always keep your eyes open to new trends and tools. I find Twitter really helps in this regard by following likeminded individuals. You don’t need to deep dive on every new technology that you see but it is important to know that it exists as you never know when your requirements may change and you can benefit from it. Case in point my situation that emerged this past summer.

BizTalk: /n Software Read Only (S)FTP Locations

2011-08-26T11:06:00.001-07:00

Another /n Software feature that I recently learned about involves Read Only FTP locations. The idea behind this feature is the the source system does not want you to remove the file once it has been downloaded. This could be for a variety of reasons including the file may be available to multiple consumers or the source system would like to archive the file once it has been consumed.

Read only FTP locations is a new supported scenario with BizTalk 2010. /n Software has supported this scenario for at least a few years. You can enable this feature by simply specifying the Delete Mode as Never on a Receive Location that is using the /n Software adapter.

So this is all very intriguing, I am sure, but it is pretty obvious so why does this deserve a blog post? The reason for the blog post is to discuss a lesser known property called DownloadCacheFile. After scanning your receive location you will discover that this property is not visible. This property is exposed through the Other property. Within this property we can specify DownloadCacheFile=c:\myCacheFile.txt

As files are processed they will be added to this file and a record is kept so that they are not downloaded in subsequent polls of the (S)FTP location.

In the BizTalk 2010 implementation, Microsoft is using a database to store this type of information where as /n Software has chosen the file route. Time will tell how well this feature works. I would imagine that there is an upper limit where the file becomes so large that it slows performance or potentially corrupts.

Also note that if you are using a Clustered host instance to host your FTP Receive locations, this file should be in a shared storage location so that the Clustered host instance has access to it no matter which Server it is actively running on.

BizTalk: Configuring /n Software’s SFTP Adapter for Public/Private Key Security

2011-08-26T09:54:00.001-07:00

I was recently involved in a project that required secure FTP connectivity between my organization and a partner organization. We continue to leverage /n Software’s SFTP adapter as we are still running BizTalk 2009 in production and we also find that our partners tend to leverage Secure FTP using SSH (SFTP) as opposed to FTP over SSL (FTPS). BizTalk 2010 now includes an FTP adapter that supports SSL so we are likely to continue to use the /n Software adapter based upon SSH requirements even once we upgrade to BizTalk 2010.

I am very far from being a Security/Certificate expert so I did learn a few things being involved in this project. Hopefully if you have Secure FTP requirements that you will find this post helpful.

Generating Public and Private Keys

In my scenario, our trading partner required us to provide them with our Public key. They wanted us to generate Private and Public keys. They would then take our public key and install it in their SFTP server. To generate these keys I simply used GlobalScape’s CuteFTP 8.3 Professional.

Note: Your mileage may vary here. There are some requirements that were enforced by our Trading partner including the Key type and the number of bits required for encryption.

1. Click on Tools -> Global Options

2. Click on Create identity file

3. Use DSA

4. Provide a Passphrase

5. Enter a location for files to be generated and ensure the key length is set to 1024

Configuring BizTalk Receive Location

Since the Receive Location configuration is a little lengthy I am going to break it down into the various sections.

SFTP

Pretty basic settings here dealing with when we want to Delete, what file masks we want to look for and the folder on the remote SFTP server that we want to navigate to once we have successfully established connectivity.

SSH

The first area to focus on in the SSH section is the SSH Accept Server Host Key. When you set the Accept Any to Yes the SSH Accept Server Host Key property will revert to Any. I equate this action with when you try to connect to an SFTP server using a client like CuteFTP. You will get prompted with a dialog asking if you would like to accept the public key being pushed from the SFTP server. The way I understand how this works is that this public key will get validated against your private key. Should everything match up you should be able to establish a connection.

The next property to focus on is the SSH AuthMode. We want to set this value to be Public Key. There are a few different options when it comes to setting the Authentication mode. One includes setting a password but for this particular partner they wanted to use Public Key authentication. When we set this value we then need to provide the location of our private key which happens to be called Identity (without a file extension).

To provide our Private key we need to click on the ellipse button.

We will now be prompted with a dialog that allows us to select our Private key from a variety of sources. In my scenario I wanted to select my key from the file system. In order to do so I needed to select the PEM tab and then browse to my key. Since I have enabled my key to use a password I need to provide a password and then I can click the Open button. Once I did this my private key would appear in the TextArea box and I can click Ok.

The rest of the configuration deals with some connectivity details including the name of the FICTICOUS Host and a SSH User (a tribute to Steve Jobs who has recently stepped down from being Apple’s CEO) . The SSH Port is using the default port of 22 and even though it appears as if I have provided a password that is the default mask provided by the adapter.

Conclusion

So as much as people love to talk about the cloud these days and using the Service Bus and WCF bindings, the bottom line is that companies continue to rely upon “legacy” technologies such as SFTP. In my scenario, this trading partner is a very large institution so it is not as if they are a little “Mom and Pop” shop.

BizTalk 2010 Training Kits now available

2011-08-25T21:22:00.001-07:00

Both Developer and Administrator Training Kits have been recently released. Inside of these training kit you can expect functional Virtual Machines, with the BizTalk software installed, training module documents and PowerPoint presentations. This is the content that you would typically find in a Microsoft Official Curriculum course. The best part about these kits is they are made available for free.

I had the opportunity to collaborate with Jerry Anderson from Pluralsight on the Administration course. My contributions primarily focused on Module 6: Monitoring a BizTalk Environment with System Center Operations Manager (SCOM).

The BizTalk 2010 Developer Training Kit was released back in July and is available here. The Administration course was released on August 21 and can be found here.

Enjoy!!!

BizTalk 2010: Line of Business Systems Integration Book has been released

2011-07-18T21:22:00.001-07:00

For the past 10 months I have been involved in a project with 4 other talented individuals. This project was a little different than the projects that I have been involved with in the past. While other projects have always had some sort of documentation deliverable, documentation, in the form of a book was the main deliverable this time around.

While many BizTalk books exists, we felt there was an opportunity when it came to discussing Line of Business integration with BizTalk Server 2010. With the amount of talent involved in this book, I am very confident we have filled some of the void that exists on this topic.

I would like the thank the following authors for their tremendous efforts:

Richard Seroter
Sergei Moukhnitski
Thiago Almeida
Carl Darski

I also want to thank our Technical Reviewers who spent a lot of hours reviewing our content and definitely increased the quality of the book:

You may now find the book available on the Packt Publishing site and on Amazon.com.

Sample Chapter available for upcoming BizTalk LOB Book

2011-07-17T20:57:00.001-07:00

I see that Packt Publishing has included a link to a sample chapter in our upcoming book called Microsoft BizTalk 2010: Line of Business Systems Integration. The sample chapter selected is: Integrating with Dynamics 2009. You can read more about the book here and can find the sample chapter download here.

I don’t have a publish date but I expect it to be imminent. The content has been provided to the printer. I will update this post once a publish date is available.

BizTalk Server still has a pulse

2011-07-17T16:45:00.001-07:00

While this information is now well known and has been covered by both Richard Seroter and Saravana Kumar, I wanted to throw my 2 cents out there as I was away on vacation when this information was released. Now that I am back, I have had some time to watch Tony Meleg’s presentation from the 2011 World Partner Conference. I have a feeling that this “talk” will be one of those that is referenced for years to come. Much like we continue to bring up certain “Scott Woodgate” talks.

A New Platform is born

For the past couple years, Microsoft has put significant emphasis on cloud computing and providing software and platform services that run in Microsoft’s data centers. From a platform perspective, these investments have been made to Azure. Today we have capabilities to run Web and Worker roles, SQL and Azure Storage, Caching and relay messaging in a production environment.

Microsoft continues to invest in this platform and is increasing the amount of capabilities that are provided in this platform. We are now starting to see some of the “middleware” capabilities being introduced in the form of CTPs that include Workflow, Pub/Sub and Queues. A future CTP will showcase additional “integration” capabilities including pipelines that will host message transformations.

New Platform’s impact on BizTalk

It has been very clear that Microsoft is betting on the cloud and as a result all new software investments will occur in the cloud first and then make its way to on-premise. The first example of this strategy was the release of CRM 2011 which was available in the cloud prior to making its way into on-premise.

To align with Microsoft’s cloud strategy, some tough decisions had to occur in order to include BizTalk functionality in the cloud. A key area that had an impact on BizTalk’s ability to move to the cloud was in the area of Workflow. A decision had to be made as to which “Workflow-ish” technology Microsoft would pursue between Windows Workflow Foundation and Xlang. The driver for this decision was Microsoft’s desire to reduce the amount of duplication of efforts across their platforms. As you can see from Tony’s presentation, Xlang did not make the cut. In my opinion, it makes sense to decide upon Windows Workflow as it can be used outside of BizTalk and is already used in other platforms like CRM and SharePoint. Tony does mention in his presentation that removing the Xlang Orchestration engine from BizTalk is like performing a major heart surgery and will take several years to fulfill.

Microsoft also has aspirations of ensuring symmetry between cloud and on-premise versions of its software. The current BizTalk architecture just didn’t provide a good fit especially when consider the desire to reduce the duplication of efforts across platforms. It was pretty clear that Workflow would be the engine of choice being a .Net based technology and its adoption by other platforms.

Another area that is at the top of customer’s “ask list” was more flexible messaging options. While BizTalk’s current durable messaging is a core requirement for many customers, the ability to by-pass durable messaging in order to increase performance is a core requirement for others. We have the ability to perform non-durable messaging with WCF and WF today so once again this is a natural fit over trying to change the existing BizTalk messaging engine.

Platform end state

Below is an image that I snagged from Tony’s presentation. It represents the new platform’s “end-state” in terms of capabilities. Within this diagram you should be able to find existing BizTalk capabilities and their alignment to the new platform including:

Adapters
Rules
Pub/Sub
Transformations
Pipelines
BAM
TPM

We also have some enhancements or new capabilities not found within BizTalk natively like Caching, Web Apps, Workflow and the new Composition Model and AppFabric App Manager. I have discussed a few of the Composition Model and AppFabric Manager on my other blog http://www.MiddlewareInTheCloud.com and am looking forward to “BizTalk” app development being able to leverage these features. I was reminded of the convenience of the Composition Model when I recently deployed a BizTalk application to multiple servers that included deploying Web Services. The BizTalk deployment model could be optimized in my opinion when it comes to distributed deployments. Yes there are tools that help like MSBuild and the BizTalk deployment framework but it should be simpler. I would love to see the simplicity that the AppFabric Application model provides to “BizTalk” applications.

Timing

Tony asks people to “lower expectations” and “think long term”. Microsoft recognizes that they will not be able to move existing BizTalk functionality to this new platform in either the cloud or on-premise any time soon. It will take multiple releases to achieve the goals of the platform. Their timeline is somewhere around 5 years. In the mean time Microsoft is going to continue to release versions of BizTalk Server. Yes, you read that right…BizTalk is not dead. However, Tony cautions that we will not see a lot of innovation being baked into the new BizTalk bits. The updates to BizTalk server will be to align to updated platforms like Windows Server, SQL Server and Visual Studio. Updates may also include bug fixes or key customer asks that are reasonable to implement. However, the innovation will occur in this new platform.

In the short term, we will see features of the new platform being rolled out in the form of CTPs. We shouldn’t expect the new middleware features to hit production until sometime in 2012.

(another image that I swiped from Tony’s presentation)

Migration

As we move from our existing platform to the new platform, we should be expect some pain. Obviously Microsoft will take reasonable steps to ensure of smooth transitions to the new platform but we can expect some challenges along the way. There will not be an “easy button”.

Guidance (my opinion – take it as is without warranty)

The first piece of advice is: DON’T PANIC. It is still early and nothing is written in stone at this point. Microsoft has mentioned that this is a long term strategy and it will take multiple releases to address the vision. In the mean time they will continue to support and release more BizTalk. From a personal perspective, I am responsible for a Middleware team at a good sized Energy Distribution provider. We run BizTalk 2009 and I can confidently say that our business runs on top of BizTalk. If BizTalk goes down, many parts of our business slow down if not grind to a halt. We have been making significant investments in BizTalk for over 5 years. My intentions are to continue use BizTalk. We have an upgrade planned in the next 8 months and that will be a BizTalk upgrade. Most likely 2010, but if a newer v.Next release miraculously appears we will consider it.

From a developer perspective, this news is a bit of a wake up call. It is time to start diversifying your skill set. Microsoft is providing initial looks at this technologies through CTPs. Get involved! Download the bits, try them out and get familiar with some of the new concepts and models. I don’t think there is any reason to be really nervous though. The underlying core integration skills that you have built up over the years will not be for nothing. Some of the terminology will change, the toolset will probably change but many existing integration patterns will continue to exist whether there is a “cloudy thing” or not. If anything, I think there will be a lot of new opportunities for developers with Microsoft Integration skill sets. There will be a lot of existing BizTalk –> New Platform migrations that will need to occur in the next 3-10 years. Who better to work on these projects than people with BizTalk experience who have adopted the new technology.

First look at Azure AppFabric June (2011) CTP

2011-06-25T12:32:00.001-07:00

Recently Microsoft has released another Azure AppFabric CTP. This particular one focuses on Azure “AppFabric Applications” or otherwise known as “Composite Applications”. The goal of Azure AppFabric Applications(as I see it) is to build, deploy and manage multi-tier applications as a single logical entity in Windows Azure.

Coming from the BizTalk world I am very familiar with distributed applications. Some of the challenges in developing and maintaining these types of applications is understanding all of the moving parts that are involved in these solutions. AppFabric Applications is a step in the right direction as it provides a holistic view of our “stuff” that makes up a distributed application including Web Applications, Custom and 3rd party Services, Service Bus capabilities (Relay/Queues), Workflow and Storage(DB/Blob/Table).

Managing, and understanding, different application/solution tiers independently can be a bit of a nightmare. One of the features of AppFabric Applications the ability to automatically generate a diagram that describes all of the core components of our distributed application and the related dependencies.

If we take a look at a diagram that describes one of the sample solutions we can quickly discover what our solution is made up of. In this case we have a Web Application that will push data to an AppFabric Queue. In turn we have a service that pull this data off of the queue.

So at this point you may be thinking “whoopee” I can draw that in Visio in 2 minutes. Well Visio will not be able to deploy this entire solution to the cloud in the matter of a few mouse clicks. Once this application is in the cloud we then have the ability to provision our various tiers by “turning a knob” or in this case pulling down a menu to select the amount of instances we require.

Another nice benefit is that we can trace and instrument our entire solution from once place. This is another pain point of distributed systems. There is usually logs all over the place that have to be aggregated to get a sense of the performance and health of an entire application. This is another benefit of using Azure AppFabric Composite apps.

Unfortunately my experience with this technology is only in the Local Development fabric as I am currently waiting to get access to the “Labs” environment in Azure. But if you check out some of the resources that I have outlined below you can see some demos of the Management features available in the Cloud.

Summary

Benefits:

Ease of deployment
Greater developer productivity
Effortless scale
Centralized managed and monitoring

CTP includes

AppFabric Developer Tools
AppFabric Application Manager
Composition Model
Support for running Custom Code, WCF and WF

What you need

Windows Azure AppFabric CTP SDK and Windows Azure AppFabric Tools for Visual Studio

Resources:

Before you start, I recommend watching the following videos:

Channel 9 announcement video
Alan Smith’s webcast
TechEd 2011 North America video

I also suggest you take a look at the Tutorials and Samples.

The Samples that are available include introductory apps like a StockQuote app that has a Web Front end and then consumes a back end service to a Contoso Pizza application that includes a Web Front End that consumes a back end service which leverages some WF workflow.

Closing

This new CTP has me really intrigued. I plan to investigate further and will blog more about my findings.

BizTalk 2009 – Be careful when setting Service Windows on Send Ports

2011-04-11T12:38:00.000-07:00

Note: This blog posts pertains to BizTalk 2009. It has not been tested against other versions although I suspect the behavior is the same.

We recently had a situation where a downstream system was having issues processing certain types of messages. We were asked to “queue” messages received from the source system until the issue was resolved. In order to do this we simply stopped the Send Port but left it enlisted. Since the Send Port is still enlisted, a subscription still exists within the MessageBox database. Any messages received while the Send Port is in this state are essentially “queued”.

When you send a message into BizTalk and have the port Stopped(but enlisted) you can expect a Suspended message that has an error description of “Service instance was suspended because the corresponding service (orchestration, sendport, ...) was in the stopped state. Instance can be resumed after corresponding service is started.”

Even with this Send Port stopped, we can still right mouse click on the suspended instance and resume it without issues.

We knew that we had to have this send port stopped for a few days while the issue was resolved. Since we have multiple people working within our Middleware team and also have automated processes to ensure our applications are online, we decided to set a Service Window on the Send Port in addition to having these messages queued as a precautionary measure.

The intent of this Service Window of 1 second allows us to log into the server before 9am to ensure the Send Port is not started. Then we would disable the service window and use a time that is before the current time. The current time, for the purpose of this blog post, is 10:06 am.

If we send in another message with this Service Window we will see a screen much like we saw without the schedule in place.

If we check our BizTalkServerApplicationQ table within our MessageBox database we will discover that no records exist:

If we check our BizTalkServerApplicationQ_Suspended table we will discover that metadata exists for our message that we just received.

So what are these tables? These tables make up BizTalk’s work queues for our BizTalk Server Application Host. Stay tuned and note which tables have records as you will soon see a discrepancy.

With our message “queued” and our Service Window still set for 9:00:00 am to 9:00:01 am I am going to resume this message and it will be delivered successfully even though the Send Port is stopped.

Great! So what is the point of this post? Change your window to be 1 minute instead of 1 second and you will get an entirely different behavior.

With the Service Window set to be 1 minute and with the current time set to 10:21 am, I will now submit another message to BizTalk. Once again I will get a suspended message.

If we take a look at the BizTalkServerApplicationQ_Suspended table we will discover that our message is in there but is not in the BizTalkServerApplicationQ table

When resuming the message this time, it will not get delivered to the end point. Instead it gets into a “Retrying and idle” state.

If we further investigate this message we will determine that the Message Status is set to “Queued (scheduled for later delivery)”

If we check out the status of our tables in the MessageBox we will determine that the message is no longer in the “Suspended” table.

We can now find the message in the “Q” table. Also note the Start and End Windows.

So what can we do if we want to resume this message since the Downstream system is now available and they want the 1800 messages that have been queuing up over the past 3 days?

Maybe we should remove the Service Window from the Send Port? (hint – it doesn’t help)

Maybe we should start the app? (hint – it doesn’t help)

Maybe we should restart the host instance? (hint – it doesn’t help)
Since the message is in a “dehydrated” state maybe we need to wait 5/10/20 minutes for BizTalk to wake up? (hint – it doesn’t help)

There are really two solutions in my mind:

Wait for the actual service window, with application online and Service Window checkbox removed, and these messages will get processed – promise.
Update the service window in the “Q” table so that BizTalk will send these messages when this window is met. To demonstrate this I will update the table with a timestamp that is in the near future. It is currently 10:46 am and I will update the timestamp for the window to start at 10:50 am.

Without any intervention the outstanding message(s) will get processed/delivered at 10:50 am.

Key Takeaways

Be aware that Service Windows are “attached” to the messages as they are being processed. There is no way to modify this except through the database.
Changing the Service Window of a Send port where messages already have a Service Window has no effect. (It is too late)
Setting a Service Window of 1 second has no impact.

Internal BizTalk Conference

2011-03-08T21:57:00.001-08:00

Prior to the MVP Summit, Johan Hedberg and Mikael Håkansson stopped by Calgary and participated in a Mini-BizTalk Technical conference for my company and three other sister companies. In addition to their presentations we had a round table discussion to find out how the other companies are using BizTalk and had two other presentations; one by myself and another by a colleague Luciano Barbieri.

Below you will find links to the various sessions. I wish I would have recorded these sessions as the demos were excellent. Maybe next time.

Johan Hedberg – BizTalk 2010 and Mapping best practices
Mikael Håkansson – BizTalk Performance
Luciano Barbieri – Automating BizTalk Builds
Kent Weare – Extending your enterprise via BizTalk 2010 and the AppFabric Service Bus

Catching SOAP Faults from CRM 4.0 Web Services

2011-03-08T18:23:00.001-08:00

A natural follow up to my BizTalk 2010: Calling Dynamics CRM 4.0 Web Services post is one that deals with the Exceptions, or Faults, that these Web Services may return. When using the CRM 4.0 Adapter, the adapter would take care of handling SOAP exceptions and bundling them up into a CRM Response message that had a Return Code and Error Nodes. Whether your operation was successful or not, you could always expect the same type of response message from CRM.

<ns0:Response xmlns:ns0="http://schemas.microsoft.com/crm/BizTalkAdapter/Response">

<Header>

<ReturnCode>1</ReturnCode>

<ErrorCode />

<ErrorString />

<Retryable />

</Header>

Now that we are not using this adapter anymore we need to be able to catch our own exceptions coming out of CRM. If we don’t perform these actions below(or similar actions) we can expect an error message like the following.

Inner exception: Received unexpected message type 'http://schemas.xmlsoap.org/soap/envelope/#Fault' does not match expected type 'http://schemas.microsoft.com/crm/2007/WebServices#CreateResponse'.

The issue is we have a Solicit Response Send Port in which we are sending a typed Request message into CRM and are expecting a typed Response message in return. When we encounter a SOAP Fault, a typed message is being returned, it just isn’t what we are expecting. In order to avoid these situations, we need to perform the following actions within our BizTalk solution.

Note: these actions are not specific to CRM, but may be used in other Web Service scenarios.

Create a multi-part message that includes a part that is of type BTS.soap_envelope_1__1.Fault. You will find this schema in the Microsoft.BizTalk.GlobalPropertySchemas assembly.

Right mouse click on your selected operation and select “New Fault Message”

Select the Message Type to be the value of our Multi-part message that we just created.

Add a scope around your send/receive shapes. The transaction type can be “None” and the Exception Object Type should be set to the Fault that we just created within our Operation.

So while technically this is defined as a message, within our current scope exception handler it is actually an object. So if we wanted to dump the contents of this message to the Event Viewer we could perform the following actions by assigning the message to an XML Document and then getting the OuterXml so that we can send this text into the event viewer.

At this point we can stop if we are so inclined. If we wanted to actually use this information in our CRM Response message we can assign this “exception” object into an instance of a message that is of the same type (our multi-part message).

We then can use a Message Assignment shape to assign this object into an instance of our message.

Now that we have a typed message, we can use this message in a map to instantiate an instance of our CRM Response. To keep things simple I am just going to concatenate the faultcode, faultstring and faultactor values and assign to the CreateResult node. If we wanted to get the actual details out, we will need to write a .Net helper method or use XSLT to extract this content out since we have an untyped “Any” node.

After all of these changes, our Orchestration should look like this:

We can now deploy our application and test it. In order to generate a SOAP Fault, I am going to send in the same message as in my previous post, but this time I am going to make the First Name to be greater than 50 characters.

Now when we process this message, we will not get an unhandled exception or suspended message. Instead, we will see an informational message in our event viewer that contains the details that we are interested in.

Also, we will send this error information to a folder in the form of a CreateResponse message that will include some SOAP Fault details.

<ns0:CreateResponse xmlns:ns1="http://schemas.microsoft.com/crm/2007/CoreTypes" xmlns:ns2="http://microsoft.com/wsdl/types/" xmlns:ns3="http://schemas.microsoft.com/crm/2006/Query" xmlns:ns0="http://schemas.microsoft.com/crm/2007/WebServices" xmlns:ns4="http://schemas.microsoft.com/crm/2006/Scheduling" xmlns:ns5="http://schemas.microsoft.com/crm/2006/WebServices" xmlns:ns6="http://schemas.microsoft.com/crm/2006/CoreTypes">

<ns0:CreateResult>soap:Server - Server was unable to process request. -</ns0:CreateResult>

</ns0:CreateResponse>

Note: some other blog posts mention using an XPATH statement, within our WCF Send Port, for both types of messages that we are expecting; typed CRM response and SOAP Fault. In my scenario, this step was not required since I am expecting a typed SOAP Fault exception object within my Scope – Exception handler.

BizTalk 2010: Calling Dynamics CRM 4.0 Web Services

2011-03-07T21:29:00.001-08:00

The project that I am on ran into a critical and somewhat embarrassing situation this past week. As we moved from a single node project environment into a multi-node test environment we discovered that the Dynamics 4.0 Adapter cannot be installed on multiple BizTalk Servers in the same group. Yes – you read that correctly. I have never heard of this kind of limitation and struggle with the idea the adapter was actually made available without this feature. You can read more about this limitation in the FAQ section here.

With this in mind, we needed to quickly make a 180 degree turn. I had heard of other people calling the ASMX CRM Web Services from BizTalk via WCF Adapter. The process of consuming these ASMX services is much like consuming any other Web Service. That is until you start your application and get the dreaded 401 Unauthorized error.

System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'Negotiate,NTLM'. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized

This seems like a pretty common error within the BizTalk and CRM forums. However, I was not able find any resolutions to the issue. It wasn’t until I started stepping through a .NET SDK sample that I realized that I was missing something and that was a required SOAP header. For your convenience I have built an end to end sample that will walk through creating a Lead within a Dynamics 4.0 system. Keep an eye open for the Message Assignment shape where I assign this header.

Once we have created a BizTalk project we need to “Add Generated Items” and then select “Consume WCF Service”

Even though Dynamics 4.0 still uses ASMX style Web Services we can select “Metadata Exchange (MEX) endpoint.

Enter the location of your CRM Web Service i.e. http://server/MSCrmServices/2007/CrmService.asmx and click on the “Get” button.

Once the WSDL definition has been loaded, click the “Next” button to continue.
Click “Import” to create your schemas and sample Orchestration.
You should find that the following artifacts have been added to your BizTalk Solution.

Our “main” schema is called CrmService_schemas_microsoft_com_crm_2007_WebServices.xsd. Within this schema you will find all of the available Actions (Create/Update/Delete/Fetch etc) and the related entities (both custom and out of box). In this scenario we are going to create a “Lead” within our CRM system. So we can find the “Create” node and then scroll down and find the “Lead” entity.

To keep things simple, I have created a little helper message that I will use in a Map to create an instance of this CRM Create message.

Inside my map, I will simply map from this helper message to the actual CRM Request.

With the building blocks in place, the next step was to update the Orchestration that was generated for us. Pretty standard stuff going on here. We want to receive an instance of our “helper” message, transform it, send the request to CRM and then write the response to disk.

When we generated our CRM schemas, BizTalk also created a Port Type called “CrmServiceSoap” that contains all of the available operations including “Create”.

Earlier in this post I mentioned that we need to set a SOAP header on the CRM Request message. In order to do this we want to add a Message Assignment shape after our Transformation shape.
Within this shape we want to set the following:

msgCreateCustomer (WCF.OutboundCustomHeaders) = "<headers><CrmAuthenticationToken xmlns=\"http://schemas.microsoft.com/crm/2007/WebServices\"><AuthenticationType xmlns=\"http://schemas.microsoft.com/crm/2007/CoreTypes\">0</AuthenticationType><OrganizationName xmlns=\"http://schemas.microsoft.com/crm/2007/CoreTypes\">YourOrganization</OrganizationName><CallerId xmlns=\"http://schemas.microsoft.com/crm/2007/CoreTypes\">00000000-0000-0000-0000-000000000000</CallerId></CrmAuthenticationToken></headers>";

The two important settings are the “AuthenticationType” and the “OrganizationName” properties. The Authentication property can take one of three values:
- 0 – AD
- 1 – Passport
- 2 – Internet Facing Deployment
Since this is an on-premise installation we will go with 0.
The “OrganizationName” property becomes extremely important in multi-tenant configurations where you may have multiple organizations or departments sharing a CRM implementation. Since we only have 1 organization in our deployment it is safe to just set our default value.
It is now time to deploy and configure the BizTalk application.
When BizTalk generated our CRM schemas and sample Orchestration, it also generated a Binding file. When we import the binding file we will discover that two send ports have been created. The first is a basicHttpBinding and the second send port uses a Custom Adapter with a Custom binding. What is the difference? Really it comes down to some addition security configuration that is available with the Custom adapter.

Using these binding as is didn’t work for me. I needed to change the WCF-Custom Adapter to use the basicHttpBinding. I also had to make a few Security related changes including setting the Security “mode” was set to “TransportCredentialOnly”

I also needed to set the “clientCredentialType” to “Windows”

I wasn’t quite done here. I also need to add an Endpoint Behavior. In this case it was a “clientCredentials” behavior. Within this behavior, I set “Windows” - “allowedImpersonation” = “Impersonation”

At this point, I am ready to configure the rest of the application and start it. Once started, I can process my sample helper message.
Within my sample file, I have provided the following information. Once processed by BizTalk, I expect this Lead to be created within our CRM system and the CRM response to be send to disk.

The response from CRM is a GUID.

Once we launch the Dynamics CRM GUI, I am able to find my Barry Sanders record.

Conclusion

If you have high-availability requirements then I highly recommend abandoning the Dynamics CRM 4.0 adapter. Currently, and I am not sure if there are any plans, CRM 2011 does not have an adapter so now is as good as time as any to make the move. You can read more about CRM 2011 integration on Richard Seroter’s blog. Also look for a chapter on CRM 2011 integration in our upcoming book.

New BizTalk 2010 book unveiled – Line of Business Systems Integration

2011-02-22T12:04:00.001-08:00

Received news this morning that the book that I have been working on for the past 7 months has now reached “RAW” status. What is RAW you ask? RAW stands for “Read As we Write”. The idea is to get the book, in soft copy format, into the hands of readers early. The goal is to get early feedback from readers with the possibility of incorporating this feedback before the book is published. This is a relatively new program from Packt but when it was presented to us we decided to get the book into the community sooner than later. I must caution that the book, in this format, is not the final version. Think of it as a CTP (Community Technical Preview) with the polished version of the book set for the end of April timeframe.

What is the book about?

The book primarily deals with Line of Business integration with popular LOB systems like SAP, Dynamics CRM, SalesForce.com, Dynamics AX, SQL Server and SharePoint. In addition to these chapters we also introduce the Line of Business Adapters Fundamentals and connecting to the AppFabric Service bus.

Why write another BizTalk book?

As experienced developers we understand the nuts and bolts of BizTalk. We can create Schemas, Ports, Maps and understand various BizTalk patterns. The challenge is often understanding the systems that we are integrating with. When integrating with commercial products like SAP and Dynamics there is always some pre-requisite knowledge required to effectively integrate with it. The goal of this book is to expose some of this pre-requisite knowledge so that an experienced BizTalk developer can get up to speed quickly with the technology that they need to integrate with.

Who are the authors?

I have yet to meet a person who was an expert in each of these technology areas. With this in mind a talented team of BizTalk specialists was assembled to take on this adventure including:

Richard Seroter

Sergei Moukhnitski

Thiago Almeida

Carl Darski

Myself

Technical Reviewers

Anyone who has written a book can appreciate the “unsung heroes” of a published book. The unsung heroes of this book are our Packt publishing team and Technical Reviewers. We have an outstanding roster that includes the following people:

Packt

Kerry George

Alina Lewis

Zainab Bagasrawala

Technical Reviewers

Mick Badran

Stephen W. Thomas

Ben Cline

I can say, without any hesitation, that their efforts have significantly increased the quality of the book.

For more details on the RAW program, I encourage you to visit the PACKT website.

Stay tuned for more information as the actual published date nears.

Off Topic–Payback is a ….

2011-01-02T10:19:00.001-08:00

So you may recall a bet that took place around the 2010 Olympic timeframe which had me, and Team Canada, matched up against fellow BizTalk MVP Mikael Håkansson and Team Sweden. As we all know, Canada won the Olympic Gold Medal and I won the bet.

The Christmas break is well anticipated holiday for many obvious reasons in Canada. While most people in Canada are stuck indoors for the coldest part of the year, many are glued to their television sets watching the World Junior Hockey tournament. This tournament represents the best under 20 year old Hockey players on the the planet. Most of these players will end up playing professional Hockey somewhere and many will make the NHL.

On December 31st, 2010, Canada squared off against Team Sweden in a round robin game. Once again Mikael taunted me with a blog post so I couldn’t resist opening up that wound of his from earlier in the year.

This time around it is my time to eat crow. Sweden beat Canada 6-5 in a shoot-out and in return here is my tribute to Team Sweden and Mikael.

There are still some games left and I am confident that Team Canada can pull it off. There is a good chance that Canada could play Sweden so perhaps there will be a rematch bet?

Microsoft MVP Awarded: Part 4

2011-01-01T09:46:00.001-08:00

I recently received word that I have been re-awarded for another year. Contributing to the BizTalk community continues to be something I enjoy and am passionate about so it is always nice to be recognized by Microsoft for my efforts.

Looking back at 2010, it sure was a busy year. I spent a lot of time in the areas of SharePoint and SAP integration. I attended TechEd in New Orleans where I also helped out at the BizTalk booth. I gave a SharePoint and App Fabric Service Bus presentation to the Calgary .Net User group and had the opportunity to go to Sweden and speak about Monitoring your BizTalk environment with SCOM. Travelling to Stockholm and spending time with Mikael Håkansson and Johan Hedberg was one of my highlights even if Mikael signed both Johan and I up for an Olympic distance triathlon.

2011 is shaping up to being just as busy if not busier than 2010. Here are some of the things coming up for me:

I am currently the Integration Architect on a Dynamics CRM/Project Server 2010/SharePoint 2010/SAP project so I will continue to be very involved with integrating line of business systems. Expect more SAP and Dynamics CRM content on this blog. I am thinking about building a series around CRM 4.0 integration much like the ShareTalk series that I previously published on this site.
Speaking of Line of Business systems, the rumor is true. I have teamed up with a group of really talented authors to write a BizTalk book. The book will be unlike any of the current BizTalk books and will target experienced BizTalk developers. The list of authors and technical reviewers is impressive, I am very confident that the end product will be great. More information will be forthcoming, but you can expect the book to be available towards the end of April 2011.
Learning more about AppFabric is also on my “resolution” list for 2011. While I have a very good idea of what AppFabric is and what you can do with it, I just haven’t had a lot of “hands on” experience with it.
While unrelated to this blog, I just love my Windows Phone 7 device and have previously written the “BizTweet” app for it. I look to expand my skills in this area as well. I have a couple ideas that I plan on pursuing, one that does involve BizTalk and SAP.

While no where near Seroter numbers, this blog received just over 40 000 hits over the past year, so I thank you for taking the time to stop by and read my content. All the best for the upcoming year.

Off Topic: Introducing BizTweet for Windows Phone 7

2010-11-18T21:48:00.001-08:00

In addition to blogging about BizTalk, I have decided to start blogging about Windows Phone 7. Therefore, I have created a new blog to focus on WP7 and will continue to blog about BizTalk here.

I recently had my first WP7 App published in the marketplace. The app is called BizTweet and may be found in the Social category. The purpose of the application is to aggregate information related to BizTalk and AppFabric. So if you have a WP7 and are interested in BizTalk and/or AppFabric then I encourage you to check it out.

For more details, check out the BizTweet blog post.

BizTalk: HTTP Adapter and preventing Cached Responses

2010-10-04T14:50:00.001-07:00

A while back I blogged about a service that would go fetch Natural Gas Prices, the price of Oil and Stock Quotes from Yahoo. The information that is returned from BizTalk is surfaced in an Xcelsius dashboard along with a lot of other business critical data from SAP. Our executive team accesses this information from a web part in SharePoint site. As people launch their browsers, they see the Stock Quotes and other commodity prices get updated. Since this is a dashboard, people will view the data for a few minutes and then close their browser. This type of user behavior never uncovered a flaw in the application. It is not like someone sat on the dashboard all day long waiting for the stock price to change.

A request came in to turn this Dashboard into a Windows 7 widget. Once this widget was in place, we uncovered that the stock quotes were not being updated. The Widget simply acts as a container for the dashboard. So we dug out Fiddler and could determine that the BizTalk service was not being called on a regular interval. The reason? Caching. There was no cache command or expiration date sent on the HTTP header going back to the dashboard so it would not be called on a regular basis. Since this Widget does not get restarted like a Web Browser does, the stock quotes would remain static for the duration of a user’s desktop session.

To avoid this situation, I needed to provide an explicit command in a Message Assignment Shape to prevent my responses from being cached:

msgStockQuoteResponse(HTTP.UserHttpHeaders) ="Cache-Control: no-cache";

Since the client was instructed not to cache the response, it would now go ahead and call the service when it goes to refresh the rest of its data.

There are many options that you can set within the HTTP Header. For example, if you wanted to expire content every 2 minutes, you could set your header to Cache-Control: max-age=120. If you are interested in what other features can be set in an HTTP Header, I recommend checking out this site.

BizTalk 2010 – RTM has arrived!

2010-09-26T08:39:00.001-07:00

As you are probably aware by now, BizTalk 2010 has RTM’d. You can catch the announcement here. I am not going to recap all of the new features as you can read about them here. However, here are a few points worth noting about the new release.

The BizTalk 2010 Developer’s edition is now free. So if you are curious about BizTalk and getting a licensed copy has been a barrier for you in the past, now is your chance to test drive BizTalk at no cost. Full details and download may be found here.
For those AppFabric/WCF/WF developers, there is a new compelling feature called BizTalk AppFabric Connect which allows you to leverage BizTalk components such as the BizTalk LOB Adapter Pack and BizTalk Mapper within your .Net projects. This is a true enabler for connecting your WF applications to Line of Business Systems (LOB)
If you have written applications in the past that have used the traditional line of business adapters, you will now be forced to use the BizTalk Adapter pack when communicating with SQL Server, SAP, Oracle and Siebel. Microsoft had deprecated these adapters in BizTalk 2009 and they have been officially removed in 2010. For the past year or so, I have been using the BizTalk Adapter pack when communicating with LOB systems and there are some subtle differences between the legacy and new LOB adapters. Overall, I am satisfied with the experience and there are no “show stoppers” that we have run into when making the switch. So if you are planning a BizTalk 2010 upgrade and currently use the legacy adapters, ensure migrating these applications to use the BizTalk adapter pack is in your project plan as you will need to regenerate schemas using these new adapters.

BizTalk 2009 + SharePoint 2010

2010-09-15T09:37:00.001-07:00

Note: The following steps do not represent a supportable configuration by Microsoft and extreme caution should be used when following these steps. To my knowledge, these steps are complete, but it is possible that I missed something. These steps have been tested in a Send Port Subscription scenario. Use these instructions at your own risk.

At the organization that I work for, we are rolling out SharePoint 2010. We currently run BizTalk 2009 and at this point in time, the BizTalk 2009 SharePoint Adapter Web Service isn’t supported with SharePoint 2010. BizTalk 2010 has been updated to support SharePoint 2010. We really have no choice but to head in this direction since upgrading to BizTalk 2010 just isn’t an option at this point in time.

When you launch the BizTalk 2009 installation on a SharePoint 2010 Server you will not be able to install the Windows SharePoint Services Adapter Web Service as it is not able to detect the pre-requisite Windows SharePoint Services v 3.0 files.

Mick Badran has a post that helped me out tremendously. However, I found that I had to take a few additional steps so I will recap exactly what we did in order for us to get this to function in our environment.

We had an existing MOSS 2007 server that had the BizTalk Adapter Web Service installed and functioning correctly. Since you cannot use the BizTalk installation wizard to install this Web Service, we decided to reverse engineer what was deployed on MOSS 2007 and manually deploy on SharePoint 2010.

Copy the c:\program files\Microsoft BizTalk 2009 folder from your MOSS 2007 Server to your SharePoint 2010 Server
Create the BTSharePointAdapterWS Web Application in IIS Manager and set the physical path to C:\Program Files (x86)\Microsoft BizTalk Server 2009\Business Activity Services\BTSharePointV3AdapterWS

Note: This path represents a 64 bit installation.

We leveraged the existing App Pool that the SharePoint Site was using
Modify the web.config to set the appropriate SharePoint Enabled Hosts, if required. For us, we use AD Groups and had to change to reflect our “DEV” AD group
Rebind the Microsoft.SharePoint.dll to point to version 14.

We removed the Document Tag so that we could browse the WSDL properly. See this post for more info.

We then exported two keys from the Registry on our MOSS 2007 Server and import them on your SharePoint 2010 Server:
- [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server]
- [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\BizTalk Server]
At this point we thought we were done and attempted to push a message from BizTalk 2009 to SharePoint 2010 but discovered the following message in event viewer:

Event Type:    Warning
Event Source:    BizTalk Server 2009
Event Category:    (1)
Event ID:    5743
Date:        9/10/2010
Time:        1:37:51 PM
User:        N/A
Computer:    SERVER

Description:
The adapter failed to transmit message going to send port "SendPort1" with URL "wss://SERVER:80/DropOffLibrary". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.BizTalk.KwTpm.OfficeImporters2, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.BizTalk.KwTpm.OfficeImporters2, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
at Microsoft.BizTalk.KwTpm.WssAdapter.BTSharePointAdapterWS..ctor()

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

--- End of inner exception stack trace ---

This error was triggered by the Windows SharePoint Services receive location or send port with URI wss://SERVER:80/DropOffLibrary.

Windows SharePoint Services adapter event ID: 12462".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Once again it was back to the MOSS 2007 server and more specifically, a trip to the GAC where we discovered the following BizTalk Dlls. This list included the Microsoft.BizTalk.KwTpm.OfficeImporters2.dll that was mentioned in the event viewer. We then pulled these dlls off of the installation CD and gac’d them on the SharePoint 2010 server using gacutil.

After Gac’ing these assemblies we had success pushing messages from BizTalk 2009 to SharePoint 2010.

Note: There is another option which I have not tried that Kirk Hofer discusses here. It involves installing WSS v3.0 SP2, but not configuring it, so that the BizTalk install Wizard thinks it is installed. Our SharePoint team was not crazy about having these other dlls on their fresh SharePoint 2010 server so we were unable to proceed with this approach.

Managing your BizTalk Environment using SCOM

2010-08-27T02:05:00.000-07:00

I recently had the opportunity to speak at the Swedish BizTalk user group in Stockholm. It was an excellent opportunity to present to one of the top BizTalk user groups worldwide. My presentation consisted of slide ware, a tour of the SCOM 2007 R2 console and several BizTalk/SCOM demos using BizTalk 2010 beta and the BizTalk 2010 Management Pack beta. You may find the presentation below.

BizTalk integration with SharePoint and the Cloud Presentation

2010-06-22T19:42:00.001-07:00

I recently had the opportunity to speak at the Calgary .Net User Group. My presentation included slideware and demonstrations involving integrating BizTalk with SharePoint 2010 and the AppFabric Service bus. You can view the presentation below:

BizTalk 2010 Beta – Where are the AppFabric Service Bus bindings?

2010-06-19T12:45:00.001-07:00

After installing the Windows Azure platform AppFabric SDK V1.0 - April Update I was expecting all of the AppFabric Service Bus relay bindings to be available inside of the BizTalk Admin Console but they were not. I could run the AppFabric Echo .Net Sample without any issues. After re-installing both the AppFabric SDK and BizTalk 2010 Beta I was still unable to find these bindings in the Admin console. I checked the machine.config files (both 32 bit and 64bit) and both files were updated. The problem with this is that I checked the .Net 2.0 config files.

After stumbling across Wade Wegner’s blog, the picture became much clearer. Wade explains “I soon realized that the issue was the machine.config file. When you install the Azure AppFabric SDK the relevant WCF extensions are added to the .NET Framework 2.0 machine.config file, which is shared by .NET Framework 3.0 and 3.5. However, .NET Framework 4.0 has its own machine.config file, and the SDK will not update the WCF extensions.”

To resolve the issue you need to run a utility called: RelayConfigurationInstaller.exe which can be found in C:\Program Files (x86)\Windows Azure platform AppFabric SDK\V1.0\Assemblies (64 bit machine) or C:\Program Files\Windows Azure platform AppFabric SDK\V1.0\Assemblies (32 bit machine) and provide an input file that includes

<?xml version ="1.0"?>
<configuration>
<startup>
<requiredRuntime safemode="true"
imageVersion="v4.0.30319"
version="v4.0.30319"/>
</startup>
</configuration>

The name of this file should be called RelayConfigurationInstaller.exe.config.

example:

C:\Program Files (x86)\Windows Azure platform AppFabric SDK\V1.0\Assemblies>RelayConfigurationInstaller.exe /i

After this command was run successfully, I could now see the “Cloud Bindings” including the NetTcpRelayBinding

Presenting at Calgary .Net User Group - June 22, 2010

2010-06-16T16:23:00.000-07:00

Topic: BizTalk integration with SharePoint and the Cloud
Speaker: Kent Weare
Date: 22-June-2010
Location: Nexen Conference Center
801-7th Ave. S.W., Calgary, AB. (Plus 15 level)
Map
Registration: 5:00 pm - 5:30 pm
Presentation: 5:30 pm

http://dotnetcalgary.com/

Abstract
Part 1 - Discover how you can leverage BizTalk to integrate with SharePoint and InfoPath

In this session, Kent will demonstrate some of the ways that you can integrate with SharePoint using the BizTalk WSS Adapter and Web Services. Kent will also discuss some real world scenarios where this technology has been used to provide value to the business.

Part 2 - Discover how you can leverage BizTalk to integrate with the AppFabric Service Bus

Learn about Microsoft's cloud based Service Bus and how you can leverage BizTalk to communicate with external applications and services. Kent will demonstrate BizTalk's ability to expose on premise services to the cloud and consume AppFabric Service Bus end points. Kent will also discuss some scenarios where it may be appropriate to leverage these integration patterns in the real world.

TechEd 2010 – Day 4

2010-06-11T07:42:00.001-07:00

TechEd 2010 North America is done for another year. I thought the conference went really well and people were full of enthusiasm. A lot of Product Groups have shipped, or are shipping products this year. There were over 11 300 in attendance for this event eager to hear about the latest bits in the pipeline . I was at TechEd in LA last year and can tell you there wasn’t near that many people in attendance.

BizTalk and the Cloud

I saw a really interesting session where BizTalk was brokering a connection through the Service Bus into a SAP and a Mainframe system. A Web app was hosted in Azure and BizTalk was listening for requests via the AppFabric Service Bus. It was probably the riskiest demo I saw, but Chris and Elizabeth pulled it off without a hitch. I have been thinking about these types of scenarios quite a bit since I have done a fair about of integration with SAP in the past. SAP certainly cannot connect into the Azure Service Bus at this time nor do I expect them to be able to do that any time soon. Mainframe systems are no different since they most likely will not have a transport mechanism capable of communicating with the Microsoft cloud. Many businesses leverage SAP to run their core business, having BizTalk capable of connecting into the cloud provides an additional layer of abstraction and security when accessing SAP data.

More Cloud

There was certainly no shortage of Cloud talk at this conference. The next session I saw was called Appfabric: Extend .Net technologies to the Cloud. It was presented by Aaron Skonnard and Keith Brown. You can tell these guys are very comfortable on the presenter’s stage. It was probably best presentation that I have seen in a long time.

Some of the topics discussed include:

You can think of Azure AppFabric as an extension of the .Net framework
No Azure Workflow as a Service in this release
Relay bindings allow your application access into the Service Bus
Direct connections allow you to communicate peer to peer with other clients in order to improve performance and reduce charges
Eventing pub/sub can be used in multi-cast situations
To increase interoperability use Http Relay Bindings

View from the BizTalk Booth

A had a few conversations with attendees who were using other middleware products such as Oracle or IBM and wanted to know how BizTalk stacked up in terms of feature set. None of the features that they were requesting were absent from the BizTalk stack. They also wanted to see how easy the Microsoft tools are to use and the impression that I got is that they felt they were on par with what they were use to seeing.
Another attendee wanted to know whether they should be using WF or BizTalk. Quite frankly this scenario can still get a little blurry for me. We talked about things like low latency, durable messaging, flat file requirements, Adapters and management of the solutions.

Next Year

Next year TechEd will be held in Atlanta from May 16th-19th. There is currently an early bird special where you can save $500 for registering early. http://northamerica.msteched.com/?fbid=qjus7wvTc2O

TechEd 2010 – Day 3

2010-06-10T06:45:00.001-07:00

More BizTalk

We took a closer look at the Dashboard Settings in the BizTalk Admin Console. There are 3 main categories that you can configure:

Group Configurations
Host Configurations
Host Instance Configurations

Probably the most compelling setting allows you to control the polling interval when connecting to the MessageBox for new messages. You even have the ability to configure these settings based upon a Messaging or Orchestration scenario at the Host level. In the demo, we saw a simple Receive and Respond scenario. The default configuration setting is to have BizTalk connect to the MessageBox every 500 ms. This message took .68 seconds to be processed end to end. The Polling interval was then changed to 10 ms and the message took .09 seconds to process. Some pretty dramatic improvements. The end result is that the BizTalk Host Instances are not waiting as long for their next polling interval so messages will be processed more frequently. These actions are not without some risk though. The more frequently you poll, the more stress you put on you SQL Server database so mileage will vary.

The portability of the Dashboard settings were also demonstrated. You now have the ability to export and import your configuration settings across environments so you are not manually tweaking on several servers. You also have the ability to script the import and export of these settings

The new BizTalk Management Pack for SCOM was demonstrated. They have rebuilt this MP from the ground up and now include 2 separate views:

Application
Deployment

The Application view focuses on artifacts that would typically be addressed by BizTalk developers. So if you have a message that failed in a pipeline or a retrying sent port then you would find them in this view. More infrastructure related items would be found in the Deployment view. So if you had a database or host instance offline, you will find these types of alerts in the Deployment view.

Unfortunately, the new SCOM pack will only work with BizTalk 2010 so if you are still on BizTalk 2009 or previous, you cannot use this MP. The good news is that the MP will ship with BizTalk 2010 so there will be no gap between those two products and their release dates.

Other bits

The support for SQL 2005 and Windows 2003 is gone for BizTalk 2010
BizTalk 2009 (and earlier) will not be certified for Windows Server 2008 R2.
Business Activity Monitoring (BAM) Portal will support 64 bit app pools - Not exactly true, please see comment by Rahul for further clarification
BizTalk Admin Console is still not a native 64bit app. You can run in WOW mode on 64 bit machine though

BizTalk Business to Business integration

I later sat in on a B2B presentation that discussed EDI and Trading Partner Management. I have not had a lot of exposure to these 2 features of BizTalk so it was interesting to get a look under the hood. The TPM management is greatly improved and is more intuitive to use as you can now group Business Units under a Parent entity.

They also discussed the BizTalk Mapper since EDI schemas tend to be large and complex. Leveraging new features like Search, Indicative Match, copy and pasting functiods, highlighting selected nodes and links goes along way.

The enhanced FTP adapter was discussed and here are some of the core features:

Support for Read Only Receive scenarios
FTPS protocol is now supported
Improved performance when connecting to AIX systems
Atomic Sends for Text files

Communications Server 14

Office Communications Server (OCS) has been rebranded to just Communications Server or “CS 14”. I am not that familiar(at a deep technical level) with OCS and related technologies but since we do use it I thought I would check out the Q and A session that deals with interoperability.

I think it is pretty safe to say that this industry is in a state of transition. You have traditional telecom companies like Cisco and Avaya who are trying to hold onto their footprint and then you have Microsoft who is very hungry in this vertical. The frustrating part is that these companies have so much at stake that they are not collaborating much (from what I can see) and that leaves customers who have a blend of this technology frustrated as it is tough to develop roadmaps with so much changing. It may be a good idea to sit on the sidelines for a year to see how this all shakes out.

View from the BizTalk Booth

More great conversations at the BizTalk booth:

An Infrastructure Architect wanted to know what impact BizTalk has on his environment since his organization is planning on implementing BizTalk
Another attendee is looking and managing the provisioning of Datacenter assets through BizTalk. He wanted BizTalk to host services that would expose functions like “deploy Windows Server”, “Deploy SQL Server”, “Deploy Application ‘x’ to Server ‘y’”. It was a very interesting discussion about how they wanted to provision systems to their private cloud.
More questions about using the BizTalk Mapper activity in WF
There genuinely seems to be interest in BizTalk. Perhaps people are just being polite, but a lot of people are talking about introducing BizTalk to their environment(s).

TechEd 2010 – Day 2

2010-06-09T07:03:00.001-07:00

I opened up day two with a session involving Connecting to Line Of Business (LOB) systems in 15 minutes

We saw:

A demonstration of the BizTalk mapper activity in WF
Many questions from the audience regarding the licensing model. Some people questioned that if you had to buy a BizTalk Standard license to use the Mapper and LOB adapters then why wouldn’t you just use BizTalk. The response was that there may be some scenarios where you still want to avoid the BizTalk message box but want to connect to LOB systems like SAP without having to write a lot of code to transform data between the two systems. The BizTalk Mapper will easily pay for itself over time as it allows you to convert data through a GUI tool instead of writing a lot of code.
You will not have to install all of BizTalk in order to use these two new activities in WF. Microsoft plans on providing a “AppFabric Connect” feature that allows you to just install the BizTalk bits that you need
Licensing for these components will be based on the same model that BizTalk standard edition uses so you will be paying per proc(and not by the core)
Oracle adapter exposed as WCF service that SharePoint BCS (Business Connectivity Services) consumed...no code is required.
Connecting to IBM lob systems in a similar manner. Use the BizTalk HIS components outside of BizTalk and connect to legacy IBM systems without BizTalk.
I got the sense that a lot of .Net developers were excited to see these capabilities but not so excited that they needed to pay for BizTalk licenses in order to use these bits.

Another interesting session I saw involved extracting SAP data so that it can be used by Microsoft BI tools. The tool is called Simplement Data Liberator. We heard from one of their customers who uses a blend of SAP and Microsoft tools. They explained some of their Roadblocks to “True BI”;

SAP tables difficult to understand
SAP tables difficult to extract
Need too many experts
Explicit skill set required to use a SAP BI tool
Needed real time access to SAP data
Needed SharePoint integration
Better value with Microsoft Consultants
Wanted wider talent pool

SAP has over 50 000 tables so trying to extract data manually or through Custom code is a daunting experience. The Simplement Data Liberator tool has solved this challenge. The Tool replicates SAP data into SQL Server. You can get both German and English Meta data and SAP Security info is also replicated. This tool eliminates the need to write ABAP for reporting in SAP. Surface SAP data into Microsoft BI products by using SQL queries.

Other features:

Work with only tables you need
Load into SQL server via replication
Manipulate/Transform data such as dates (ETL)
Attached meta data so that you can use English Table names
Use SharePoint, Excel services or SQL Server Reporting Services (SSRS) to query data

We saw a demo where a customer was updated in the SAP GUI system and within seconds (3-5) the data was in the SQL BI system. I must say that I was impressed by the demo, it seems like this vendor understands SAP very well and has figured out a way to lower customer costs when consuming SAP data.

View from the BizTalk Booth

More great questions and conversations about BizTalk:

An excellent conversation about End to End Message Security. You always want to trust your developers to do the right thing, but how do you prevent a developer from adding their own receive location, or subscription in BizTalk for malicious purposes; to prevent a rogue developer from depositing money into their Bank Account via BizTalk. If anyone one has thoughts on this type of scenario and have figured out a way to solve it I would be interested in hearing about it – use comments below.
Whether BizTalk is truly pub/sub due to the polling nature of the Host instance connecting to BizTalk MessageBox looking for more work. This attendee was looking for more Event Driven processes inside BizTalk.
A DBA was interested to know how BizTalk will connect to their Database and some of the “gotchas” to look out for
A few people that closed the door on BizTalk after BizTalk 2004 that are now back taking another look at BizTalk 2010. I can’t say this enough, the product has matured significantly over the past and it is worth taking another look at
Connecting BizTalk to the AppFabric Service bus in order to connect with other external parties

TechEd 2010 – Day 1

2010-06-07T15:25:00.001-07:00

Below are some of my ad hoc notes from TechEd 2010 in New Orleans.

The Atmosphere

Much more encouraging than last year, in my opinion. The numbers seem to be up and the convention centre is very busy even though the facility is huge. Microsoft has rented over 1 mile worth of space to host the event. At a meeting that I attended on Sunday, a Microsoft rep indicated that registrations had exceeded their expectations this year. They attributed this to improvements in the North American economy.

Keynote

A great deal of the keynote was dedicated to cloud computing. The phrase that has been floating around Microsoft circles for the last little while is “we’re all in” meaning Microsoft is betting big time on the emergence of cloud computing.

Some items that were discussed include new tools available for Developers and IT Pros to monitor cloud applications including:

Intelitrace for Azure
SCOM Management packs for monitoring Web Roles and other artifacts in the cloud
SQL Management Studio integration for managing SQL Azure databases

Microsoft discussed a client, Chicago Tribune, and their shift from moving On-Premise assets to the Cloud. Some of the key drivers for this move includes:

Reduction in the amount of Data centers required to run their operations
Pay as you go Scalability. As you can imagine there would be some very newsworthy days that would stress a media outlet’s IT infrastructure
Reliability and Uptime. If your site is down, people will just go to your competitor’s site.

Other announcements

Windows 7 SP1 and Windows 2008 R2 SP1 will enter Beta in July (2010)
Windows 2008 R2 SP1 will include enhancements such as Dynamic Memory
Office Communication Server 14 is schedule for Q4 – 2010 Key Features include

Tighter SharePoint and Exchange integration
Picture displays in the OCS Client as pulled from SharePoint
Pictures include a presence border (Green/Yellow/Red)

Microsoft has over 74 000 Enterprise voice users internally and saves $1 million per month by using OCS and Live Meeting for long distance and conference calls
Use OCS to search for Skills within your organization
720p video supported with OCS 14
Exchange pushes polices down onto Windows Mobile 7 phones
Windows Mobile 7 supports multiple exchange accounts including on-premise and in the cloud
Windows Mobile 7 easily separates your work and you life within one phone
Edit documents from SharePoint using your Windows Mobile phone

Other Sessions

BizTalk 2010 and VNext

Customer wants include better alignment with Platform releases (SQL/Windows/VS)
Session was full – BizTalk interest is alive and well
Enhanced Mapper exists in 2010 that allows for

Functoid “Move to page”
Reduction of noise by focusing on specific areas
Node Search
Undo
New Ribbon at the top of Mapper – Ribbons are in-style

Mapper and LOB Activities are available within WF Workflows

A license will be required – potentially a BizTalk Standard license
Potentially, BizTalk Dev edition may become a free version

Enhanced BizTalk Management Pack for SCOM will be included

Better terminology and modeling used
Better support for multi-node BizTalk deployments
Adoption of SCOM still low for BizTalk deployments (around 10-20% of the room uses SCOM – non-scientific survey)

Settings Dashboard available in 2010

Allows for more granular control over performance settings
Some Settings remain Global, some Host specific and some Host Instance/Machine specific
Settings are scriptable via WMI

Upgrade

Full support from BizTalk 2006 R2 and BizTalk 2009

New FTP Adapter

FTPS protocol is supported
Support for read only scenarios
Improved performance when connecting to AIX systems

VNext

Plans still vague but looking at more Cloud integration – perhaps we will see BizTalk in the cloud or Transformations as a service in the cloud

Building Communication enabled Business Processes with Microsoft Communications Server 14

This session contained some really cool demos and makes me wish I had more spare time. I saw some demos where a CRM application was extended to include OCS integration. You may be saying what is the big deal you can do this today? What was really neat was that they were able to push the CRM context into OCS so that when you called/IM’d the recipient their OCS window would be expanded to include context about what you were contacting them about. Imagine you needed to speak to “Bob” about customer “XYZ’s” payment. Wouldn’t it be nice to include the data that you are looking at as part of the conversation so that Bob does not have to log into the CRM? This is now possible by using the Microsoft Communications Server 14 API. The API seemed very intuitive and includes XAML components that allow you to drag and drop these components onto your WPF/Silverlight canvas. I see plenty of opportunities of this type of functionality in Contact Centre or CRM type applications.

View from the BizTalk Booth

This year I am spending some time helping out at the BizTalk Booth. I thought it would be interesting to post some of the questions or trends that I am seeing while visiting with other attendees.

Many people still don’t know what BizTalk is/does. Once you explain it to them, they usually provide a response similar to “I have a broken down process that is hard to manage and is unreliable, can I use BizTalk in that scenario”? The answer in many cases is Yes.
A great question came up regarding BizTalk and SAP Enterprise Services with respect to performance
Some people expressed frustrations over previous versions specifically some of the management tools in BizTalk 2004. For those that have been there before and have moved on, I encourage you to come back and take a look at what BizTalk 2010 has to offer. The tooling, tracking and management of applications has greatly improved.
Great questions around how do you manage your BizTalk environments? The answer is SCOM, but you need to invest some time so that you do not create a SPAM engine. I wouldn’t want to manage a BizTalk environment without a tool like SCOM as it provides great visibility into the Health of your BizTalk application. Tune, tune and then tune again to find the right balance.

ShareTalk Integration – WSS Adapter Web Service Request format is unrecognized

2010-05-16T10:04:00.001-07:00

When you install the SharePoint Adapter Web Service for BizTalk, and browse to the BTSharePointAdapterWS.asmx Web Service, you will be prompted with an error that makes it seem like your installation of this Web Service was not successful:

Server Error in '/BTSharePointAdapterWS' Application.

Request format is unrecognized.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.InvalidOperationException: Request format is unrecognized

In fact your Web Service may be functioning correctly as mine was. The issue is that the Service metadata is disabled by default. To enable the Service metadata, you will need to update the web.config of the BTSharePointAdapterWS Application

Comment out, or remove the <remove name="Documentation"/> line in order to enable displaying the Service metadata.

Metadata will now be displayed correctly.

Microsoft Canadian Leadership Summit Day #2

2010-05-14T15:47:00.001-07:00

On the 2nd and final day of the Canadian Leadership Summit, I focused on Dynamics xRM and Windows XP to Windows 7 Migration.

xRM/CRM

In the context of “xRM”, the ‘x’ stands for “anything” meaning if you have a requirements to track relationships then Dynamics may be a good fit.

Relationships with many different entities

Employees
Customers
Vendors

Dynamics CRM Global Customer list

ING
Nortel
LG
Volvo
Sara Lee
Minnesota Twins
Vancouver Canucks
City of London
Pfizer
Barclays
Tourism Whistler

Dynamics xRM Deployments

Over 22 000 customers
1.1 million users
4000 worldwide partners
50 worldwide hosting partners
Dynamics CRM runs on top of the xRM platform and is available in the Cloud (hosted by Microsoft) or On Premise
Experience is delivered through PC, Browser or Phone

Rapid Deployment

The session contained a video that included a complete Sports team management application built in less than 24 hours on the xRM platform by 4 Microsoft employees during the World Partner Conference in the summer of July 2009. Surveys were posted on Twitter asking for ideas and requirements as to what people would like to see built. The deliverable included a semi-detailed design and fully functional system within the allotted time. The team then presented the solution to the attendees of the conference. I have found a LiveMeeting of this event here. Use Meeting ID 8B74CC to view the content.

CRM Key Capabilities

Point and Click Application Development and Deployment
Out of the box user experiences (Outlook, Office, Mobile, Offline)
Leverages Microsoft Application Platform Technologies
- SQL
- Windows Server
- SRS/SAS
- WF
- .Net

CRM 5 – The Road Ahead

Here are some of the themes that Microsoft is focusing for the next version of the software:

Simplicity

Insightful
Role-Tailored
Collaborative

Innovation

Social
Connected
Extensible

Value

Cloud and On-Premise
Solution Partners
Smooth Upgrade

SharePoint and XRM are Complementary

SharePoint and XRM have overlapping capabilities. SharePoint has evolved beyond just storing Word documents. It has become a platform that is capable of performing some tasks that a CRM is capable of . On the flipside CRM, has some lightweight document management features. The end result is that the two platforms work better, together. If you owned both sets of technology, it makes sense to leverage both solutions to build the total package.

SharePoint better with unstructured data
xRM is better for structured Data

The next portion of the session involved Christian Gosselin, the CRM Practice Leader for CGI in Quebec, Canada. CGI is a very experienced CRM implementer with over 50 Certified professionals on platform.

Christian, discussed some of the Business and IT Challenges that make CRM a viable option:

Typical Organizational Challenges

Rapid Innovation is a key mandate
Waiting for IT to deliver requested Applications
Business builds shadow applications(MS Access) to survive
Poor productivity due to many different user interfaces and tools
Lack of intelligence/insight

IT Challenges

Limited budget and resources
Trying to support all requests coming from business
Option of building or buying (both have big negatives in cost, time and scope)
Supporting hundreds of legacy applications
Difficult to deliver quick wins

xRM as a platform

xRM was built from a platform perspective and then added the application as opposed to some other vendors building the CRM application and then duct taping the platform around it in order to make it extensible.

xRM is a complete business application platform:

Data and Metadata Services
User experience services
Access and Security Services
- Active Directory
Analytical Services
Extensibility Services
Integration Services
- User Interface – Mash-ups
- Web Services
- Plug-ins
- Data Integration
Architecture Services
- Deploy once
- Currency Conversion
- Multi-language support
Platform Management Services
- System Centre Operations Manage
Process Automation and Orchestration Services
- WF
- BizTalk
Point and Click Customization Services

Related Microsoft Technologies

Outlook
Office
SQL Server
.Net

One Platform, many Applications

Organizations use xRM for a variety of applications including:

Project Management
Recruitment Management
- Professional Sports
Property Management
Employee Management
Asset Management
Supplier Management
Program Management
Customer Management

Benefits of building on Dynamics xRM

Drive Efficient and Effective IT
Develop the right business applications
Deliver Rapid Innovation and Agility
- Don’t have to rebuild the plumbing
Demonstrate Business Success
- Let IT become the enabler and not the bottleneck

Real Life Projects

Christian then described some of the projects that his organization has been involved in:

Financial Services Client
- Centralized and shared application accessible across Canada in English and French
- Stronger, more profitable customer relationships
- More Competitive advisor recruitment
- Better regulatory compliance
- New business insights
Government Client
- Needed to communicate with other Government entities
- Not one line of code was written
  - Customizations were accomplished through configuration
- Better collaboration between Ministry and other agencies
- Higher quality of information
- Reduced manual data entry
- Simplified communications
  - Ability to follow up on communications is more rapid
- Increased business agility
Pharmaceutical Client
- Tracking of drug/medical agents
- Implementation was 9 weeks
- More time spent on business requirements than on time to implement
- Centralized application accessible from all of the the world in multiple languages
- Minimal end user training
- Ability to easily generate custom reports with Excel
- Ability to quickly create new LOB applications

Dynamics CRM as a business application platform generally saves 50-70% of the development time as opposed to custom development

Windows XP to Windows 7 – A roadmap for success

The next area that I focused on was migrating from Windows XP to Windows 7. A key component to a Windows 7 migration is the Windows Deployment kit wizard. We saw a 7 year old upgrade his Windows XP system, with data, to Windows 7 using this wizard. I should note that his father works for Microsoft in the Windows division so I suspect he had a little help.

When configuring the Windows Deployment kit, a wizard will walk you through the various options including:

Choose Migration type
- Refresh this computer
  - save information and install new OS
- Upgrade this computer
Specify computer name
Join domain or workgroup
Specify where to save your data and settings
- Can specify network location
  - Local Disk
  - Network
Specify language and locals
Time Zone
Specify BitLocker configuration
Branding

Advice

Refresh existing hardware – don’t have to wait for hardware replacement
Make the process fast and non-disruptive
Test wisely, but don’t expect things to break as often as before
That means fewer helpdesk calls

Microsoft’s Environment overview

Microsoft has offices in 105 countries
89 000 employees globally
70 000 venders globally
220 windows deployment servers and product servers globally on a virtual server infrastructure
User is supported by 5 call centers globally

How did it go?

No new hardware purchases beyond 3 year refresh cycle
Very few compatibility issues - mostly with Internet Explorer 8
Single image for each platform(x64 and x86) for entire globe
Helpdesk staff was not increased
Deployment numbers
- Beta > 60 K in 3 months
- RC >90K in 3 months
- RTM > 100k in 3 months

How was it possible?

User excitement
- easier to achieve these days because of user satisfaction
Executive sponsorship and local goals
Vastly improved toolset
- Task sequence automation
- Hard-link data migration
- Multi-cast and multi-stream deployment model

Early Adopters of Windows 7

T-Mobile
Dell
EDS
Avanade
Halliburton
Sheraton
Del Monte
StatollHydro
WiPro
Samsung
BMW
City of Miami

Recommended Deployment Tools

Microsoft Deployment Toolkit(MDT) Lite Touch and/or
System Center Configuration Manager (SCCM) 2007 Zero Touch
Automates what would be manual processes

Microsoft Canadian Leadership Summit Day #1

2010-05-13T09:11:00.001-07:00

I recently had an opportunity to attend the Microsoft Canadian Leadership Summit in Redmond and thought I would share some of my findings. The Leadership Summit allows Microsoft Canada to invite key customers and partners to discuss technology trends that are relevant to the Canadian business climate. Below is a recap of some ad hoc notes that I made through-out the presentations.

Why use Microsoft products?

Standardization allows you to reduce costs
Flexibility is required to change directions as the economy demands it
Microsoft technologies allow you to manage costs and have agility
Microsoft provides building blocks that allows you to fill the gaps between legacy applications, 3rd party and core Microsoft offerings no matter whether they are on-premise or in the cloud.

Pressures continue to Intensify

New generation of workers
Security and Compliance continues to be a challenge as more and more integration occurs
Mobile and Distributed Workforces are increasing in numbers
Innovate without additional resources

Consumerization of IT

We use to go to work to have a more advanced IT Experience

Fast network download speeds
Stable email environments

Now people at home have more IT capabilities at home than in the office

Video conferencing
IM
Facebook
Wireless Access points

Customers can use Microsoft products to provide a balance between improved collaboration while implementing security and management of those services. Some examples of these technologies include the Social networking features in SharePoint 2010 and Office Communication Server (OCS).

The Cloud

Decision to move to the cloud should be a business decision and not a technology decision
Need to trust in enterprise-class services that Microsoft provides
Delivering consistent connected experience across devices (PC, Mobile phone, Laptop, Consumer devices)
Microsoft has had success with a lot of large “house hold” companies moving their assets to the Azure cloud. Announcements will be made shortly
Understanding Canadian Federal and Provincial privacy laws continue to create challenges

Low Cost Computing

More companies are able to leverage low cost platforms for mission critical applications. Windows Server is an enterprise ready OS that is much more affordable than other enterprise offerings. Migrating your traditional AIX/Oracle environments is feasible , and achievable , on the Windows Platform
Technology exists to maximize the efficiency of your IT infrastructure
Take advantage of new ways to acquire and deliver technology

Microsoft's Commitment to Customers

Over 1 Billion active Windows Users
500 Million Office users
$9.5 Billion invested in R&D
Long Term, tenacious and Partner-Centric

Business climates are changing faster than they have before

Every minute on YouTube, 24 hours of video is uploaded by users
Starbucks – Office Online
- Growth through stores
- More cross selling and up selling
- needed to extend technology into the store level
- Microsoft Online Services - Email and SharePoint Team sites available to 19000 employees in-store to allow employees to collaborate on how to grow the company$

Deep investments in infrastructure

$2.3 Billion invested in cloud infrastructure
Geo-replicated data to multiple data centres
public and private cloud flexibility
30 000 engineers working on cloud services

Microsoft’s Commitment to services excellence

Financially backed uptime guarantees
SAS 70 and ISO 27001 compliant
carbon footprint innovation
Rapid innovation model

Office 2010 launch

Some key features:

Ignore conversation – will remove current user from all future Distribution Lists emails and place related emails into the deleted folder
Pictures of email recipient, from “My Site” show up in the bottom of compose email message
- click on the picture and you see additional context from Facebook, LinkedIn and SharePoint
SharePoint 2010 Search –
- refined search and meta data
  - filter by author, date and other meta data
- Specify content in links i.e. PowerPoint
- Preview slides before opening document
- People Search
PowerPoint over the web
- high fidelity without the need of Live Meeting
SharePoint - My Site
- status updates, similar to Facebook
- tagging competencies allows you to easily search and identify an expert in the related competency
- Post on “Note board’ – much like Facebook’s “wall”
- Enriched Org chart including pictures and metadata
  - surfaced using Silver Light
- Improved Blogging capabilities
- Tags and Notes
  - tag pretty much anything in SharePoint – blogs, documents which create a Tag Cloud
  - follow tags via newsfeed
- Offline capabilities – use SharePoint workspace
SharePoint 2010 wide scale adoption – early adopters include:
- Global Crossing
- Sony
- Associated Press
- Six Flags
- Telus
- Constellation Energy
- Energizer
8.6 Million Beta Downloads
80 % Beta users encouraged others to download
450 Customers in early adoption program
Edmonton Oilers (Professional Hockey) use SharePoint to collaborate on potentials players by aligning video with meta data and statistics
- Player Dashboard
- Current prospects
- Player video page
Oiler organization feels that SharePoint 2010 is very intuitive and straight forward to use
- leverage messaging and collaboration features
1000 Microsoft Partners are developing SharePoint 2010 solutions
20 000 Microsoft Partners partners trained
500 000 developers are using SharePoint

Microsoft Online Services

Why Customers are embracing the cloud:

Reduce CAPEX & OPEX
Predictable & Pay per use subscriptions
Focus on Market differentiation
Energy Efficiency
Latest versions with no server upgrades
Choice, Flexibility and Resource Agility
Merger, Acquisition and Divestitures
Up and Running quickly

Interesting quote from a Gartner reference":

“By 2012, 20% of businesses will own no IT Assets”. Quite frankly I have a hard time believing this statement.

Microsoft Azure data canters include: Chicago, San Antonio, Northern Virgina, Dublin, Amsterdam, Singapore

No plans for Canada to have a data centre

Concerns about privacy and laws due to provincial jurisdiction

Customers moving to cloud

McDonalds
Rexel
Staples
Eddie Bauer
Phillips
GlaxoSmithKline
Blockbuster

Microsoft Online Services

40 million Online paid seats
more than 5oo government agencies
Over 50% of the fortune 500
9000 business customers

BizTalk Adapter Pack 2.0 – Part 4 Sending XML IDocs to SAP

2010-04-18T08:59:00.001-07:00

When I started this series, I was more focused on the challenges that are involved with receiving XML IDocs and the schema versions that are tightly coupled to the version of SAP that you generated them on. In Post 1 we discussed how the SAP Adapter will use the DOCREL version that is provided by SAP and line this up with your schema forcing you to be in sync with SAP Upgrades. I wanted to better understand the implications on the Send side so you will find some of my notes below. The story is a little better on the Send side when sending XML IDocs.

When generating your XML IDoc schemas ensure that the RecieveIdoc Format = Typed.

Since BizTalk will be sending IDocs to SAP, you want to select “Client (Outbound operations)” and ensure that your schema has a name of “Send”.

You still want to be aware of the version of SAP that you will be connecting to but it is not as important as when you are choosing to Receive an XML IDoc. You are best off asking your BASIS admin what version you are on.

My initial test scenario was generating a ZCONF32 – 700 version of my IDoc. The version of SAP that I am connecting to is 700 so everything should work correctly.
As expected when I wired up my Receive Port to my Send Port, I am able to submit an IDoc to SAP successfully. Something that I was interested in knowing was the namespace that was included in the context properties.

Something also to note is that when you import the Binding file, that was generated when you created your IDoc Schemas, is the SOAP Action header configured in your send port. You will notice that the Action includes the namespace for the Schema that was generated.

My next scenario included downloading schemas for version 710 of the IDoc(knowing that I am connecting to an SAP version that is set to version 700). I updated my map to include this version of the schema. All of the “connection lines” in the mapper updated successfully which makes me think that there are no structural differences between these two versions – at least not with the data elements that I was trying to map. I redeployed the application and left the existing send port in tact. I was prompted with a warning/error in the event viewer:

Event Type:    Warning
Event Source:    BizTalk Server 2009
Event Category:    (1)
Event ID:    5743
Date:        4/18/2010
Time:        8:50:45 AM
User:        N/A
Computer:    BizTalkServer
Description:
The adapter failed to transmit message going to send port "SendSAPIDoc" with URL "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX?RfcSdkTrace=False&AbapDebug=False". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.Xml.XmlException: Start element 'Send' from namespace 'http://Microsoft.LobServices.Sap/2007/03/Idoc/3/CONF32/ZCONF32/700/Send' expected. Found element 'ns0:Send' from namespace 'http://Microsoft.LobServices.Sap/2007/03/Idoc/3/CONF32/ZCONF32/710/Send'.

Server stack trace:
   at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
   at System.ServiceModel.Channels.ServiceChannel.EndRequest(IAsyncResult result)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at System.ServiceModel.Channels.IRequestChannel.EndRequest(IAsyncResult result)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.RequestCallback(IAsyncResult result)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This error leads me to believe that the “magic” when sending XML IDocs is in the SOAP Action Header in the Send Port. Upon updating the SOAP Action Header to reference the 710 version of the IDoc, I had success.

I checked the WE02 SAP Transaction and found that my IDoc was processed successfully as indicated by the green light.

Conclusion

What I am able to conclude from this exercise is that you are not bound by the version of DOCREL like you are when Receiving IDocs but rather by the structure of the IDoc itself. I am pretty confident that you can update your SAP version without having to regenerate your schemas as long as the structure of the IDoc itself does not change. This puts you in a similar risk category as using flat file schemas which I will discuss in the next post of this series. Something that became evident is that the SOAP action header must match the version of the schema that you have generated.

BizTalk Adapter Pack 2.0 – Part 3 Receiving multiple Flat File IDocs using SAP Adapter

2010-04-11T14:44:00.001-07:00

In Part 2 of this series we discussed how you can receive IDocs from SAP in flat file format in order to avoid tight coupling with the version of SAP you are running as discussed in Post 1.

As I alluded to in the end of the Part 2 post, there is an issue when you include more than 1 IDoc schema in a Receive Pipeline that will be use the same Program ID and Receive location.

The behavior that I experienced when working through this scenario is the first message in my pipeline would be processed successfully but the second type(Vacations) would not.

When I tried to process my 2nd type of IDoc (Employee Vacations) I would get the following error:

Event Type:    Error
Event Source:    BizTalk Server 2009
Event Category:    (1)
Event ID:    5719
Date:        4/11/2010
Time:        10:10:49 AM
User:        N/A
Computer:    *BizTalk Server*

Description:
There was a failure executing the receive pipeline: "SAPVersions.ReceiveSAPIDocs, SAPVersions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=147585b889447deb" Source: "Flat file disassembler" Receive Port: "ReceiveIDocsFromSAP" URI: "sap://CLIENT=XX;LANG=EN;@a/SAPSERVER/XX?ListenerGwServ=sapgwXX&ListenerGwHost=SAPSERVERci&ListenerProgramId=*ProgramID*&RfcSdkTrace=False&AbapDebug=False" Reason: Unexpected data found while looking for:
'Z2HR_WS_SEG000'
The current definition being parsed is idocData. The stream offset where the error occured is 520. The line number where the error occurred is 2. The column where the error occured is 0.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

In the rest of the post I will discuss how you can fix this problem manually and then discuss the Hot Fix from Microsoft that addresses this situation. I recommend proceeding with the Hot Fix route as it is a more sustainable solution. The manual steps include modifying a schema that was generated by the Consume Adapter Service Wizard. The risk is you can manually modify one of these schemas only to have a colleague regenerate the schema and overwrite your manual change. If you do proceed with the Hot Fix route you will want to ensure any other developers that will be generating IDoc schemas are also using this Hot Fix. I also want to highlight that this Hot Fix should be applied to developer’s workstations as this fix addresses a design time issue.

Fixing issue manually

Open up your schema that contains the “meat” of your IDoc information in an XML Editor view. Whenever you generate SAP IDoc schemas you are bound to have multiple XSD schemas created. You can expect 1 IDoc that includes the various complex types, another that acts as the “Request” container IDoc, a Response xsd and the “meat” xsd which will include most of the “core” details of the schema.
Scroll down until you find a section that includes tag_name=”EDI_DC40” tag_offset="0" and replace it with tag_name="ZHR_VAC" tag_offset="39" where “ZHR_WS” is the name of your IDoc type.

What you will find as you open your schemas is that all of them will have this tag_name="EDI_DC40" tag_offset="0" string included in the IDoc which prevents BizTalk from distinguishing one IDoc from another.
Once you have modified all IDocs in this fashion, redeploy and you should find that both(in my case) IDocs were processed successfully.

Fixing issue with HotFix

In KB977528 Microsoft describes the issue as “This problem occurs because the first Flat file disassembler pipeline component in the custom pipeline tries to disassemble all the IDOCs that are received. However, the IDOCs should be processed by different Flat file disassembler components that have different schemas”.

When using the HotFix, you download schemas the same way as you previously did

Note when you open up the schema, you will now notice that the same change that we applied manually has now been incorporated into the schema definition automatically.

If you have changed the name of you XSD files, make sure you update your receive pipeline, deploy and restart your host instance(s). You should now be able to receive both types of flat file IDocs without issue

Note:

My developer workstation is a 64 bit machine so when I navigated to the Hot Fix Web site it detected that my system was a 64 bit system and provided me with a 64 bit patch. The problem is that Visual Studio runs in 32 bit mode and since this fix is a design time issue my issue was not resolved until I downloaded the 32 bit version of this KB. I logged onto a 32 bit machine and navigate to the same site in order to get the 32 bit fix.

Thus far we have been focusing on receiving IDocs. A reader has asked if I can demonstrate how to sent a flat file schema to SAP using the BizTalk Adapter Pack so that will be the next post in this series.

BizTalk Adapter Pack – SAP Binding ReceiveTimeout

2010-03-14T22:11:00.001-07:00

One of the most important configurations to make inside a SAP Receive location is the “receiveTimeout” property. The documentation describes this property as:

“Specifies the WCF message receive timeout. Essentially, this means the maximum amount of time the adapter waits for an inbound message. The default is 10 minutes.

Important

For inbound operations such as receiving IDOCs, we recommend setting the timeout to the maximum possible value, which is 24.20:31:23.6470000 (24 days). When using the adapter with BizTalk Server, setting the timeout to a large value does not impact the functionality of the adapter.”

The default value(10 minutes), is too short of a period for BizTalk to stop listening for messages from SAP. For instance, if your organization has a weekly back up scheduled for SAP, it will likely take longer than 10 minutes. The documentation indicates that there is no impact to using a large number so I am not sure why the default is only 10 minutes.

Just to give a practical example of what you can expect when SAP goes down (either planned or unplanned) from a BizTalk perspective.

SAP going down

Event Type: Warning

Event Source: BizTalk Server 2009

Event Category: (1)

Event ID: 5740

Date: 3/13/2XXX

Time: 10:01:31 PM

User: N/A

Computer: SERVER

Description:

The adapter "WCF-Custom" raised an error message. Details "The WCF service host at address sap://CLIENT=XXX;LANG=EN;@a/SAPServer/00?ListenerGwServ=SAPGateWay00&ListenerGwHost=SAPServer&ListenerProgramId=IDOC&RfcSdkTrace=False&AbapDebug=False has faulted and as a result no more messages can be received on the corresponding receive location. To fix the issue, BizTalk Server will automatically attempt to restart the service host.".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning

Event Source: BizTalk Server 2009

Event Category: (1)

Event ID: 5740

Date: 3/13/2XXX

Time: 10:01:31 PM

User: N/A

Computer: SERVER

Description:

The adapter "WCF-Custom" raised an error message. Details "Microsoft.Adapters.SAP.RFCException: Details: ErrorCode=RFC_FAILURE. AdapterErrorMessage=An exception has occurred on the listener while executing RfcWaitForRequest..

at Microsoft.ServiceModel.Channels.Common.Design.AdapterAsyncResult.End()

at Microsoft.ServiceModel.Channels.Common.Channels.AdapterReplyChannel.EndTryReceiveRequest(IAsyncResult result, RequestContext& requestContext)

at Microsoft.Adapters.Internal.LayeredChannelBindingElement.LayeredInboundChannel`1.System.ServiceModel.Channels.IReplyChannel.EndTryReceiveRequest(IAsyncResult result, RequestContext& context)

at System.ServiceModel.Dispatcher.ReplyChannelBinder.EndTryReceive(IAsyncResult result, RequestContext& requestContext)

at System.ServiceModel.Dispatcher.ErrorHandlingReceiver.EndTryReceive(IAsyncResult result, RequestContext& requestContext)".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning

Event Source: BizTalk Server 2009

Event Category: (1)

Event ID: 5740

Date: 3/13/2XXX

Time: 10:01:32 PM

User: N/A

Computer: SERVER

Description:

The adapter "WCF-Custom" raised an error message. Details "The faulted WCF service host at address sap://CLIENT=XXX;LANG=EN;@a/SAPServer/00?ListenerGwServ=SAPGateWay00&ListenerGwHost=SAPServer&ListenerProgramId=IDOC&RfcSdkTrace=False&AbapDebug=False could not be restarted, and as a result no messages can be received on the corresponding receive location. BizTalk Server will continue trying to start the service host until it succeeds or the receive location is disabled.

To fix the problem, you may choose to:

1. Use the error information given to fix the problem.

2. Restart the receive location.

3. Keep waiting for BizTalk to recycle the service host. Another event will notify if the service host is successfully started.

SAP coming back up

Event Type: Information

Event Source: BizTalk Server 2009

Event Category: (1)

Event ID: 8112

Date: 3/13/2XXX

Time: 11:03:13 PM

User: N/A

Computer: SERVER

Description:

The WCF service host at address sap://CLIENT=XXX;LANG=EN;@a/SAPServer/00?ListenerGwServ=SAPGateWay00&ListenerGwHost=SAPServer&ListenerProgramId=IDOC&RfcSdkTrace=False&AbapDebug=False was successfully restarted, therefore the associated receive location can now receive messages.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

You are now able to receive messages from SAP without any intervention. Had we not set an appropriate ReceiveTimeout, the receive location would have still be down when SAP tried to push this IDoc to our system.

You never want your middleware to be the cause of an outage so it is important to understand when your dependant servers/services have scheduled maintenance so that you can ensure that your application can sustain this downtime. Configuring the ReceiveTimeout to an appropriate value will reduce the chance that BizTalk will be down due to someone else's maintenance window.

BizTalk Adapter Pack 2.0 – SAP Adapter IDoc Schema Versioning Part 2

2010-03-14T21:40:00.001-07:00

In my first post, I discussed how the BizTalk Adapter pack Consume Adapter Service wizard tightly couples the IDoc schema version with the version of SAP that you used to generate the schema with. The goal of this post is to discover how you can avoid this tight coupling and hopefully survive an SAP upgrade. I know the organization that I work for would not be pleased with having to update all of our IDoc schemas when we do our next SAP Upgrade just because the version of SAP has been incremented.

When the BizTalk Adapter Pack came out I was pretty excited that I could throw away my Flat File schemas and pipelines. While true, it comes at a cost. When using the XML Receive pipeline with the new Adapter, the Adapter will use data in the SAP Control record to determine what version of the message you retrieving from SAP and will assign a namespace to your message, that contains SAP version information, before depositing it in the message box. If have not changed your SAP version since generating your IDocs you are in business. If you have upgraded your SAP system or have generated the wrong schemas for your SAP version then you will be in a lot of pain.

To get around this tight coupling issue, you can revert back to the old style of IDoc processing which involves flat files and pipelines. Now there is nothing really wrong with this model, but it does require a few extra steps and is not as “clean” a solution as using the XML Receive pipeline.

I will now take you through how you generate a flat file schema that can be used to process different versions of the same IDoc. When I say different schemas this means that the version, and therefore namespace, are different but structurally the documents are the same. Note that I will not go through all steps required to generate an IDoc schema, only those that are important to generate a Flat File schema. To see a more comprehensive walk through of generating an IDoc schema, please see my Webcast.

When using Consume Adapter Service wizard, ensure that “GenerateFlatFileCompatible” is set to true.

When you set the GenerateFlatFileCompatible property to true, you will now see the flat file specific data within the Schema specification.

Without setting GenerateFlatFileCompatible , the schema will not contain this flat file information.

You also want to indicate the “ReceiveIDocFormat” will be a String. You will find out why this is important later on in this post.

Once the schemas have been imported into Visual Studio, enable “Flat File Extensions” to the “core schema”. When using the BizTalk Adapter Pack Consume Adapter Service wizard you will generate more than 1 schema unlike the old .Net Connector version where you would always only generate 1 schema. By “core schema” I mean the one that contains the “meat” of your schema, not one that contains the shell(all of the imported schemas) or any of the base type schemas.

You will now need to create a Receive Pipeline and add a Flat File disassembler

Add your “shell” schema to the document schema property

Build and Deploy your application
When configuring your SAP Receive location you will want to specify your Receive Pipeline as opposed to XML Receive

On the Binding tab, set the “ReceiveIdocFormat” to String

On the Messages tab, provide an XPath expression that will extract the flat file data so it can be disassembled by the Receive Pipeline. The XPath expression that I used is:
/*[local-name()='ReceiveIdoc']/*[local-name()='idocData']

The adapter is essentially wrapping this flat file in XML tags so that it can make it through the adapter stack as an XML message.

Also ensure the “Node encoding” is set to String as the default is “XML”

Start your application and you are good to go

Up until this point, I have not modified the namespace of my schema and everything worked well as expected

However, the purpose of this post is to demonstrate how we can break away from the default namespaces that are generated by the adapter.

Since I have redeployed the application, the message subscription have also been updated:

Start your application up and it should function with this new namespace and therefore subscription.

The next test is to generate a different version of an SAP IDoc and ensure receiving a different version of the IDoc does not break our application. If you recall from Post 1 we received an error in this situation. From the Consume Adapter Service wizard, I am deliberately going to generate a schema that has a DOCREL greater than my current SAP version.

My subscription is updated and is looking for a “710” IDoc where as SAP is going to generate a
“700” IDoc

I received, processed and delivered the message downstream without issue

Even though SAP, populated the Control record with DOCREL = 700, I am able to process this message because this DOCREL is not being used to derive the target namespace like it is when you use the XML Receive pipeline. This is really where the value of the Flat file pipeline comes in as it will use the namespace in the schema that is configured inside the pipeline instead of having the Adapter generate a namespace automatically. This also works as the structure of the 700 IDoc has not changed in the 710 IDoc. If the structure of the IDoc changes with the version number then you have no other option than to regenerate your schemas.

Gotcha!

So everything is fine and dandy until you try to deploy another IDoc with the same Program Id/Partner Profile. There is bug with the BizTalk Adapter Pack that prevents you from deploying multiple IDoc Flat File schemas with the same Program ID. Look for a workaround with my next post in this series. I should be able to turn that post around quicker than this one.

Off Topic: Canada Wins....Mikael loses

2010-03-01T07:44:00.000-08:00

During this past MVP summit there was an abundance of trash talk regarding Olympic Hockey. Mikael Hakanssons and I made a Blog bet with the loser of the bet dedicating a post to the winning country. As I am sure you have heard, Canada beat USA in overtime 3-2 on Sunday. I will give Mikael credit for being a good sport even though he jumped on the USA bandwagon as soon as Sweden was eliminated. You can find his post here.

Consuming a BizTalk hosted WCF Service with InfoPath 2007

2010-02-27T09:34:00.001-08:00

I recently built a POC that wrapped an SAP RFC call in a WCF Service exposed by BizTalk. I exposed schemas as WCF services instead of opting for exposing the Orchestration as a WCF Service. The purpose of the Service was to validate a utility meter’s status inside of SAP. Some of the statuses may include active/inactive/meter not found. The RFC that I used is called RFC_Read_Table. This is a pretty generic, out of box, RFC that allows you to query SAP tables. The RFC is fairly complex due to a lack of good documentation available. Perhaps I am just not part of the secret SAP consultant’s club where you can actually get useful documentation. I plan on documenting my findings in an upcoming post so that other people do not have to endure the pain I went through in order to get it working.

One of the benefits of InfoPath is the ability to consume a web service (asmx). I was pleasantly surprised that InfoPath can also consume some WCF endpoints(.svc). The endpoint type that I exposed was based on the basicHttpbinding which happens to be one of the more interoperable bindings. At this point, the SOAP adapter has been identified as an Adapter that will be removed in an upcoming release of BizTalk. The idea behind this is that people should be moving to the WCF adapter. There are some gaps between the SOAP and WCF adapter, which is outside the scope of this post, but my advice would be if you can use the WCF Adapter then you should. If you cannot, due to a limitation, then you should provide this feedback to Microsoft so they can either enhance the WCF adapter or hold off on removing the SOAP adapter.

The InfoPath service consumption wizard is fairly straight forward with the exception of “how the form works with the Web Service”. The 3 options that you do have include:

Receive and submit data
Submit data
Receive data

When I think of a client consuming a Web Service, I think in terms of the client providing a Request message and the Server providing a Response. With this in mind, I decided to use the “Submit data” option. What I ran into is that I could submit data to my BizTalk Service, BizTalk would process the request and provide a response but nothing was being updated in my form. I discovered that I needed to use the “Receive and submit data” option instead. The description is counter intuitive as it implies I am receiving data first then I am submitting it. Perhaps if Microsoft would have called this option “Submit and receive data” this would have been a little clearer.

I have recently been teased by a few colleagues and friends about the amount of detail and screen shots that I have put in my blog posts. At the risk of more torture, here is a step by step guide of how I got this all working.

Design a Form Template

Select “Web Service'” and click OK

If you want to submit data to a service and use the response data in your form, select “Receive and submit data'”

Provide the destination of your WSDL and click “Next”

The name of my operation is “ValidateMeter”, click “Next”

Thus far, the information that has been provided is related to the Web Service response so I have provided a name for this data connection that reflects this.

Since I am submitting data to this same service, I need to provide the WSDL URL again.

Once again I need to provide the operation that I will be communicating with.

The “ValidateMeter” operation has a single input parameter. I need to specify which field from the request schema will be sent to the Web Service. To do this I want to click on the following button:

Select the field from Schema

Click “Next” to proceed

I now need to provide a name for this data connection

At this point I have a blank canvas and I also have the ability to start dragging and dropping fields from my Web Service Request and Response schemas onto the canvas.

I have expanded, the “queryFields” node and have dragged the “MeterNumber” element onto the canvas. If you recall the value that will be populated in this field will be sent as part of the Request message to the ValidateMeter Service.

When BizTalk provides the response from SAP, I want to display this value on the form. In order to do this, I need to expand the “dataFields” node and select “MeterStatus” from the Web Service Response schema.

To test my form, I can click on the “Preview” button

I can now provide a Meter Number and click the “Run Query” button. Within a couple seconds I will see my Meter Status text get populated. In this particular instance, the Meter Number 123456 does not exist so the Meter Status text box is updated to reflects this.

BizTalk 2009: BizTalk Mapper - get value from last node

2010-02-26T15:10:00.001-08:00

I ran into a situation where I needed to get the last value from a re-occurring node that SAP was returning.

In order to extract the last record, I utilized the Record Count functoid and fed that value into the Index functoid. The Index functoid is also fed with the value from the "WA" element that I am really interested in. The result is then passed to a String Concatenate functoid where the data is enriched before being assigned to the destination document.

BizTalk Adapter Pack 2.0 – SAP Adapter IDoc Schema Versioning Part 1

2010-01-19T22:13:00.001-08:00

A colleague and I have been working through some anomalies when it comes to SAP IDoc schemas that were generated using the BizTalk Adapter Pack 2.0 SAP Adapter. This is the adapter which is based upon WCF and NOT the .Net Connector based adapter.

When you go to generate an IDoc using the Consume Adapter Service wizard in Visual Studio, you will be presented with a screen like this. You will most likely have a few options when selecting an IDoc and version to use. To be sure that you have selected the right version of IDoc, you should contact your BASIS admin.

As you select different versions of IDocs, notice that the Node ID changes. This Node ID will soon become your Target Namespace in the XSD schema that you are about to generate.

This Target Namespace is very important since BizTalk uses the Target Namespace and root node to determine subscriptions.

If you have chosen the wrong version of IDoc you will soon know. You will receive a subscription error indicating that you have received a message that BizTalk does not have a Schema for.

Event Type: Error

Event Source: BizTalk Server 2009

Event Category: BizTalk Server 2009

Event ID: 5719

Date: 1/17/2010

Time: 1:54:31 PM

User: N/A

Computer: Server

Description:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "WcfReceivePort_SAPBinding_IdocZHR_KPV3R710_Custom" URI: "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX?ListenerGwServ=SAPGWXX&ListenerGwHost=SAPSEVER& ListenerProgramId=XXXXXX&RfcSdkTrace=False&AbapDebug=False" Reason: Finding the document specification by message type "http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive" failed. Verify the schema deployed properly.

If you refer to the first image in this blog post, you will see that the Node Id has a value of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//710/Receive#Receive where as BizTalk has received an IDoc with a Namespace of http://Microsoft.LobServices.Sap/2007/03/Idoc/3/ZHR_KP//700/Receive#Receive. So how can this be? SAP by default generates an IDoc based upon the current release version. But you also have the ability to send a specific IDoc version as configured in the SAP Program ID. This provides some flexibility in the event you need to use a legacy IDoc version.

When SAP sends an IDoc out, it will populate this version information in the control record. I have turned on tracking on the receive port and as you can see this information has been populated by SAP.

Since I am using an XML Receive pipeline and SAP has no idea what namespace BizTalk is requiring, the SAP Adapter is generating this namespace at run-time based upon the DOCREL and IDOC_VERSION elements.

The lesson here is to make sure you know what version of the IDoc is currently configured with your Program Id within SAP. I must also caution that while this approach does work, it may not live through your next SAP upgrade. At least not without updating your BizTalk projects. With each SAP major release, you can expect this DOCREL value and potentially the IDOC_Version to increase. So the next time you upgrade your SAP environment, you may get into the same situation that I have described(subscription errors) and it may take some significant refactoring to get your BizTalk schemas and maps back in sync with your SAP version.

In the next post of this series, I will discuss how you can use a Receive Pipeline and Flat File Pipeline Disassembler to override this namespace so that it does not become dependant upon the DOCREL and IDOC_VERSION elements…stay tuned.

ShareTalk Integration (SharePoint/BizTalk) – Part 10 Adding Metadata to SharePoint columns in a messaging only solution

2010-01-16T14:14:00.001-08:00

I had a requirement where I needed to upload images to a SharePoint library and use the information from the image filename to populate columns within the document library. Performing these tasks within an orchestration is pretty trivial as you can set context properties in a Message Assignment shape that will drive the behavior of the WSS Adapter. I couldn’t justify performing these tasks in an Orchestration since it would involve an extra hop to the MessageBox in order for Orchestration to be invoked.

Another option is to do all of this work in a Pipeline component. I can create a Send Port subscription that would allow me to access the context properties from the message received and update them to include the WSS properties all within in a pipeline. This would allow the solution to become a pure messaging solution and I can save an extra Message Box hop. When building this solution, I referenced Saravana Kumar’s white paper on Pipeline Components. It came in handy, especially in the area of creating the Design time pipeline properties.

Within the Pipeline Component, the first thing that I wanted to to was retrieve the source file name. I am able to do this by reading the “ReceivedFileName” property from the File Adapter’s context properties. I then wanted to clean this file name up since the value of this property includes the entire UNC path: \\Servery\RootFolder\SubFolder\filename.jpg. I have written some utility methods to parse the file name from this long string.

There are 3 parts to the image file name(1213455_NEW_20100110120000.jpg) that I am particularly interested in. They include an Asset Number, Asset State and the Date/Time that the image was taken. The scenario itself is a field worker who needs to capture an image of an asset, indicate the Asset Number and its state i.e. New/Old. This information is then captured in the name of the image. Since it is an image, there is no other reasonable way to store this meta data outside the file name. This in itself is the reason why I need to to use a pipeline component. Standard WSS adapter functionality includes the ability to use XPATH statements to extract data from the message payload and populate a SharePoint document library column.

Once I have captured this meta data and massaged it to my liking, I want to then provide this context data to the WSS Adapter. The WSS adapter is a little different than most of the other adapters in that you can populate an XML document and push that into the ConfigPropertiesXml context property. The document structure itself is a flat XML structure that uses a “key-value” convention.

<ConfigPropertiesXml><PropertyName1>Column 1 Name</PropertyName1><PropertySource1>Column 1 Value</PropertySource1><PropertyName2>Column 2 Name</PropertyName2><PropertySource2>Column 2 Value</PropertySource2><PropertyName3>Column 3 Name</PropertyName3><PropertySource3>Column 3 Value</PropertySource3></ConfigPropertiesXml>

I also want to populate the WSS Adapter’s Filename context property. I can achieve this by the following statement:

pInMsg.Context.Write("Filename",
"http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", ImageFileName);

Below is my Execute method, in my pipeline component, where all of this processing takes place. You can download the entire sample here. This code is at a proof of concept stage so you will want to evaluate your own error handling requirements. Use at your own risk.

public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
{
//Get received file name by retrieving it from Context
string FilePath = pInMsg.Context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties") as string;

//strip path from filename
string ImageFileName = GetFileNameFromPath(FilePath);

    //Utility methods to parse filename
    string msgAssetNumber = GetAssetNumberFromFileName (ImageFileName);
    string msgAssetState = GetAssetStateFromFileName(ImageFileName);
    string msgImageDateTime = GetImageDateFromFileName(ImageFileName);

    //Write desired file name to context of WSS Adapter
    pInMsg.Context.Write("Filename",
        "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", ImageFileName);

    // Populate Document Library Columns with values from file name
    string strWSSConfigPropertiesXml = string.Format("<ConfigPropertiesXml><PropertyName1>{0}</PropertyName1><PropertySource1>{1}</PropertySource1>" +
        "<PropertyName2>{2}</PropertyName2><PropertySource2>{3}</PropertySource2><PropertyName3>{4}</PropertyName3><PropertySource3>{5}</PropertySource3></ConfigPropertiesXml>",
        this.AssetNumber,msgAssetNumber,this.AssetState,msgAssetState,this.ImageDateTime,msgImageDateTime);

pInMsg.Context.Write("ConfigPropertiesXml", "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", strWSSConfigPropertiesXml);
return pInMsg;
}

A feature that I wanted to provide is the ability to provide the SharePoint column names at configuration time. I didn’t want to have to compile code if the SharePoint team wanted to change a column name. So this is driven from the pipeline configuration editor. The values that you provide (on the right hand side) will set the column names in the ConfigPropertiesXml property that is established at run time.

If you provide a value in this configuration that does not correspond to a column in SharePoint, you will get a warning/error on the Send Port.

Event Type:    Warning
Event Source:    BizTalk Server 2009
Event Category:    (1)
Event ID:    5743
Date:        1/10/2010
Time:        7:38:19 PM
User:        N/A
Computer:
Description:
The adapter failed to transmit message going to send port "SendDocToSharePoint" with URL "wss://SERVER/sites/BizTalk%%20Repository/Inbound%%20Documents". It will be retransmitted after the retry interval specified for this Send Port. Details:"The Windows SharePoint Services adapter Web service encountered an error accessing column "Missing Column" in document library http://SERVER/sites/BizTalk%%20Repository/Inbound%%20Documents. The column does not exist. The following error was encountered: "Value does not fall within the expected range.".

This error was triggered by the Windows SharePoint Services receive location or send port with URI wss://SERVER/sites/BizTalk Repository/Inbound Documents.

Windows SharePoint Services adapter event ID: 12295".

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

The end result is that I can use information contained in the file name to populate meta data columns in SharePoint without an Orchestration.

Also note, I have built this pipeline component so that it can be used in a Receive Pipeline or as Send Pipeline by including the CATID_Decoder and CATID_Encoder attributes.

   [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
   [ComponentCategory(CategoryTypes.CATID_Decoder)]
   [ComponentCategory(CategoryTypes.CATID_Encoder)]
   [System.Runtime.InteropServices.Guid("9d0e4103-4cce-4536-83fa-4a5040674ad6")]
   public class AddSharePointMetaData : IBaseComponent, IComponentUI, IComponent, IPersistPropertyBag

ShareTalk Integration (SharePoint/BizTalk) – Part 9 Overwrite - Rename Mode

2010-01-10T12:23:00.001-08:00

While further investigating SharePoint-BizTalk integration capabilities I ran into a file overwrite mode called “Rename” which I thought was rather unique. I have never seen something like this in the FILE or FTP adapters so I figured that I would post something on it.

The WSS Adapter has an Overwrite property that allows you select one of the following modes:

The Yes and No values are pretty self explanatory. Orchestration allows you to determine which mode you want to set within an Orchestration at run time.

When you select the Rename value, BizTalk will use the static file name that you have configured in the Send Port configuration or the actual file name of the source document. If the document library does not contain a file with this name already, it will be added with the correct filename.

Where it gets interesting is when you try to insert a file with the same file name as one that already exists in the same document library. When this situation occurs the first portion of the file name will remain intact, however a unique identifier will be added to the end of the filename. BizTalk will not raise a warning or error indicating that this event has occurred.

So when would I use this? I think this feature could be used in one of those “exception to the rule cases” where the business has indicated that “there will never be a duplicate file uploaded into this document library” but it actually happens. It is also helpful when the view of the document library has been configured in such a way that the documents are sorted by file name. This way it makes it easier to identify one of these cases as opposed to just using a Message ID where the document could end up being displayed in a very random fashion.

Reprocessing SAP IDocs through BizTalk Server using WE19

2010-01-02T11:34:00.001-08:00

First off this technique should work with any RFC connected system but since the only RFC connected system that I use is BizTalk it will therefore have some BizTalk specific information.

I have touched on this topic before in my web cast and in my blog that integrating with SAP can be tough due to the vast differences in terminology, process and technologies used. A situation that I have come across is that you often need a Subject Matter Expert (SME) or someone very familiar with a Business Process to generate an IDoc for a BizTalk developer to test or validate their portion of the integrated application. Sometimes creating an IDoc is very simple where as sometimes it may take a few transactions and screens to generate the IDoc in order to validate one of your use cases. This can lead to some frustration by both the BizTalk Developer and SAP resource as the BizTalk developer is always bothering the SAP resource to create “just one more IDoc”.

Something that a colleague brought to my attention was an SAP transaction called WE19 that allows you to re-process an existing IDoc. This allows the BizTalk developer to have the SAP resource create an initial IDoc and have the BizTalk developer re-submit that IDoc however many times they would like. Another benefit of this is process that involved deltas. We have a process where SAP will provide a “dump” of employee data that we need to update a down stream system with. We don’t need to process this data every day so we will just send the differences, or deltas, on a regular basis. So, even in a test or staging environment, once I have received the IDoc that contains all employee data I will not get that data again unless I ask for it. If we have not had any changes to our employees I will also not get the delta information either since there is no new data to feed the downstream system with.

Using this transaction is great because I don’t need to bother anyone from the SAP side to create data for me. All I need to know is when the last time that particular IDoc was sent. I can find this information in BizTalk by querying tracked message instances or by querying WE02 in SAP. Once I find this information I need to capture the IDoc number as this piece of information is required in order to resubmit the IDoc.

Here are 3 different ways to find an IDoc number. The first two involve BizTalk and the last involves using the WE02 transaction in SAP. You may want to check with your BASIS administrator if you do not have access to the WE02 transaction.

Option #1 Obtaining the IDoc number from BizTalk Admin by viewing the message context property
- Use a Tracked Message Events query from BizTalk 2009 Admin console and right mouse click on the IDoc that you previously received and select “Message Details”

Click on the “Context” caption

Observe the “DOCNUM” value and record it (unfortunately you can’t copy or paste from this screen)

Option #2 Saving IDoc message to disk and then inspecting context document.
- Once again find the particular IDoc that you are looking for by running a Tracked Message Events query from BizTalk 2009 Admin Console. Once you find this message, right mouse click and select “Save to File”

Navigate to the folder that you saved these message to and open up the XML file that contains all of the message context information

Once again locate the “DOCNUM” property and copy that value

Option #3 Use SAP transaction WE02 to locate your IDOC

Right mouse click on row and select “Copy Text”. Note it will copy all text in row.

Resubmitting the IDoc

Now that you have the IDoc number using one of the approaches above you can navigate to SAP transaction WE19, populate the “Existing IDoc” field and click on the clock with the green checkmark in the upper left hand corner(underlined in red) .

The selected IDoc will then be displayed and then click on the “Standard outbound processing” button

You now have the ability to provide the number of instances of this IDoc that you would like sent so you can use this as a load testing tool for BizTalk as well. Click the green checkmark to proceed.

You should now see a successful message indicating that the IDoc has been re-sent.

Summary

I know I have found the WE19 transaction to be very helpful so hopefully you will find it helpful as well. It can be pretty frustrating when you have a deadline and there is no one around to create IDocs for you. Using this approach will lessen your dependency on having a SAP resource around. You can also use this same technique to load up BizTalk with messages and test the robustness of the SAP adapter no matter whether it be the dot net connector version or the WCF version.

Microsoft MVP Awarded: Part 3

2010-01-02T10:23:00.001-08:00

I was recently informed that my MVP status has been renewed for 2010 in the BizTalk Server track. The MVP program is an excellent opportunity to interact with product groups and other MVPs from around the world. An example of this collaboration was the BizTalk Light and Easy Viewing series that I had an opportunity to participate in this past year. It was a great community project that involved experts from all over the world building webcasts demonstrating BizTalk 2009’s capabilities which were made available to anyone who was interested in viewing/downloading them.

The highlight of the program for me is definitely the annual MVP Summit which is hosted in the greater Seattle area each year. I have met a lot of great people through this event and continue to communicate with them on a regular basis.

BizTalk Adapter Pack 2.0 Migration Wizard – SAP IDOCs

2009-12-31T15:30:00.001-08:00

I have had a couple readers ask me some questions about the BizTalk Adapter Pack Migration Wizard and how well it migrates projects that leveraged the .Net connector based SAP Adapter. This is something that I have been interested in exploring for quite some time but just hadn’t gotten around to it. When we initially went live with our BizTalk 2009 environment we decided to reduce the complexity and and not migrate the SAP projects to the new WCF based adapter knowing that we had a little time to take care of this task. We have now started to migrate some of these apps to use the WCF based adapter so I figured I would re-create a use case from an existing integration scenario to see how well the wizard works.

In order to re-create this use case I decided to start the slate clean by creating a new BizTalk 2009 project and then adding an extended CONF32 IDoc called ZConf32. If you are unfamiliar with the term “extended IDoc” please see one of my previous posts. Some of this post may be redundant, as it discusses the old adapter, but it could help someone trying to integrate with SAP with the legacy adapter as well.

Note: From here on in I will refer to the .Net Connector SAP adapter as the legacy adapter and the BizTalk Adapter Pack 2.0 SAP adapter as the WCF adapter. Nothing in this post covers the BizTalk Adapter Pack 1.0 SAP adapter however I would assume that most scenarios that apply to the 2.0 adapter also apply to the 1.0 adapter.

Creating the BizTalk project

Adding a legacy Generated Schema to the project

With the legacy adapter you need to “Add Adapter Metadata” but with the WCF adapter you would want to “Consume Adapter Services’

Select the “SAP” adapter and then select a configured port that will be used to connect to an SAP instance. I miss this feature in the WCF adapter because when the schema generation fails, you will need to re-establish the connection to SAP by providing the URI, Username and Password again. I usually copy this information into notepad but the solution is not as clean as it is with the legacy adapter.

Since this IDoc has been extended, it is referred to as “CONF32-ZCONF32”.

The latest version released will show up by default

The Wizard working…

Success

Two artifacts are added to my project: An Orchestration and my CONF32.xsd schema. I have gone ahead and renamed these two artifacts accordingly.

The ZCONF32 IDoc will be sent to SAP so we need a source document that we will transform into the ZCONF32 IDoc. The input message captures a Fleet Vehicle’s time captured while working in the field. I work in the Energy industry and we need to record the amount of time that our fleet vehicles are in the field. There is some additional information that I need to provide SAP with called a control record that will describe the message type, IDoc Type and Partner profile to name a few of the data elements. I have captured this information inside String Concatenation functiods

Here is where the “core” data is being mapped into the IDoc.

The legacy adapter treats an IDoc as a flat file. So in order for BizTalk to assemble a flat file we need to create a Send Pipeline and assign the IDoc schema to the Flat File Assembler

The end solution looks like the following:

Note: This scenario could easily be accomplished as a messaging only scenario, but I have decided to provide an orchestration for illustrative purposes.

My receive location is your standard local file drop

In the Send Port, I want to choose the legacy SAP adapter and ensure to select the Flat File Pipeline that will assemble a flat file message for SAP.

If you navigate to a screen called “WE02” in the SAP GUI (assuming you have permissions) you can see all of the IDocs that SAP has sent/received. The status of the IDoc will also be captured here (green light) and indicate whether or not the process was unsuccessful/successful or still in processing.

You can drill into a specific IDoc to validate the data that BizTalk sent was complete and is accurate.

So that completes creating a fairly simple integration scenario with SAP using the legacy adapter. Next we will take a detail look into what is involved in migrating this project to use the new WCF Adapter.

First, you will need to download the BizTalk Adapter Pack 2.0 Migration wizard which you can find here. Unzip the file to a location on a BizTalk Server that has a functional installation of the WCF-SAP adapter. You will find the following 3 files but don’t bother running the “BizTalkAdapterPackMigrationToole.exe” just yet.

Ensure that you have the following pre-requisites already installed and then copy the Microsoft.WizardFramework.dll into this folder.

Ensure the following are installed:

Microsoft BizTalk Adapter Pack 2.0

WCF LOB Adapter SDK SP2

BizTalk Server 2009 or BizTalk Server 2006 R2

Visual Studio 2008 or Visual Studio 2005

To setup the migration directory:

Copy %Microsoft Visual Studio Install Dir% \Common7\IDE\ Microsoft.WizardFramework.dll to the same folder.

Now you should be able to run the “BizTalkAdapterPackMigrationTool.exe”

I am now going to load up the project the I previously demonstrated into this tool
Provide the URI for the SAP instance that you would like to connect to. Here would be a good place to leverage a re-useable connection such as a send port as every time you run this tool you need to provide these details.

The Wizard has determined that I am using the ZConf32 IDoc in my project and would like to generate a new schema using the WCF adapter.

You have the option to specify a release version of an IDoc. In the legacy project, I generated a schema based upon version 700 so I am going to select 700 for this IDoc. As mentioned BizTalk will be sending this IDoc to SAP so I will select the “Send” Operation

Gotcha!!! I previously mentioned that I am using an Extended IDoc called ZConf32. Well the base IDoc is called “CONF32” so I need to type this information into the “BaseType” drop down. This drop down is not populated with any data so you need to provide it.

If you don’t provide this information you will probably be prompted with an error similar to the following:

You have two options when running the migration wizard:
- The first option is intended to be less intrusive to your existing BizTalk solution and takes advantage of Outbound or Inbound maps, depending upon your scenario, that are applied at your physical ports. You essentially modifying your solution at the edges which has some advantages. I will touch this feature a little more at the end of this blog post.
- The second option is to update the existing solution with the new artifacts including maps and orchestrations. This option is more intrusive as you are making changes inside your solution that will impact the way your application functions.

So which option is better? Like most things in IT, it depends. If you have a complex solution with very “busy” maps I would opt for the first option – making changes at the edges. This option poses less risk as you are limiting the amount of change you are introducing. This option does require some updates to the core application due to the synchronous nature of the WCF based adapter. Previously IDocs were sent asynchronously (one way) where as now they are processed using a Request-Response (two way) process. So if you had an Orchestration that had a logical Send Port and Send Shape, you would now need to account for the response message coming back into the BizTalk Orchestration.

If you have a simple integration, like the one I am describing I would opt for option #2. Adding additional maps will decrease performance(most likely marginally) and also becomes a more complex application to support operationally as you are now supporting additional maps. If you have the same developer who built the BizTalk solution migrating it then they would probably also feel more comfortable in option #2. If you have brought in someone in to perform the migration they may feel more comfortable with the first option.

Success

Conversion report

When I open up the migrated solution, I noticed a few new artifacts including my IDoc Request message (SAPBinding3.xsd) and my IDoc Response message (SAPBinding4.xsd). I also have a folder called “NewSchemaTypes” that includes some of the complex types that the Request message uses. Like any schema that has been generated by the BizTalk Adapter Pack I also have a binding file called “SAPMigration_Binding.xml”

As advertised the migration option that I chose has updated my orchestration including the message type for a message called “msgZConf32”.

It also updated the Logical Port Type to use the message type of the WCF based schema.

What it failed to do is update the map that transforms the incoming request document into the ZConf 32 IDoc. What I decided to do was replace the legacy schema with the new WCF based schema.

Any existing functoids remained in the map however I had to reconnect all of the lines.

Although the Migration Wizard updated my logical Port, I needed to add a new port type since the old port type was a one way send port.

I also had to create a new response message which I called “msgZConf32Response” and added a Receive shape that would be responsible for handling the response message from SAP.

At this point I was ready to build and deploy. The build process was successful but the deployment process was not.

Error 1 Failed to add resource(s). Resource (-Type="System.BizTalk:BizTalkAssembly" -Luid="SAPMigration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=bb0707bffbca2a29") is already in store and is either associated with another application or with another type.

Having seen this error before I figured that the name of the application had probably been removed from the project file. So I just added the Application Name and then I was able to deploy.

I still had the legacy Send Port configured in my application so I needed to add a WCF based Send Port.

In order to add this send port, I needed to import the binding file that was generated by the migration wizard

Let’s explore what is included in the binding file that was generated by the wizard:

Notice that BizTalk uses the WCF-Custom adapter that leverages an SAP Binding and you no longer need to build flat file pipelines when communicating with SAP using this adapter.

You will need to update the action as the action that is included in the binding file will result in an error being generated:

The adapter failed to transmit message going to send port "processVehicleTime_PortType_SendZConf32" with URL "sap://CLIENT=XXX;LANG=EN;@a/SAPSERVER/XX". It will be retransmitted after the retry interval specified for this Send Port. Details:"Microsoft.ServiceModel.Channels.Common.UnsupportedOperationException: Incorrect Action <BtsActionMapping xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

</BtsActionMapping>. Correct the specified Action, or refer to the documentation on the allowed formats for the Actions.

Check out this link for more details.

Here I am using the default values including enabling BizTalk compatibility and Safe Typing.

Don’t forget to include you credentials as they are not brought in with the binding file.

One last thing that I need to do before starting the application is to bind the Orchestration to use this new Send Port.

At this point I have started the application and processed a message. The message was successfully submitted to SAP as confirmed in the WE02 transaction.

From a BizTalk tracking perspective this what the old and new process looks like:

Old process

New Process – since we are using a Request-Response port type we now receive a message back from SAP(XMLReceive)

As previously mentioned, I decided to use the intrusive approach when migrating this application. I am now going to quickly walk through the less intrusive approach that involves using mapping on the edges to deal with the differences between the legacy and WCF based schemas. Unfortunately this did not migrate as I would have expected so I don’t have a whole lot to show you.

The project was not updated as advertised. Only the new schemas was created, no new maps were created.

The orchestration remained the same as advertised

Summary

So as you can see the migration wizard had some mixed results. I am not quite sure why the Map, in either scenario, was not created/modified. The map itself was not overly complex although it did included some functoids. Another feature that I did not like was that you don’t have the ability to apply a naming convention to the schemas that were generated. If you were to add these schemas to a BizTalk solution from Visual Studio you do have the ability to provide a pre-fix using that tool so I am not sure why you could not use this same feature in the migration tool.

If you use option #2(the intrusive approach) it will go ahead and update your ports and message to use the new schema so I do think that there is some value in using the migration tool. Also, if you want to update several IDocs within one project you should be able to perform this migration by running the tool once. I definitely do not think that the tool is a silver bullet and you will need someone who is knowledgeable with BizTalk to perform these migrations. In defense of the tool, it is not an officially supported tool but is there to act as an aid and I do believe that it does accomplish this. All in all I think it took me about 20 minutes to migrate my fairly trivial application so the effort was not momentous.

If you are interested in seeing how this tool works with the new WCF based SQL Server adapter please check out Richard Seroter’s post.

BizTalk Server Futures and Roadmap

2009-12-08T16:49:00.001-08:00

I realize that I am a little late to this party, but figured I would provide my thoughts on the BizTalk Server Futures and Roadmap presentation by Balasubramanian Sriram that occurred at PDC 09 this past November. To view a re-play of his presentation check out the following link. Also note that some of the content from this post has been 'borrowed' from his presentation.

Current State

BizTalk adoption continues to be strong. Currently over 10 000 Global customers are benefiting from BizTalk Server. If my memory serves me correctly, they have added approximately 1500 customers over the past 2 years.
Major industries using BizTalk include Electronic Parts companies, Telecommunications, Aerospace and Defense, Chemical Companies, Railroads and Insurance. In Canada I am aware of a few other industries that are also benefiting from BizTalk Server including Utilities, Oil and Gas, Healthcare and Government.

Short-term Road Map

Here is a list of some of the Customer "wants" that are being addressed:

Platform support

Platform Alignment (Windows Server 2008 R2, SQL Server 2008 R2, Visual Studio 2010)
Tighter integration with Windows Workflow
Common Application Model (scaling up .net apps into an integration server)

Productivity

Ease of use in transformation scenarios
Trading Partner Management
Out of box connectivity (more adapters)

Enterprise Capabilities

ESB
Enterprise manageability (consolidated view of integration assets)
Low latency scenarios

How are some of these wants going to be addressed in the short term?
BizTalk Server 2009 R2

Platform support

VS 2010, Windows Server 2008 R2, SQL Server 2008 R2

Productivity Improvements

Single dashboard to apply and manage performance parameters
Out of box support for event filtering and delivering (RFID)
Powershell access to management tasks
New SCOM object model to better reflect BizTalk Artifacts

B2B Scenarios made easy

Mapper enhancements to make complex mapping easier to create and maintain
FTPS to provide secure transactions between businesses
Updated B2B accelerators for latest protocol versions

I am extremely encouraged to see theses added features in an R2 release especially since it seemed like BizTalk 2009 shipped yesterday. In particular, I am looking forward to the following features:

Platform support - It is nice to see that we should not have to wait as long as we previously did to be able to leverage some of these new technologies. It is always a bit of a downer when your organization is trying to stay current with technology only to have your application "slow the train down" because your application is not supported on the latest version of OS/DB/IDE etc.

Single Dashboard - It would be nice to get a composite view of your BizTalk environment from a single source. This Dashboard is also suppose to provide performance metrics and server health from one location. When working in an enterprise environment, multi-node groups are the norm, not the exception so it will be nice to have improved visibility.

New SCOM object model - I haven't been overly impressed with the SCOM management pack for BizTalk 2009. I find that some of the terminology that is used in the Management pack does not align with the terminology in the BizTalk Administration Console. For instance SCOM may notify you that you have an "Adapter" that is currently offline. No where in the BizTalk Admin console can you turn on/off, enable/disable an adapter. You can certainly add or delete an adapter but I am pretty sure that is not what SCOM is referring to. However, you can certainly enable/disable a receive location or stop a host instance which will impact an adapter's ability to function, but SCOM doesn't explicitly describe it this way. I also feel that the SCOM management pack doesn't correctly understand clustered host instances. I have received many alerts indicating that Clustered Host instance 'XYZ' is not running on Server 'b'. Well the reason for this is that the Host instance is currently active on Server 'a' and the nature of a Clustered resource is that it runs in an active/passive state by design.

Mapper Enhancements - Balasubramanian was correct when he mentioned the mapper as a "value add" for BizTalk. Since this is the case, then more should be done to improve a developer's productivity when using this tool. I have shown this tool to SAP resources describing how great the tool is only to find them cringing as we try to follow the connector lines when mapping an IDOC. I am extremely encouraged by what I saw and can definitely benefit by some of the new features described including only showing nodes that are involved in a map, auto scrolling, moving sections of maps to a new page and searching for nodes.

FTPS Adapter - This is welcomed change for me. The reality is that not every single interface is going to use the latest version of WCF. Providing an updated FTP adapter that supports security will be beneficial for BizTalk customers. I am still waiting for an update to the POP3 and SMTP adapters so that we can communicate with Exchange over MAPI. There is perceived risk associated with POP3 that makes Exchange administrators cringe when you ask them to open up POP3 connectivity. Yes 3rd party adapters do exist, but after several years, I do have expectations that something like this could be included in the box.

Future

Fear not, Microsoft will continue to support the BizTalk investments that have already been made to date. However, BizTalk will eventually run on top of AppFabric.

Here are some of the features that are envisioned for the future:

Use data contracts in maps and have the ability to transform to a schema and vice versa. One of the limitations of AppFabric (Dublin), in its current state, is the lack of a mapper tool to allow for transformation of messages. This new capability should fill in this current gap and also allow you to mix the use of data contracts with schemas in maps.
Create workflow activity based on map
Low-latency scenario versus durable messaging. This is a feature that many have been asking for. This feature is really intriguing in the sense that you can choose which path you would like to take when designing your message interactions. In Balasubramanian's scenario he was wanting to book a trip. Whenever read requests are required durable messaging is probably not required so why take on the additional performance hit that using MessageBox incurs. However when performing insert or update operations you probably want to use durable messaging to ensure that people's trips are booked. Using this hybrid approach allows for the best of both worlds within the same technology stack. Cool!
BizTalk will use innovations in AppFabric while preserving your investments
Current demo included Workflow in running in BizTalk Host – long term vision is for BizTalk to run in AppFabric host

A question that I have is: currently BizTalk is a product and has a licensing cost associated with it. AppFabric is really a framework that has no licensing costs associated with it. As these two platforms converge, which "components" will continue to have a licensing cost versus no licensing costs?

Overall it appears to me that the future for both of these platforms is extremely bright and Microsoft continues to make key investments into integration.

BizTalk 2009: Windows Server 2008 and SAMBA shares

2009-11-08T14:06:00.001-08:00

Another adventure that we experienced during a recent BizTalk 2009 cutover was the behaviour that Windows 2008 has on SAMBA shares. For those of you who are unfamiliar with SAMBA shares they basically provide you the ability to access *nix shares from Windows based computers.

In our previous setup(BizTalk 2006 running on Windows 2003) we had no issues communicating with SAMBA shares. Our infrastructure team recently updated the test system that we connect to version > 3.x of SAMBA where we had no issues when communicating with Windows 2008/BizTalk 2009 servers. When we went live with this cutover in Prod, the SAMBA version had not been updated in that environment yet and we were running an older version of SAMBA (2.2.x). The result of this was the following error which led to Host Instances going offline.

Event Type:        Error
Event Source:    BizTalk Server 2009
Event Category:                (1)
Event ID:              6913
Date:                     11/7/2009
Time:                     7:00:50 PM
User:                     N/A
Computer:          Server
Description:

An attempt to connect to "BizTalkMgmtDb" SQL Server database on server "SQLServer\SQLInstance" failed.

Error: "Login failed. The login is from an untrusted domain and cannot be used with Windows authentication."

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

The underlying cause for these errors was the BizTalk Host Instance account, that is used to connect to these shares, was being locked out due the issues in connecting with SAMBA. Pretty much a BizTalk Developer/Admin/Architect’s worst nightmare.

After performing some online searches we ran into the following article. The Article simply states that “Windows Vista and Server 2008 have a default version requirement of MS-LAN Manager communication that prohibits communication to older Linux-based Samba installations. This can be fixed via group policy or the local security policy.”

You can read the article for more details, but what helped us was setting the LAN Manager authentication level to “Send LM & NTLM responses”. Upon forcing a Group Policy update we were back in business thanks to the help of a few members of our infrastructure team.

We also looked into providing Authentication credentials but that wouldn't help since these are only provided when the Host Instance does not have access. The BizTalk user did have access so those credentials are sent before these configured credentials are passed anyways.

BizTalk 2009 – NSoftware FTP Adapter FTP protocol error: 550 rename: Cross-device link

2009-11-08T09:18:00.001-08:00

During a recent Production implementation of BizTalk 2009 we ran into an issue with the temporary folder option in the NSoftware FTP adapter. The error itself is not an NSoftware bug, but rather a feature/limitation of the FTP Protocol.

Event Type:    Error
Event Source:    nsoftware BizTalk FTP Transmit Adapter
Event Category:    None
Event ID:    0
Date:        11/7/2009
Time:        2:50:22 PM
User:        N/A
Computer:    Server
Description:
Source: nsoftware BizTalk FTP Transmit Adapter (3.5.3488.0)
Source URI: FTP://%User%@%FTPServer%:21/%RemotePath%/%RemoteFile%
Message Type: Error
Current Thread: 71
Transmission failed for message "019ef16e-0b57-4309-8a37-b5c05a61e7f9": Error uploading FTP data: FTP protocol error: 550 rename: Cross-device link

If this error persists, you may enable Warning or Verbose logging modes to enable the adapter to report more information regarding the progression of this error.

Our scenario involved writing files in a specific sequence using delivery notification and using a temporary folder to support some reliability requirements. The idea behind using a temp folder is you get the data to a “staging” area and once the adapter can confirm that all data was written it performs a rename operation on that file which is extremely quick. It also reduces the risk of a consuming application retrieving the data from the file as it is being written. This is especially important in *nix environments since file level locking has not been implemented the same way as it has in Windows environments.

Here is how our destination folders have been configured:

/home/work

/home/arch

/home/temp

Since we are using delivery notification, the “work” file has to be written to first. Once that operation has completed, then the arch(archive) file can be written. Both of these messages are being sent through a dynamic send port and we have a routing rules repository where we store the specifics of where the message is suppose to be delivered to. Our configuration called for both operations to use the same temp location. (Hindsight is 20/20)

In Test, everything worked out perfectly but when we ran this process in Prod we ended up with the aforementioned error. The difference between these environments is that the “arch” folder exists on a separate disk than “work” or “temp” in Prod. Essentially the FTP Protocol cannot perform the rename operation from the “temp” folder across to a folder on a different disk that has been mounted to the same server.

I was able to validate this by opening up a command prompt and establishing an FTP session. Once I connected I tried to rename a file and place it on the “arch” disk. I got a similar error:

ftp> rename list.txt /home/arch

350 File exists, ready for destination name

550 rename: Cross-device link

ftp>

In a previous blog post I discussed that NSoftware use to have a property called temporary file extension. This property is no longer a property that is exposed in the Send Port configuration but is still supported when you dive into their documentation. To use this property , you want to click on the “…” button beside the “Other” property.

In this property dialog box, populate TemporaryUploadExtension=.tmp

This feature acts similarly to the temp folder, the difference is that the “temp” file will be written to the same folder as the file that you are transferring . A consuming system that respects file extensions will still be “OK” with this situation since the file name will have “.tmp” added to it until the entire file has been written and at this point will remove this temporary extension via the rename operation.

This situation just highlights how important matching your QA/Test environments and Production really is. The other lesson is that not everything is exactly as it appears. In our case the folder structures were exactly the same. The difference was that one of the folders was mounted to a different disk. Finding this out is not a lot of fun when you are in the middle of a go live.

ShareTalk Integration (SharePoint/BizTalk) – Part 8 Integrating with the Cloud

2009-10-27T20:23:00.001-07:00

First off I apologize for the length of this post, but I trust that you will find some benefits with it. The goal of this post is to demonstrate an on-premise form being completed and having that information sent to the cloud via BizTalk 2009. The cloud will route this request to an off-site service that will respond to this request indicating whether or not the request was valid and return a result back to BizTalk. BizTalk will use this status to kick off some workflow. I will also dive into some of collaboration features of WSS 3.0 and demonstrate how you can still provide form exception handling and form re-submittal in an asynchronous scenario.

You may think that I am just trying to stuff as much Microsoft technology into one blog post as possible. What I am really demonstrating is giving BizTalk a user interface and allowing it to communicate with a off-premise service via the cloud. The service itself is not hosted in the cloud, but rather in someone else’s data centre. I am using the Microsoft cloud to relay the request using the App Fabric Service Bus(formerly known as the .Net Service Bus).

Note: I started this blog post back in early November prior to November 2009 CTP so I have had to go through and resolve the breaking changes that the new CTP introduced. I am not planning on getting into these breaking changes in much detail, but for further details check out the App Fabric team’s blog post.

To give this post some business substance, I am going to walk through a condensed use case in the Power industry in Alberta, Canada. The Power industry is broken down into a few segments: Generators, Transmission companies, Distribution companies and Retailers. The scenario I am going to walk through is on that involves a customer switching their Retailer. Distribution companies provide the “infrastructure” that allows power to be delivered to Residential and Industrial customers. Distribution companies are also required to provide meter usage data to Retailers where they will use this information to provide the end customer with a bill. Customers generally have an option when it comes to choosing their Retailer but not so much the Distribution company. In this scenario, the customer would like to switch retailers. The Retailer must send the Distribution company a “Switch Retailer Request” in order for this customer to be officially “switched”. The Distribution company needs this information so that they can send the correct meter data to the Retailer so they can bill their customer.

Information Worker will fill out InfoPath “Switch Retailer Request”
InfoPath form will be saved into a WSS Document Library
BizTalk will consume this InfoPath form and submit the data to the Distribution Company’s “Switch Retailer" Service. This Service is an on-premise service that is accessible via the ~~.Net Service Bus~~ AppFabric Service Bus.
The ~~.Net Service~~ App Fabric Service Bus will provide a “simple” SiteID validation. A SiteID is a identifier used to designate a Customer site – kinda like an address.
If the SiteID validation is successful then the retailer will be switched and the InfoPath form will be sent to the “Success Switch Retailer Request” document library
If the SiteID validation fails then the InfoPath form will be sent to the “Failed Switch Retailer Request” document library
When the InfoPath form is sent to the Failed Switch Retailer Request” document library an email will be sent to the information worker informing them that the switch was not successful. The Information worker can then update the form to fix the error. An error description is also provided in an “Error Message” column in the document library.

On Premise Switch Retailer Service

The Switch Retailer Service was written using C#, WCF and the ~~July CTP of the .Net Services SDK~~ November CTP of the App Fabric Service Bus SDK. I started with a pure WCF Service(so that I could generate Schemas for BizTalk) and then added the bits required for this service to be exposed via the .Net Service bus. This is a similar approach to the one that Richard Seroter used in his blog post.

I started with a couple of data contracts:

SwitchRetailerRequest – this message will be sent from BizTalk and will contain pertinent information like the Customer’s Site ID, the Activation Date, the Retailer Id etc.

SwitchRetailerResponse – this message will be sent back to BizTalk from the SwitchRetailer service. As part of this message, a status will be sent back and an ErrorMsg when the Switch Retailer fails.

My interface defines an operation called “CustomerSwitchRetailer” and as you can see it is expecting a SwitchRetailerRequest and will return a SwitchRetailerResponse. I have also provide a namespace and a name for this Service contract.

The implementation of this service includes a call to the “validSite” method. This method contains the logic that indicates whether this site is valid.

The host for this service is a console application. If you are familiar with the “Echo” sample then this will look very familiar. When the CTP changed, I had to update this method due to some of the breaking changes.

Here is the significant portion of my services’ App.config. I started with the .Net Services SDK Echo sample and have adopted it for my use. Changes include creating a behaviour called “SbEndpointBehavior”. In this behavior, I have indicated that I want to use “UserNamePassword” credentials as the credentialType. When BizTalk connects to this service, we will need to match this credentialType in the BizTalk Admin Console. The credentials themselves are credentials that you configure in the ~~.Net Service bus developer portal~~.

The UserNamePassword credential type is no longer supported in the latest version of the CTP. The prescriptive guidance is to use one of the new authentication types including Shared Secret, Simple Web Token or SAML. For the purpose of this example I have used Shared Secret.

Also notice the binding. I have chosen the “netTcpRelayBinding” that ships with the .Net Service SDK. When I initially created the WCF service I was using the basicHttpBinding but since I want to take advantage of the ~~.Net~~ App Fabric Service Bus’ relay capabilities I want to use this binding.

BizTalk

If we shift gears to the BizTalk portion of the solution, you will find the following artifacts:

SwitchRetailer_InfoPath.xsd schema. This schema is the message that BizTalk will receive from SharePoint and is also the underlying data structure in our InfoPath form. The “ErrorMessage” element is not displayed on the InfoPath form but will be used later to populate a SharePoint column with the error message that is being returned from the Switch Retailer service.

Service Schema – This schema was initially generated from consuming the WCF service that I built(before converting it to a .Net Service bus service). A powerful feature of WCF, and therefore the ~~.Net~~ App Fabric Service bus, is the separation between contracts and bindings. I am still using the same contract between the WCF service and the one exposed by the .Net Service Bus, but the binding will be different.

I need a map that will transform the data in the InfoPath form into the Service Request method

Once I have this request mapped, I want to send it to the service and receive the response.

As part of the response process, I need to determine the status of Switch Retailer. I want to use a Distinguished field to make the retrieval of this status a simpler endeavor. Since adding Distinguished Fields(or promoted properties) to a Service schema is not advised since you may update that schema later and lose the designation, I have created a helper message and want to map the Switch Retailer Response message to this helper message.

I have a simple decide shape that will check the value of this distinguished field. In the event that the status is true then I want to populate the “Success” SharePoint document library and in the event that the status is false then I want to populate the “Unsuccessful” SharePoint document library.

Note: The WSS Adapter can be used in a dynamic fashion, so I could eliminate 1 port and use a dynamic port instead of a static port. To keep this example simple, I have opted to use 2 static ports.

In the “Else” branch I want to populate this return message with the Error Message from the service response message. I will use this information to populate a column in the "Failures” document library

To allow for the Error Message to flow through to SharePoint, you will need to create a column in the document library(see below), but you will also need to configure BizTalk to pass this data along to SharePoint. So in your “Failures” send port, set both the “Namespaces Aliases” and then set the Column 01 Value with an appropriate XPath statement.

Namespace Aliases: ns0=http://BizTalk_Server_Project3.SwitchRetailer_InfoPath

Column 01 Value: %XPATH=//ns0:SwitchRetailer/ErrorMessage%

SharePoint

I am not going to go through each step in detail if I have already gone through it prevously but I will try to link back each step to a previous blog post where applicable.

Creating the InfoPath form from an XSD (Post 6)

Publish Info Path form (Post 6). As part of this process a document library called “Switch Retailer Document Library” was created.

Once you have published your form, when you click on “New” –> “New Document” your Switch Retailer Info Path Form should now appear.

Recall in the BizTalk section that I have to route Switch Retail responses to either a Unsuccessful or Successful document libraries (Post 3). I have now gone ahead and done this.

In the case of the “Failures” document library I want to be able to provide error information into this document library so that when the information worker comes into fix the issue that they are aware of what the issue is before they open the form.

Click on “Settings” –> “Create Column”

When a Switch Request Fails, I would like to notify the information worker of this situation. Yes, BizTalk has an SMTP adapter, but I have decided to leverage SharePoint to provide this functionality. In order to do this I need to set up my outgoing SMTP settings in WSS 3.0.
- Launch the “SharePoint 3.0 Central Administration” tool

Click on “Application Management”

Click on “Web application outgoing e-mail settings”

Ensure that the correct Web Application has been selected and then provide the required email settings. For this example I am just routing emails through my ISP.

Also ensure that the user who will be receiving these failure alerts has an email associated to their account. To do this jump back to your “Web Site” and click on the “Site Actions” - “Site Settings” link

Click on “People and groups”

Click on the link of the user that you are interested in receiving email

Ensure that you have an email provided

Navigate back to your “Failures” document library

Click on “Actions” –> “Alert Me”

Provide a title for your alert, add any required users that you want to receive this email and then indicate when you want the user(s) to receive the email. I have decided to configure these users to only receive alerts when a “new” item is added.

Configuring the demo

Start the “Switch Retailer Cloud” service

Since I am using the November 2009 CTP SDK and have opted to use the Shared Secret authentication type, I need to provided a Service Namespace, my Issuer Name and my Issuer Secret. These details are available from the Azure Developer Portal.

Next I want to ensure my BizTalk configuration is correct. The first step is to establish my receive location to SharePoint that will look for Switch Retailer InfoPath forms that have been submitted to the “Switch Retailer Document Library”

The next step is to establish my configuration that will allow BizTalk to connect to the App Fabric Service bus. In this two-way send port, I am going to use the WCF-Custom Adapter. This Adapter will allow me to use the netTcpRelayBinding which allows me to communicate with the App Fabric Service Bus. For this reason, you need to have the App Fabric Service Bus CTP installed on your BizTalk server.

I needed to create a Endpoint Behavior and since the Switch Retailer Service uses a transportClientEndpointBehaviour, the client(BizTalk) needs to match this Endpoint Behavior. Since UserNamePassword is no longer an option, I have opted for “SharedSecret”.

I now need to create two Send Ports; one that will route Successful Retailer switches and one that will route Unsuccessful Retailer switches.

Running the Demo

In the previous steps, we have started the Switch Retailer Service, configured BizTalk and are are ready to start calling this service via SharePoint/BizTalk. The first InfoPath form submitted will be successful, followed by a failed request.

InfoPath form has been populated and submitted

Switch Retailer Request has been placed in Document Library

BizTalk is polling this Document Library on a regular basis

The App Fabric Service receives this request and routes it to the Switch Retailer Service. The service will validate the request and provide the response to BizTalk.

As you can see the response was successful and the document was routed through the SharePoint Success Send Port.

The InfoPath form has been routed to the “Success” Document Library

I will now send an invalid Switch Request by providing a Site ID that only has a length of 3.

The Service has detected this invalid Site Id and logged it accordingly

BizTalk has received the response and detected that the Switch Retailer failed. BizTalk will therefore route this message to the “Failed” document library.

Failed InfoPath form has been delivered and the Error that has occurred is populated in the “Error Message” column.

I had previously configured SharePoint to notify me, via email, whenever a new document was placed in the “Switch Retailer Failures” document library.

I have fixed up the Site Id and re-submitted

The Switch Retailer Service has received this updated request and this time it was successful

Generally hooking up BizTalk to a user interface can be a risky endeavor as the UI can be blocked while BizTalk is processing. If a downstream system is down, a time out could exist on the UI client if BizTalk is working through its retry progressions. By using the approach above, we have eliminated the potential of the UI client being locked up while BizTalk is calling the Switch Retailer service. By leveraging some workflow capabilities in SharePoint and BizTalk we can still provide the user meaningful feedback in reasonable amount of time asynchronously.

ShareTalk Integration Series - PDF Available

2009-10-19T07:36:00.000-07:00

Bil Simser, a colleague and SharePoint MVP has captured this series as a PDF document. I do have plans for one or two more posts in this series and he has offered to update the PDF as new posts appear.

Here is a link to his post where you can find this PDF document.

ShareTalk Integration (SharePoint/BizTalk) – Part 7 Adding Content Approval

2009-10-18T16:59:00.001-07:00

In the last post, I showed you how to submit InfoPath forms to SharePoint and have BizTalk consume those forms once they were posted. This is great, but what if you need to have a supervisor approve the form before BizTalk consumes it? In this post we will use a feature in WSS 3.0 called “Content Approval” to mange the state that a document is in.

Open up your Document library, click on “Settings” – >”Form Library Settings”

Click on “Versioning settings”

Ensure that you select “Require content approval for submitted items”, modify your security as required and then click on “OK”

If you navigate back to your Document Library, you will notice a new column called “Approval Status”

Notice that when you add a new form that it has a status of “Pending”. Also, if you haven’t done so disable your BizTalk receive location because BizTalk will not observe the status of the document – for now anyways.

When you click on the pull down menu for the form, you now have an “Approve/reject” option – click it.

Click on the “Approved” radio button, add a comment and click “OK”

Notice now that the document has been approved and the Approval Status reflects this change.

So if we want BizTalk to only consume forms that have been approved, how does this help us? What we can now do is create a “View” and then populate the “View Name” property in the BizTalk Receive Location.

Click on the “All Documents” drop down and select “Create View”

Click on “Standard View” link

Provide the View with a name: “Approved Forms”

Create a filter that indicates only to display items in this view when “Approval Status” is equal to “Approved” and click OK.

Add another form to this library but don’t approve it. Now toggle between the “All Documents” and “Approved Forms”

Modify your BizTalk Receive Location so that it will only pull documents from the “Approved Forms” view.

Enable this receive location and you should find that only the “Approved” form is consumed. The “Pending” form should still be in the document library.

What really impresses me about building these scenarios is the depth of the Microsoft platforms. I haven’t been in Visual Studio much at all through this series. There is so much functionality that is available out of the box which makes these scenarios very quick to build.