Saturday, January 2, 2010

Reprocessing SAP IDocs through BizTalk Server using WE19

First off this technique should work with any RFC connected system but since the only RFC connected system that I use is  BizTalk it will therefore have some BizTalk specific information.

I have touched on this topic before in my web cast and in my blog that integrating with SAP can be tough due to the vast differences in terminology, process and technologies used.  A situation that I have come across is that you often need a Subject Matter Expert (SME) or someone very familiar with a Business Process to generate an IDoc for a BizTalk developer to test or validate their portion of the integrated application.  Sometimes creating an IDoc is very simple where as sometimes it may take a few transactions and screens to generate the IDoc in order to validate one of your use cases.  This can lead to some frustration by both the BizTalk Developer and SAP resource as the BizTalk developer is always bothering the SAP resource to create “just one more IDoc”.

Something that a colleague brought to my attention was an SAP transaction called WE19 that allows you to re-process an existing IDoc.  This allows the BizTalk developer to have the SAP resource create an initial IDoc and have the BizTalk developer re-submit that IDoc however many times they would like.  Another benefit of this is process that involved deltas.  We have a process where SAP will provide a “dump” of employee data that we need to update a down stream system with.  We don’t need to process this data every day so we will just send the differences, or deltas, on a regular basis.  So, even in a test or staging environment, once I have received the IDoc that contains all employee data I will not get that data again unless I ask for it.  If we have not had any changes to our employees I will also not get the delta information either since there is no new data to feed the downstream system with. 

Using this transaction is great because I don’t need to bother anyone from the SAP side to create data for me.  All I need to know is when the last time that particular IDoc was sent.  I can find this information in BizTalk by querying tracked message instances or by querying WE02 in SAP.  Once I find this information I need to capture the IDoc number as this piece of information is required in order to resubmit the IDoc.

Here are 3 different ways to find an IDoc number.  The first two involve BizTalk and the last involves using the WE02 transaction in SAP.  You may want to check with your BASIS administrator if you do not have access to the WE02 transaction.

  • Option #1 Obtaining the IDoc number from BizTalk Admin by viewing the message context property
    • Use a Tracked Message Events query from BizTalk 2009 Admin console and right mouse click on the IDoc that you previously received and select “Message Details”


    • Click on the “Context” caption


    • Observe the “DOCNUM” value and record it (unfortunately you can’t copy or paste from this screen)



  • Option #2 Saving IDoc message to disk and then inspecting context document.
    • Once again find the particular IDoc that you are looking for by running a Tracked Message Events query from BizTalk 2009 Admin Console.  Once you find this message, right mouse click and select “Save to File”


    • Navigate to the folder that you saved these message to and open up the XML file that contains all of the message context information


    • Once again locate the “DOCNUM” property and copy that value



  • Option #3 Use SAP transaction WE02 to locate your IDOC


    • Right mouse click on row and select “Copy Text”.  Note it will copy all text in row.



Resubmitting the IDoc

  • Now that you have the IDoc number using one of the approaches above you can navigate to SAP transaction WE19, populate the “Existing IDoc” field and click on the clock with the green checkmark in the upper left hand corner(underlined in red) .


    • The selected IDoc will then be displayed and then click on the “Standard outbound processing” button


    • You now have the ability to provide the number of instances of this IDoc that you would like sent so you can use this as a load testing tool for BizTalk as well.  Click the green checkmark to proceed.


    • You should now see a successful message indicating that the IDoc has been re-sent.




I know I have found the WE19 transaction to be very helpful so hopefully you will find it helpful as well. It can be pretty frustrating when you have a deadline and there is no one around to create IDocs for you.  Using this approach will lessen your dependency on having a SAP resource around.  You can also use this same technique to load up BizTalk with messages and test the robustness of the SAP adapter no matter whether it be the dot net connector version or the WCF version.


Unknown said...

Hi Kent,

I'm also working on SAP and BizTalk 2009 communication, thanks for your post.

Have you already experienced ordered delivery with WCF-SAP Adapter ? How to ensure this Adapter stores IDocs messages into MessageBox in the same order SAP provides them ?

Thanks for your answer.

Kent Weare said...

Hi Christophe,

I have dealt with ordered delivery inbound into SAP but not outbound. For inbound we are receiving a messages from a MQ Series queue, marking the receive port for order delivery, using a Singleton orchestration and then marking the send port for order delivery.

For your scenario I am not 100% sure but I think SAP will need to implement a QRFC interface which will inforce ordered delivery coming out of SAP. Sorry for the lack of details but I would start looking at QRFC.

Peter Oosterling said...

Reprocessing via WE19 is nothing more then creating a copy of the initial idoc. The initial idoc is not reprocessed, it is the copy whih is send instead. The initial idoc will stay on status 03 in SAP (tx WE05).
To actually reprocess the initial idoc itself you can use tx SM58. Select an idoc and go via the menu edit to Execute LUW, or to mass execute (via the background) choose Execute LUWs.
Now no copy will be made, but the initial idoc is reprocessed and if processing goes fine the status of this idoc is set to 12 (in case a status idoc is send back by biztalk)

I hope this is useful info

Kent Weare said...

Hi Peter,

Thanks for the comment. In terms of "Reprocessing" I think it depends what context you are considering. In my case, the IDoc is being reprocessed by BizTalk and not SAP. I am not looking to have the IDOC reprocessed by SAP, but rather it gives developers an easy way to resend IDocs to BizTalk which allows them to test without the envolvement of SMEs creating new IDOCS.
Thanks for your suggestions regarding having SAP reprocess IDOCs.


Unknown said...

Hello Kent

Have you experience with grabbing the status of processing an idoc that you send to SAP? We send an idoc with no DOCNUM field filled in and then would really like to know (back in BizTalk) if it was processed successfully (status=53).
I know there is ALEAUD idoc type that can be scheduled on SAP side and periodically would inform about the statuses but the issue is that we don't know the DOCNUM when sending the idoc but it is the only field to get correlation on when the ALEAUD idoc is sent down to BizTalk.
Please let me know as it is a show stopper for us now and mr google does not seem to know that...

Kent Weare said...

Hi Maciej

Interesting problem. We have not tried to implement this before, but I understand what you are trying to achieve. Since IDOC processing is an Asynchronous event, would a BAPI/RFC call be an option? This way your interface would be a syncrounous event and BizTalk would have an indication that SAP successfully received/processed the message.

Another question is what is BizTalk suppose to do when the status is not 53? Wouldn't this be a problem better solved through SAP Workflow or sending an indication to a person's SAP Inbox. I am not questioning your design, but curious to know what BizTalk would do in this situation since SAP would have a better idea why an IDOC didn't process. If you do find a solution, please post back here as I am interested as well.

Unknown said...

Thank you for your answer.

We are planning to use only BizTalk for routing (so no SAP workflows) in the design of the solution. Also going asynchrnonously seems to be a better idea having in mind that SAP does not always process the idocs when they are received and there would be SOME of them to process (hospital implementation of SAP).

What we're trying to achieve is e2e monitoring with BAM, meaning we use BAM API for monitoring in the receiving location, mapping and then SAP send adapter. To include the SAP end in the monitoring we'd like to see if the result of processing in SAP was good (53) and if not alert on that.

If you other ways of achieving that, please let me know, I'll let you know if I get it solved.


Unknown said...

Hi again
coming back on that e2e monitoring (and with a simple question).
I used this post: to find out that there is just a RFC named INBOUND_IDOCS_FOR_TID (yuppi) in SAP. Well, it does not work with SAP adapter (weee) because of the nested tables feature that it is using. No problem, my SAP colleague wrote the RFC selecting the right stuff (for me TID and DOCNUM) from the edids table and the work is done. One more small detail, he had to add the magic word SINGLE in the beginning of the select clause just to omit this ugly issue with nested tables (good we have one docnum per tid). Right, now only schedule sending of ALEAUD idocs to BizTalk (they contain DOCNUM together with the status code) and the works is done. I hope it helps someone crawling like me last days through the blogs.
Now for the question, I haven't worked with orchestrations much, so it comes - should we have one ProgId for the listener port which is starting an orchestration? I know, using CBR is better, one port with one ProgID is enough but just theoretically, is it one to one (rec port with ProgID for orchestration)?
Kind regards

Unknown said...

Hi Kent.

Thanks for all the excellent blog posts you have provided.

I have a question that ties in with this old blog post of yours.

I want to update 3rd party systems (downstream) with person/user information from SAP HR.

I have created an integration between SAP HR and Biztalk Server 2010, using the WCF SAP adapter v2010. On the SAP side SAP consultants have created a custom IDOC based on the HRMD_A07 message type, and activated change pointers for the new message type so we will receive (deltas) an IDOC when a change has occurred for users (ex: mobile number changed, email updated, new location, new hiring etc.).

The challenge is to determine from the received IDOC what really changed...because it contains a lot of information and info types, despite the filters applied on the SAP side. It is hard to determine what have changed and where from the structure of the received IDOCs.

Since you have quite a lot of experience integrating with SAP, maybe you have some suggestion on how to do this or could direct me in the right direction on what to look for or how??

Thanks in advance.


Kent Weare said...

Hi M.P.

I believe I understand what you are trying to achieve. I don't think there is a really good way to do this.

We do have a similar situation where we need to take materials from SAP and then update CRM. So for us we check to see if the material exists within CRM, if it doesn't we insert if it does then we will update. Now in this situation, CRM provides us with interfaces to do all of these things.

Another option is to store the data yourself is some sort of SQL respository and manage this yourself. I don't like this idea though as I believe that managing deltas should not happen within Middleware.

Another option would be to modify the SAP IDoc to have a modification timestamp on the various data elements. I would imagine this would also be very tedious but it is an option.