SAP IDOC Overview

Today IDocs are used in most SAP applications for transfer of message(information) from SAP system to other systems and vice versa. Though lot of documentation is available on IDocs it is difficult for a functional consultant to understand such documents due to their technical nature.  While a functional  consultant  is not expected to know the IDoc concepts in its entirety, an effort has been made to capture the minimum necessary information that one needs to be aware of in order to handle project/support issues on IDocs.

OVERVIEW
IDoc is an SAP object that carries data of a business transaction from one system to another in the form of electronic message. IDoc is an acronym forIntermediate Document. The purpose of an IDoc is to transfer data or information from SAP to other systems and vice versa.  The transfer from SAP to non-SAP system is done via EDI (Electronic Data Interchange) subsystems whereas for transfer between two SAP systems, ALE is used.
IDoc can be triggered in SAP system or in EDI subsystem. This depends on the direction in which IDoc is sent and is called as Inbound IDoc and Outbound IDoc accordingly. In case of outbound flow, IDoc is triggered in SAP through document message control which is then sent to EDI subsystem. EDI converts the data from IDoc into XML or equivalent format and then sends the data to partner system through Internet.
For inbound flow, EDI converts partner data and IDoc is created in SAP. After successful processing of this IDoc, Application Document is posted in SAP.
EDI STANDARDS AND IDOC
“EDI is electronic exchange of business document between the computer systems of business partners, using a standard format over a communication network”. EDI stands for Electronic Data Interchange.
For transmission of information electronically, two widely used standards are ANSI ASC X12 and EDIFACT. ANSI ASC X12 is a committee formed by representatives of major organizations, government bodies and EDI software companies which defines standards and guidelines for information interchange over EDI. UN/EDIFACT stands for United Nations EDI for Administration, commerce and Transport and was formed in 1985 using ANSI X12 and UNTDI (United Nations Trade Data interchange) as base standards. ANSI X12 describes business document as transactions and each transaction is represented by three digit number e.g. 850 – Purchase Order, 855 – Purchase Order Acknowledgement. EDIFACT describes business document as messages, represented by standard names e.g. ORDERS for purchase order.

IDOC TERMINOLOGIES
IDOC (BASIC) TYPE
IDoc Types are based on the EDI standards and mostly on EDIFACT standards. 
Basic Types (or IDoc Type) defines the structure of an IDoc. Each basic type describes standard IDoc segments, format of data fields and their size. Basic Type also defines number of segments and fields in an IDoc. All the fields that are necessary for transmission of message for a particular business transaction are mapped in different segments. It also defines the structure and relationship of IDoc segments along with mandatory and optional segments.
IDOC EXTENSION
Basic type contains all the standard fields that are necessary for carrying out a business transaction. However, if any additional values are to be sent to the partner then we can make use of the IDoc Extension feature. IDoc extension is extension of basic type and contains additional custom IDoc segments and fields that are not available in standard basic type.

IDOC SEGMENTS
IDoc segments contain the actual data that is sent to or received from a partner. These segments contain the actual values that are sent as part of IDoc transmission.
PARENT AND CHILD SEGMENTS
IDoc segment is termed as Parent segment if it contains its own segments. The dependent segments are called as child segments.

INBOUND/OUTBOUND IDOCS
IDocs sent outside the system are termed as Outbound IDocs and the ones that are received into the system, are called as Inbound IDocs.
 
IDOC DIRECTION
This signifies the direction is which information is sent and is similar to terminology used in mails. If information is sent outside the system then the direction is outbox when it is received into the system then direction is inbox. In SAP Outbox direction is represent by “1” i.e. outbox and Inbox direction is represented by “2”.
 
PARTNER
Partner is the Business Partner with which the exchange of information is to take place using IDoc. It can be a vendor or customer or any other system. Depending on the direction of information in which the information is sent it plays a role of either a “sending partner” or a “receiving partner”.
 
PARTNER TYPE
Partner type/role is used to identify partners within the sap systems. Partner type is KU for customer, LI for vendor and LS for Logical System.
 
MESSAGE TYPE
IDoc processing involves transmission or receipt of document in the form of a message, each of which represents a document in SAP. These documents can be Order, Shipment Confirmation, Advance Shipping Notification, Goods Receipt, or Invoice.  Message type is associated with Basic IDoc Type (Basic Type) and defines the kind of data or document that is exchanged with the partner.
 
PROCESS CODE
The process code contains the details of the Function Module that are used for IDoc processing. Message Type can be linked to the Process code.
 
PORT
IDoc Port contains the information about the way data is sent between the source or target system. The type of port defines the information contained within the port. For port type “Internet” Port will contain IP address of the target system. For port type “file”, directory or file name information is maintained. “tRFC” port contains information about the RFC destination of the target system. For IDoc transmission using ALE “tRFC” ports are used.

PARTNER PROFILE MAINTENANCE
PARTNER PROFILE (WE20)
Partner profile must be maintained for all the business partners to whom we want to send or receive the IDocs. The TCODE for maintaining the partner profile is WE20.Double clicking on the Partner will show the following screen:Partner profile contains parameters for Inbound and Outbound processing of IDocs. For each message type we can maintain, inbound/outbound options, message control, post processing options and contact information within Inbound and outbound parameters.
 
OUTBOUND OPTIONS (OUTBOUND PARAMETERS)
This involves sender/receiver port, Output mode and relation to IDoc type i.e. Basic Type and extension.
 
MESSAGE CONTROL (OUTBOUND PARAMETERS)
This contains application for which IDoc will be created e.g. EF for Purchase order, the message type of the application that will trigger the IDoc and Process Code that will convert SAP document to an IDoc. For example, if PO is to be sent to the Vendor AXXXXZ, then in the outbound option of the partner AXXXXZ we need to maintain the message type ZXX1 and link it to the Process Code ME10. So when message type ZXX1 is triggered in the PO then an IDoc will be created for the partner vendor AXXXXZ.
Process Code is linked to the Function Module in SAP that converts application data into an IDoc. Standard function modules are provided by SAP for this conversion however these can also be customized as per business needs. Change Message Indicator indicates whether the IDoc is sentas a notification of change. For example, Purchase Order change messages are sent to vendor using EDI standard message type 860. Separate message type should be triggered in the purchase order for PO change. Additional line with change message type must be added in the Message control tab with change message indicator on.
 
INBOUND OPTIONS (INBOUND PARAMETERS)
For inbound options process code is maintained in the Inbound screen only. IDoc processing can be triggered by background program and triggered immediately.
 
POST PROCESSING (INBOUND/OUTBOUND PARAMETERS)
In the post processing option we can maintain the workflow details of the users or positions to which an error notification will be sent if an IDoc processing fails.
 
TELEPHONY (INBOUND/OUTBOUND PARAMETERS)
We can also maintain the contact details in the telephony option.
 
EDI STANDARD (OUTBOUND PARAMETERS)
EDI standard screen contains the details of the Standard EDI terminology used for the IDoc transmission.For example, Message Type 850 is an EDI standard for Purchase Order IDoc and is linked to IDoc Message Type Orders.

IDOC STRUCTURE AND RECORDS
STRUCTURE
IDoc structure is divided into Control Record, Data Records and Status records.These records are stored in the transparent tables in SAP. These are EDIDC, EDID4 and EDIDS.
 
CONTROL RECORD (EDIDC)
It contains information such as IDoc number, direction, IDoc Status, Basic Type, Message Type, Partner (Sender/Receiver), date and time of creation/update, Interchange File or ISA number,etc.
 
DATA RECORD (EDID4)
It contains the details of the IDoc segments.IDoc segment has fields that contain the data necessary for posting the documents.
 
STATUS RECORDS (EDIDS)
IDoc Status defines the processing status of the IDoc. IDoc statuses are used to track the IDoc and its various processing states. Status Numbers represents IDoc status. Current status of the IDoc is present in Control record.Initial Status numbers are 64 for inbound and 03 for outbound. Successful status is 53 for inbound and 16 for outbound IDocs.

SENDING AND RECEIVING IDOCS
TRIGGERING AN OUTBOUND IDOC
Outbound IDocs can be triggered from the output message types of Purchase Orders, deliveries, Material Documents, invoices, etc. The following figure shows that once the output ZXX1 of PO XXXXXXX1 is processed an IDoc “000000XXXXXXXXX1” is added/created.
The relationship between the IDoc and the application document can be found in two ways:
1. Relationship tab of IDoc

2. Relationship tab of Application Document, e.g. PO, SO, Material Document, etc.The initial status of this IDoc will be 30, which after successful processing will convert into status 16.A successful outbound IDoc will pass through all the above statuses in reverse order (01-03-18-06-12-16). Each status represents an IDoc validation step. If an IDoc passes all the validations it would reach status 16. These different validation steps for outbound IDocs are explained below:
01: IDoc generation successful
30: IDoc is ready to be processed by IDoc Processing job
03: IDoc data is passed to the Port
18: IDoc successfully triggered EDI subsystem
06: IDoc data translated to EDI format
12: IDoc is dispatched successfully to the partner
16: Partner has received the IDoc successfully
IDoc can possibly fail at any of the above steps during validation.
 
RECEIVING AN INBOUND IDOC
The initial status of an inbound IDoc is 64 and successful status is 53.
Different validation steps for inbound IDocs are explained below:
50: IDoc received successfully in the system
64: IDoc is ready to be processed by IDoc processing job
53: Application document created and saved successfully. The document number can be found by expanding the status node 53An inbound IDoc goes through all the above statuses in reverse order (50-64-53).

IDOC PROCESSING
AUTOMATIC/IMMEDIATE PROCESSING
In this case, IDoc are processed immediately as they generated or added in the system. The check “Transfer IDoc immediately” is selected in Outbound Options and “Trigger Immediately” is selected in Inbound Option. These checks are generally used when the real time information exchange is necessary between two systems.  
MANUAL PROCESSING
IDocs can also be manually processed using the TCODE BD87 in SAP.
 
PROCESSING VIA BACKGROUND JOB
IDoc processing by background is the most preferred way of processing the IDocs. Following Programs are used from processing the IDocs using background job:
RBDAPP01 – Inbound IDocs
RSEOUT00 – Outbound IDocs
 
REPROCESSING IDOCS
On the basis of IDoc statuses different programs can be used for reprocessing of failed IDocs. These are given below:

 
TESTING AND EDITING IDOCS
If an IDoc contains error in the data then such IDocs can be edited using TCode WE02 or WE05. When an IDoc is edited the original IDoc information(backup) is saved in a New IDoc under status 70 (for inbound) / 33 (for outbound). These IDoc stays in the system for reference only and cannot be processed. The status of the edited IDoc becomes 69 (inbound) and 32 (outbound). These IDocs can then be processed using BD87 transaction or batch jobs.
Debugging of IDocs can be done using by copying the IDocs using TCodeWE19. WE19 is a test tool for Idocs processing. WE19 copies the existing idoc and creates a new IDoc which can then be modified as per testing needs. The newly generated IDoc can also be processed using BD87.
 
CONVERTING IDOC STATUS
Report RC1_IDOC_SET_STATUS can be used to change the status of IDoc. Status changes are generally needed to move an IDoc to status 68 – no further processing
 
SEARCHING IDOCS IN SAP
TCODE WE02 / WE05: GENERAL SEARCH
IDocs can be displayed in system via TCODE WE02 and WE05. If IDoc number is not known then  search can be made on the basis of IDoc Date, Direction, BASIC TYPE, MESSAGE TYPE, and PARTNER NUMBER.Partner number can  be found in the Output Messages of the documents. IDoc search can also be made on the basis of ISA or Transfer file Reference. 
TCODE WE09: SEARCHING DATA IN IDOC SEGMENTS
If we are looking for specific information within the IDocs Segments then this can be found using TCODE WE09. This is useful if you are searching for a particular information in similar kind of IDoc within IDoc segments. For example, if you want to search a particular Purchase Order number e.g. 100000001 in multiple IDocs which lies in Segment E1EDK01 of an IDoc under field BELNR. Then the search can be executed in the following manner.
 
IDOC VALIDATIONS, COMMON IDOC ERRORS AND SOLUTION
 Though, the IDoc failure may not be related to any of the above mentioned reasons, the best way to find the IDoc error is to compare the existing IDoc with the good example. Good example IDoc can be easily searched with any of the IDoc search methods as described above.

DOCUMENTATION FOR IDOC TYPES
IDoc documentation can be found using TCODE WE60 and can be helpful to obtain information of the IDoc Type or its particular segment. It also provides information such as  mandatory and optional segments, minimum and maximum number of segments, etc.
 
GENERAL INFORMATION FOR COMMON IDOC MESSAGE TYPES
The following list gives the Basic Type and Message Type combination for common idocs
 
ARCHIVING/DELETION OF IDOCS FROM DATABASE
As IDocs grow older they are archived and deleted from the database. Archived IDocs can be viewed using TCODE SARI in Achieve Explorer using archiving object as IDoc. Following are the few programs that are used for archiving and deletion of IDocs from database.