Overview of ALE / IDOCs

ALE is a set of

• Tools
  
• programs and
  
• data definitions

that provides the mechanism for distributing functionality and data across multiple system.

What Data can be Exchanged ?

Transaction Data

– SD, MM etc.

Master Data

– Material, Customer, Vendor, etc.

Control Data

– Organizational Reference Information

• Plants, Sales Orgs, etc.

Data required to enable tightly coupled, distributed applications

- Separate HR, Separate Treasury, etc.

IDOC is the acronym for Intermediate Document. It is simply a data container used to exchange information between any two processes that can understand the syntax and semantics of the data. IDocs are relatively simple to understand. But, like most simple things they are difficult to explain.

IDocs are basically simple ASCII data streams. When they are stored to a disk file, the IDocs are simple flat files with lines of text, where the lines are structured into data fields.

The typical structured file has records, each record starting with a leading string that identifies the record type. Their specification is stored in the SAP data dictionary.

Characteristics:-

• An IDoc is not a process

• In SAP system IDocs are stored in the database

• Every IDoc has a unique number (within a client)

• IDocs are independent of the sending and receiving system ( SAP to SAP as well as Non-SAP)

• IDocs are based on EDI standards (like ANSI X.12, EDIFACT )

• IDocs can be viewed in a text editor and do not contain any binary data. Data is stored in character format.

The information which is exchanged by IDocs is called a message and the IDoc is the physical representation of such a message. The name “messages” for the information sent via IDocs is used in the same ways as other EDI standards. Everybody who has ever dealt with interface programming will find IDocs very much like the hierarchical data files used in traditional data exchange. International standards like the ODETTE or VDA formats are designed in the same way as IDocs are.

Other EDI standards like XML, ANSI X.12 or EDIFACT/UN are based on a data description language. They differ principally from the IDocs concept, because they use a programming language syntax (e.g. like Postscript or HTML) to embed the data.

Features –ALE / IDocs

• Distributed System yet integrated with SAP R/3
  
• Based on ‘Application-to-Application integration using ‘Message Architecture’
  
• Reliable communication
  
• Data is exchanged using “IDocs”
  
• Support both R/2, R/3 and External system
  
• If network problem, message is buffered
  
• ALE support backward compatibility
  

IDoc Type

IDoc type is a document structure that describes the order in which fields and records occur, as well as the data type of these fields. The EDI document to be generated has an equivalent message type defined in the SAP system. The message type is based on an IDOC structure. As an example, if an EDI transaction of 850, which is a purchase order, is to be generated, the message type ORDERS is assigned to it. This message is based on an IDOC type such as ORDERS05. The IDOC type is the most important component in the EDI process.

An IDOC type is also referred to as basic IDOC type. A basic IDOC type has the following characteristics:

Name: A basic IDOC type can be assigned up to a 30 character name in release 4.0 and later. Segments of a previous version of an IDOC type are never deleted. This approach is necessary to maintain backward compatibility. In the standard list of IDOCs for example ORDERS01, ORDERS02, ORDERS03, ORDERS04, ORDERS05 types are available. The last two characters represent the version.

List of permitted segments: Only the segments defined in the IDOC type can be used in an IDOC.

Hierarchy of segments: The hierarchy of segments specifies the physical sequence and any parent-child relationship in the segments. A parent-child relationship signifies that the child segment can not exist without the parent.

Mandatory versus optional segment: When used in an IDOC type, each segment has an attribute that defines whether the segment is optional or mandatory.

Maximum/Minimum range for each segment: When used in an IDOC type each segment has an attribute that defines the minimum and the maximum number of times a data record corresponding to a segment can exist in an IDOC.

IDOC Record Types & Structures

An IDOC is an instance of an IDOC type. An IDOC has a record oriented structure. Although there are several records in an IDOC, they are still classified as one of the three record types:

• Control Record

• Data Record

• Status Record

Control Record

The control record contains all of the control information about an IDOC; this information basically includes the IDOC number, sender and receiver information, and information such as the message type it represents and the IDOC type.

Features of IDoc control record:-

1.There is only one control record per IDOC.

2.The structure of the control record is the same for all the IDocs and is defined by SAP.

3.The structure of the control record is defined by the data dictionary structure EDI_DC40 in release 4.0 onwards. The control record is stored in the EDIDC table.

4.One control record defined by the SAP Data Dictionary structure EDI_DC40, with a record length of 524 bytes.

The control record carries all the administrative information of the IDoc, such as its origin, its destination and a categorical description of the contents and context of the attached IDoc data. This is very much like the envelope or cover sheet that would accompany any paper document sent via postal mail.

For R/3 inbound processing, the control record is used by the standard IDoc processing mechanism to determine the method for processing the IDoc. This method is usually a function module but may be a business object as well. The processing method can be fully customized.

Data Record

All records in the IDocs, which come after the control record, are data records. Data records contain the actual application data. They are all structured alike, with a segment information part and a data part which is 1000 characters in length, filling the rest of the line.

All records of an IDoc are structured the same way, regardless of their actual content. They are records with a fixed length segment info part to the left, which is followed by the segment data, which are always 1000 characters long.

Regardless of the used IDoc type, all IDocs are stored in the same database tables EDID4 for release 4.x and EDID3 for release and 3.x. Both release formats are slightly different with respect to the lengths of some fields.

Characteristics of Data Record:-

• A data record has two parts: an administrative section and a data section, as shown in the figure below. A data record size can have maximum size of 1063 bytes in release 4.0 and on. The data record is defined by the data dictionary structure EDI_DD40 for IDocs from version 4.0 on and EDI_DD for version 3.0-3.1 IDocs. The structure of the administrative section differs somewhat in the two versions.
  
• The data section of a data record is a stream of 1,000 bytes where the actual data resides. The data section is mapped to a segment type, as defined in the administrative section, to interpret the meaning of various data values in a record.
  
• Data records for IDocs from version 4.0 on are stored in the EDID4 table, while version 3.0-3.1 IDoc data records are in EDID2.

Status Records

The status record conforms to the data dictionary structure EDI_DS40 for IDocs from version 4.0 on and EDI_DS for previous versions. Status records are stored in the EDIDS table. It contains information on the state of the IDOC as it passes through the various stages of processing.

• The STATUS field has a length of two bytes (data type CHAR), with a range of values: status codes of 01 to 49 are reserved for outbound processes, and 50 and above for inbound processes.

• The status record also includes date and timestamps for when that particular state was reached. The status records maintain a history of the IDOC states. The latest status code is also maintained in the control record.

• An IDOC may have one or more status records (multiple status records), which are stored in table EDIDS. These records can be accessed or created only by SAP function modules (APIs), and are not externalized. So we could not see status records in an IDoc flat file outside the SAP system.

Fields in the Status Record Table (EDIDS):

Field Name	Description
MANDT	Client
DOCNUM	IDoc number
LOGDAT	Date of status information
LOGTIM	Time of status information
COUNTR	EDI status counter
CREDAT	Date status record was created
CRETIM	Time status record was created
STATUS	Status of IDoc
UNAME	User Name
REPID	Program Name
ROUTID	Name of the sub-routine (routine, function module)
STACOD	Status code
STATXT	Text for status code
SEGNUM	Number of SAP segment
SEGFLD	Field name in SAP segment
STAPA1	Parameter 1
STAPA2	Parameter 2
STAPA3	Parameter 3
STAPA4	Parameter 4
STATYP	EDI: Type of system error message
STAMQU	Status message qualifier
STAMID	Status message ID
STAMNO	Status message number
TID	Transaction ID
APPL_LOG	Application log

Syntax Rules for an IDoc

When an IDoc is created, it goes through a syntax check to ensure its integrity. The syntax of an IDoc is governed by the definition of its IDoc type. The syntax rules checked for an IDoc are as follows:-

• Only valid segments as defined in the IDoc type are allowed.

• Segments specified as mandatory must exist.

• A data record cannot exceed the maximum number of repetitions as defined for the segment type.

• Segments must occur in the same physical order defined the IDoc structure.

• A child segment cannot exist without its parent segment. A parent segment, however, can exist without a child segment.

EDI Process Flow of an Outbound IDoc

The outbound process sends documents from the SAP system to a business partner (vendor, customer or bank). The outbound process consists of six steps.

1. The application document is created. The first step in the outbound process involves creating an application document, such as a purchase order or sales order, and saving it in the database tables. The data in the application document format suitable for the application modules, screens, and internal programs.

2. The IDoc is generated. The application document just created id now formatted to an IDoc format. A document in an IDoc format is suitable for processing by EDI components.

3. The IDoc is transferred from SAP to the operating system layer: The IDoc created in the step 2 resides in the SAP database repository. This IDoc document must be passed down to the operating system layer for processing by the EDI Subsystem. Here in step 3 the IDoc is transferred to the OS as a text file.

4. The IDoc is converted to EDI standard. IDoc is an SAP proprietary format. For EDI purposes, the document in IDoc format is converted into an EDI standard format. A type of third-party software called a translator carries out the transformation process. SAP refers to these translators as EDI subsystem, and has certified several subsystems for connectivity to SAP. SAP takes no responsibility for translation.

5. The EDI document is transmitted to the business partner. After the document is converted to an EDI standard format, it is transmitted to a trading partner based on the partner’s EDI settings.

6. The EDI subsystem reports status back to SAP. When an IDoc is under the control of EDI subsystem, the EDI subsystem can optionally report the state of processing at various milestones back to the SAP system. The status records are generated by the subsystem and passed back to SAP. This mechanism provides complete visibility of the translation and the transmission process from within SAP.

EDI Process Flow of an Inbound IDoc

The inbound process simply reverses the steps of the outbound process. The inbound process receives an EDI document (such as a purchase order response, sales order or payment information) from a business partner (such as a vendor, a customer or a bank) and creates SAP documents from it.

The inbound process consists of four steps:

1. The EDI transmission is received. EDI documents are received from a business partner via the VAN. These documents are in one of the EDI standard formats and deposited in the EDI subsystem’s repository.

2. The EDI document is converted into an IDoc. Subsystem’s translator converts the EDI format document into an IDoc format suitable for SAP applications.

3. The IDoc is transferred to the SAP layer. The IDoc text file is stored at the SAP’s operating system layer. The EDI subsystem triggers an inbound program in the SAP layer. Then a SAP application reads the IDoc file and stores an IDoc in the SAP database repository for further processing.

4. The application document is created. The IDoc is then passed to a posting program. This program creates an application document such as a sales order, purchase order acknowledgment, invoice etc. The document can be viewed using standard application transaction.

Triggering the Inbound-Outbound Process

After receiving an inbound EDI transmission and creating an IDoc file, the EDI subsystem is often responsible for triggering the inbound process.SAP provides a program named startrfc to start RFC enabled functional module EDI_DATA_INCOMING to process the IDoc file. In outbound process when an IDoc leaves SAP system for processing by the EDI subsystem, SAP looses control over the IDoc for EDI translation and EDI transmission. But the EDI subsystem creates a status file and maintains the log of processing stages with time-stamp, date-stamp, status codes etc.

In outbound process the EDI subsystem is responsible for reporting to SAP any errors including:-

• Translation errors

• Syntax errors

• Transmission errors

• Connectivity errors

After the successful or un-successful processing, the EDI subsystem triggers the startrfc program in the SAP system to start RFC enabled functional module EDI_STATUS_INCOMING for processing the status file created by the EDI subsystem. The SAP system then retrieves the status logs from that file and stores in the EDIDS table for the respective IDoc number. The latest status code is also maintained in the IDOC’s control record.

Status codes passed to SAP by the Subsystem

Status Code	Description
04	Error within control information of EDI subsystem
05	Error during translation
06	Translation OK
07	Error during syntax check
08	Syntax check OK
09	Error during interchange handling
10	Interchange handling OK
11	Error during dispatch
12	Dispatch OK
13	Retransmission OK
15	Interchange Acknowledgment negative
16	Functional Acknowledgment positive
17	Functional Acknowledgment negative
22	Dispatch OK, acknowledgement still due
23	Error during retransmission
24	Control information of EDI subsystem OK
36	Electronic signature not performed (timeout)

Commonly used IDoc segments in EDIFACT mapping :

Following is a list of commonly used IDoc segments and theirs correspondingmapping segments in EDIFACT. This list is just for example purpose only; there can be one or more mapping fields which might be used for client’s requirement.

IDOC SEGMENT	MAPPING	EDIFACT SEGMENT
EDI_DC40 - Control Record	<->	UNH, UNT
E2EDK01 - Header general Data	<->	BGM
E2EDKA1 - Partner Information	<->	NAD
E2EDK02 - Reference Data	<->	BGM, RFF
E2EDK03 - Date segment	<->	DTM,PAT
E2EDK05 - Conditions	<->	MOA, ALC
E2EDK04 - Tax segment	<->	TAX, MOA
E2EDK18 - Terms of payment	<->	PCD
E2EDKT1 - Text Identification	<->	FTX
E2EDKT2 - Header text	<->	FTX
E2EDP01 - Item general Data	<->	LIN, QTY
E2EDP03 - Item Date segment	<->	DTM
E2EDP19 - Item object Identification	<->	PIA, LIN, IMD
E2EDP05 - Item condition	<->	MOA, PRI
E2EDP04 - Item Tax	<->	TAX, MOA
E2EDS01 - Summary segment	<->	CNT