The ``Direct Deposit'' protocol

ASIM team
LIP6 Laboratory
Paris, France

Next: The ``remote write'' primitive Up: Functional Description Previous: Functional Description

The ``Direct Deposit'' protocol

All memory addresses handled by PCI-DDC are physical addresses. PCI-DDC does not provide any mechanism to translate virtual to physical addresses, this task is left to the software.

From the application software point of view, the transfer unit is the message. Since a message is in the virtual memory, it can be distributed on several zones in physical memory. PCI-DDC only manages physical contiguous memory zones in both local and remote nodes. For PCI-DDC, such a zone is called a page, and then a message consists of a list of one to several pages.

PCI-DDC manages a list of pages to emit (LPE), which is a structure placed in main memory that contains page descriptors. The software driver writes page descriptors into the LPE (List of Pages to Emit). All page descriptors corresponding to the same message must have consecutive entries in the LPE. PCI-DDC has pointers to manage the LPE. A page descriptor contains a Message Identifier (MI: 24bits), a destination node number called Routing Part (RP: 16bits), a local physical address (PLSA: 32bits) which is the start address of the page in the local memory, a Page Length (PL: 16bits) in bytes and a remote physical address (PRSA: 32bits) which is the start address for the page in the remote memory. There are also few more bits to define protocol details.

Pages are the smallest entities from the software point of view but not from the hardware point of view. Pages are split up in packets by PCI-DDC. The packet is the physical exchange unit. The packet length is variable and dynamically adapted by PCI-DDC, depending on the PCI burst length and the HSL network load. Each packet contains all the informations necessary to reconstruct the message in the remote memory (i.e. PRSA and MI).

Each message is identified by a number called Message Identifier (MI). All pages associated to the same message have the same MI. The MI is used by PCI-DDC to detect message completion. There are two possible methods to detect a message completion. In adaptive networks, packets can be received in a different order than they have been sent. PCI-DDC uses a List of Received Messages (LRM) implemented in main memory. The LRM allows to keep the current packet number for each received message. PCI-DDC uses the MI as an index in the LRM. The last packet of a message contains the number of packet sent which is the expected packet number from the receiver point of view. Therefore, a message is completed when the received packet number is equal to the expected packet number. In non-adaptive networks, it is possible to detect message completion without using the LRM. This can save memory (the LRM is a large table in physical memory) and decrease the traffic on the PCI bus. In this case, message completion is detected by the Last Packet bit contained in the last packet of a message.

Beside the LPE and LRM, the last data structure used by PCI-DDC is the LMI (List of Message Identifier). That structure is used by PCI-DDC to indicate the message received without error. In case of short messages, the same structure receive the 8-byte data.

Next: The ``remote write'' primitive Up: Functional Description Previous: Functional Description