--- name: "nsend" section: "2" platform: "linux" package: "lam2-dev" version: "6.3.2-3" release: "Debian 2.2 (Potato)" platforms: ["linux"] url: "https://new.carta.tech/nsend/2" --- # nsend(2) Send and receive LAM network messages. ## NAME nsend, ntry\_send, nrecv, ntry\_recv - Send and receive LAM network messages. ## C SYNOPSIS ``` #include int nsend (struct nmsg *header); int ntry_send (struct nmsg *header); int nrecv (struct nmsg *header); int ntry_recv (struct nmsg *header); ``` ## FORTRAN SYNOPSIS subroutine NSND (nnode, nevent, ntype, nlength, nflags, ndata, ndsize, nmsg, ierror) subroutine NRCV (nevent, ntype, nlength, nflags, ndata, ndsize, nmsg, ierror) integer nevent, ntype, nlength, nflags, ndata(\*), ndsize, ierror\ \ nmsg(\*) ## DESCRIPTION The network message-passing functions add routing and packetization to the datalink functions, [dsend(2)](/dsend/2.md) and [drecv(2)](/drecv/2.md). *nrecv()* blocks if there is no synchronizing message to receive. *nsend()* will block if there is no synchronizing receiving process or forwarding process to take its message (see "Blocking"). *ntry\_send()* and *ntry\_recv()* never cause the calling process to block. The message is either immediately transferred, or an error is immediately returned, indicating that the process would have blocked. See [nprobe(2)](/nprobe/2.md) for similar functionality. ### Network Message Descriptor All of the functions accept a pointer to a network message descriptor which is an extension of the local level message descriptor used by [ksend(2)](/ksend/2.md) and [krecv(2)](/krecv/2.md). The network message descriptor is defined in \. ``` struct nmsg { int nh_dl_event; int nh_dl_link; int nh_node; int nh_event; int nh_type; int nh_length; int nh_flags; int nh_data[NHDSIZE]; char *nh_msg; }; ``` - nh\_dl\_event unused This field is unchanged by *nrecv()* but is set to the event of the synchronizing process (either the intended receiver or a forwarding process) after calling *nsend()*. See [dsend(2)](/dsend/2.md). - nh\_dl\_link unused This field is unchanged by *nrecv()* but is set to the output link number if the message was forwarded to a datalink output process after calling *nsend()*. See [dsend(2)](/dsend/2.md). - nh\_node This field is used by *nsend()* to identify the remote node running the intended receiver. It is not used by *nrecv()*. A receiving process thus cannot directly specify the source node of a message. Instead, receiving processes are "matched" to messages by one or both of *nh\_event* and *nh\_type*. Two special node identifiers are defined in \. LOCAL refers to the local node and causes *nsend()* to bypass its routing step. ORIGIN refers to the node from which [lamboot(1)](/lamboot/1.md) was invoked. This field is never altered. - nh\_event An event is an arbitrary positive integer used by the LAM kernel to synchronize processes within a node. Synchronization occurs when two events are equal. *nsend()* transfers the message to the destination node and then to the highest priority process blocked on the event in the message's *nh\_event* field and a matching type (see below). Thus, the sender calling *nsend()* must set *nh\_event* to the same value as the receiver calling *nrecv()*. This field is never altered. - nh\_type This field further filters messages that match on event. A message will be transferred to a receiver only if the *nh\_type* fields of the sender and receiver processes have at least one bit set in an identical position. In other words, the bitwise logical AND of the type fields specified by the two parties must not equal zero. A zero value matches any other value of *nh\_type*. This field remains unchanged after calling *nsend()*, but is set to the sender's *nh\_type* after calling *nrecv()*. - nh\_length This field holds the length (in bytes) of the message to be sent. If the sender and the receiver specify different lengths, the lesser amount will be transferred to the receiver. Messages longer than the maximum network packet size, defined by MAXNMSGLEN in \, will be implicitly broken down into a series of smaller messages, which will be recombined by the receiver. Because of this packetization, one call to *nsend()* can introduce many messages into the network, a possible source of confusion during debugging. This field remains unchanged after calling *nsend()*, but is set to the minimum of the sender's and receiver's lengths after calling *nrecv()*. - nh\_flags This field is normally set to 0. When the NOBUF flag (defined in \) is set in *nh\_flags* buffers will not be used. Flags used to assure that the data representation is correct for the receiving node are discussed under "Data Representation". This field is never altered. - nh\_data This field is a convenient data pouch within the network message descriptor. Its array size is NHDSIZE words, which is defined in \ and is set to 8. It can be used for sending short messages (in which case *nh\_length* is set to 0) or for appending control information to the message body. After calling *nrecv()* the *nh\_data* field is overwritten with the sender's values of the same field. The sender's *nh\_data* will not change. - nh\_msg This field holds the address of the first byte of data to be sent or received. The data must be stored contiguously in memory. This field is never altered. ### Data Representation On nodes of different architectures, data may have different representations. For example, integers may be stored with the most significant byte first in memory (big-endian) or with the most significant byte last in memory (little-endian). Also, the representation of floating point numbers may conform to the IEEE standard or may follow a vendor specific format. All fields in the network message structure, except the data referenced by *nh\_msg*, are automatically converted if passed to a node with different data representation. The *nh\_data* field is assumed to hold all integers. The *nh\_flags* field of the message structure can be set to the following data representation flags. Each flag assumes a data type, and will make the appropriate change in the data representation of the given field. They will have no effect if data conversion is not needed. - DINT4DATA *nh\_data* holds 8 32-bit integers (default). - DFLT4DATA *nh\_data* holds 8 single 32-bit real numbers. - DFLT8DATA *nh\_data* holds 4 64-bit real numbers. - DRAWDATA *nh\_data* representation will not be changed. - DINT4MSG *nh\_msg* points to 32-bit integers. - DFLT4MSG *nh\_msg* points to 32-bit real numbers. - DFLT8MSG *nh\_msg* points to 64-bit real numbers. - DRAWMSG *nh\_msg* representation will not be changed (default). If *nh\_data* or *nh\_msg* contains a mixture of data types, the user will have to change the representation using the function suites [ltot(3)](/ltot/3.md), [ttol(3)](/ttol/3.md), etc. ### Example Usage The following example passes a message between two nodes with similar data representations, utilizing a minimum level of synchronization. This is intended only as a summary of a simple case. Many variations can be constructed using the detailed information given in the above section. ``` /* Sender */ #include struct nmsg nhead; char *msg = "Hello, world"; nhead.nh_node = 10 nhead.nh_event = 6 nhead.nh_type = 0 nhead.nh_flags = 0 nhead.nh_length = strlen(msg) + 1; nhead.nh_msg = msg; nsend(&nhead); /* Receiver */ /* Assume this code is running on node 10. */ #include struct nmsg nhead; char msg[16] nhead.nh_event = 6 nhead.nh_type = 0 nhead.nh_flags = 0 nhead.nh_length = sizeof(msg); nhead.nh_msg = msg; nrecv(&nhead); ``` ### Blocking A process calling *nrecv()* blocks until the message sent by the process calling *nsend()* entirely arrives. A process calling *nsend()* blocks only until its message is picked up by: - a) a local receiver calling *nrecv()* - b) a local buffer process - c) a local forwarding process such as a datalink The only thing that is guaranteed by a successful return from *nsend()* is that the message has entirely left the calling process. The loose blocking behaviour of *nsend()* introduces a fundamental danger of LAM message passing: a sender can transmit a message that may never be received due to programming error or deadlock. This message will never be dropped or timed out. Some LAM process will always be stuck with it, waiting for a synchronizing *nrecv()* that may never happen. If that unfortunate process is a buffer, it can be located by the user and swept clean (see [sweep(1)](/sweep/1.md)). However, if the process is a link proprietor, the link is henceforth plugged and useless. Besides the legitimate buffer process, datalink processes can each hold one or more messages. NOBUF does not affect these implicit buffers. ## ERRORS - EWOULDBLOCK *ntry\_send()* or *ntry\_recv()* failed because the message could not be sent or received, respectively. A call to *nsend()* or *nrecv()* would have blocked. ## BUGS Multi-packet messages can inter-mingle packets if sent to the same node, event and type. The solution for this type of communication structure is to use [tsend(2)](/tsend/2.md) and [trecv(2)](/trecv/2.md). ## SEE ALSO [dsend(2)](/dsend/2.md), [nprobe(2)](/nprobe/2.md), [tsend(2)](/tsend/2.md)