--- name: "rsocket" section: "7" platform: "linux" package: "rdma-core" version: "65.0-1" release: "Arch Linux 2026-11-01" platforms: ["linux"] url: "https://new.carta.tech/rsocket" --- # rsocket(7) RDMA socket API ## NAME rsocket - RDMA socket API ## SYNOPSIS **#include \** ## DESCRIPTION RDMA socket API and protocol ## NOTES Rsockets is a protocol over RDMA that supports a socket-level API for applications. Rsocket APIs are intended to match the behavior of corresponding socket calls, except where noted. Rsocket functions match the name and function signature of socket calls, with the exception that all function calls are prefixed with an 'r'. The following functions are defined: rsocket rbind, rlisten, raccept, rconnect rshutdown, rclose rrecv, rrecvfrom, rrecvmsg, rread, rreadv rsend, rsendto, rsendmsg, rwrite, rwritev rpoll, rselect rgetpeername, rgetsockname rsetsockopt, rgetsockopt, rfcntl repoll\_create, repoll\_ctl, repoll\_wait Functions take the same parameters as that used for sockets. The follow capabilities and flags are supported at this time: PF\_INET, PF\_INET6, SOCK\_STREAM, SOCK\_DGRAM SOL\_SOCKET - SO\_ERROR, SO\_KEEPALIVE (flag supported, but ignored), SO\_LINGER, SO\_OOBINLINE, SO\_RCVBUF, SO\_REUSEADDR, SO\_SNDBUF IPPROTO\_TCP - TCP\_NODELAY, TCP\_MAXSEG IPPROTO\_IPV6 - IPV6\_V6ONLY MSG\_DONTWAIT, MSG\_PEEK, O\_NONBLOCK Rsockets provides extensions beyond normal socket routines that allow for direct placement of data into an application's buffer. This is also known as zero-copy support, since data is sent and received directly, bypassing copies into network controlled buffers. The following calls and options support direct data placement. riomap, riounmap, riowrite - off\_t riomap(int socket, void \*buf, size\_t len, int prot, int flags, off\_t offset) - Riomap registers an application buffer with the RDMA hardware associated with an rsocket. The buffer is registered either for local only access (PROT\_NONE) or for remote write access (PROT\_WRITE). When registered for remote access, the buffer is mapped to a given offset. The offset is either provided by the user, or if the user selects -1 for the offset, rsockets selects one. The remote peer may access an iomapped buffer directly by specifying the correct offset. The mapping is not guaranteed to be available until after the remote peer receives a data transfer initiated after riomap has completed. In order to enable the use of remote IO mapping calls on an rsocket, an application must set the number of IO mappings that are available to the remote peer. This may be done using the rsetsockopt RDMA\_IOMAPSIZE option. By default, an rsocket does not support remote IO mappings. riounmap - int riounmap(int socket, void \*buf, size\_t len) - Riounmap removes the mapping between a buffer and an rsocket. riowrite - size\_t riowrite(int socket, const void \*buf, size\_t count, off\_t offset, int flags) - Riowrite allows an application to transfer data over an rsocket directly into a remotely iomapped buffer. The remote buffer is specified through an offset parameter, which corresponds to a remote iomapped buffer. From the sender's perspective, riowrite behaves similar to rwrite. From a receiver's view, riowrite transfers are silently redirected into a pre- determined data buffer. Data is received automatically, and the receiver is not informed of the transfer. However, iowrite data is still considered part of the data stream, such that iowrite data will be written before a subsequent transfer is received. A message sent immediately after initiating an iowrite may be used to notify the receiver of the iowrite. In addition to standard socket options, rsockets supports options specific to RDMA devices and protocols. These options are accessible through rsetsockopt using SOL\_RDMA option level. - RDMA\_SQSIZE - Integer size of the underlying send queue. - RDMA\_RQSIZE - Integer size of the underlying receive queue. - RDMA\_INLINE - Integer size of inline data. - RDMA\_IOMAPSIZE - Integer number of remote IO mappings supported - RDMA\_ROUTE - struct ibv\_path\_data of path record for connection. Note that rsockets fd's cannot be passed into non-rsocket calls. For applications which must mix rsocket fd's with standard socket fd's or opened files, rpoll and rselect support polling both rsockets and normal fd's. Existing applications can make use of rsockets through the use of a preload library. Because rsockets implements an end-to-end protocol, both sides of a connection must use rsockets. The rdma\_cm library provides such a preload library, librspreload. To reduce the chance of the preload library intercepting calls without the user's explicit knowledge, the librspreload library is installed into %libdir%/rsocket subdirectory. The preload library can be used by setting LD\_PRELOAD when running. Note that not all applications will work with rsockets. Support is limited based on the socket options used by the application. Support for fork() is limited, but available. To use rsockets with the preload library for applications that call fork, users must set the environment variable RDMAV\_FORK\_SAFE=1 on both the client and server side of the connection. In general, fork is supportable for server applications that accept a connection, then fork off a process to handle the new connection. rsockets uses configuration files that give an administrator control over the default settings used by rsockets. Use files under /etc/rdma/rsocket as shown: mem\_default - default size of receive buffer(s) wmem\_default - default size of send buffer(s) sqsize\_default - default size of send queue rqsize\_default - default size of receive queue inline\_default - default size of inline data iomap\_size - default size of remote iomapping table polling\_time - default number of microseconds to poll for data before waiting wake\_up\_interval - maximum number of milliseconds to block in poll. This value is used to safe guard against potential application hangs in rpoll(). max\_events - maximum number of events for the epoll thread to handle for an epoll\_instance. All configuration files should contain a single integer value. Values may be set by issuing a command similar to the following example. echo 1000000 \> /etc/rdma/rsocket/mem\_default If configuration files are not available, rsockets uses internal defaults. Applications can override default values programmatically through the rsetsockopt routine. ## SEE ALSO [rdma\_cm(7)](/rdma_cm/7.md)