---
name: "mlx5dv_wr_post"
section: "3"
platform: "linux"
package: "rdma-core"
version: "65.0-1"
release: "Arch Linux 2026-11-01"
platforms: ["linux"]
url: "https://new.carta.tech/mlx5dv_wr_post"
---

# mlx5dv\_wr\_post(3)

Attach a DC info to the last work request mlx5dv\_wr\_raw\_wqe - Build a raw work request mlx5dv\_wr\_memcpy - Build a DMA memcpy work request

## NAME

mlx5dv\_wr\_set\_dc\_addr - Attach a DC info to the last work request

mlx5dv\_wr\_raw\_wqe - Build a raw work request

mlx5dv\_wr\_memcpy - Build a DMA memcpy work request

## SYNOPSIS

-

  ```
  #include <infiniband/mlx5dv.h>
  static inline void mlx5dv_wr_set_dc_addr(struct mlx5dv_qp_ex *mqp,
                                           struct ibv_ah *ah,
                                           uint32_t remote_dctn,
                                           uint64_t remote_dc_key);
  static inline void mlx5dv_wr_set_dc_addr_stream(struct mlx5dv_qp_ex *mqp,
                          struct ibv_ah *ah,
                          uint32_t remote_dctn,
                          uint64_t remote_dc_key,
                          uint16_t stream_id);
  struct mlx5dv_mr_interleaved {
      uint64_t        addr;
      uint32_t        bytes_count;
      uint32_t        bytes_skip;
      uint32_t        lkey;
  };
  static inline void mlx5dv_wr_mr_interleaved(struct mlx5dv_qp_ex *mqp,
                          struct mlx5dv_mkey *mkey,
                          uint32_t access_flags, /* use enum ibv_access_flags */
                          uint32_t repeat_count,
                          uint16_t num_interleaved,
                          struct mlx5dv_mr_interleaved *data);
  static inline void mlx5dv_wr_mr_list(struct mlx5dv_qp_ex *mqp,
                        struct mlx5dv_mkey *mkey,
                        uint32_t access_flags, /* use enum ibv_access_flags */
                        uint16_t num_sges,
                        struct ibv_sge *sge);
  static inline int mlx5dv_wr_raw_wqe(struct mlx5dv_qp_ex *mqp, const void *wqe);
  static inline void mlx5dv_wr_memcpy(struct mlx5dv_qp_ex *mqp_ex,
                      uint32_t dest_lkey, uint64_t dest_addr,
                      uint32_t src_lkey, uint64_t src_addr,
                      size_t length)
  ```

## DESCRIPTION

The MLX5DV work request APIs (mlx5dv\_wr\_\*) is an extension for IBV work request API (ibv\_wr\_\*) with mlx5 specific features for send work request. This may be used together with or without ibv\_wr\_\* calls.

## USAGE

To use these APIs a QP must be created using mlx5dv\_create\_qp() with *send\_ops\_flags* of struct ibv\_qp\_init\_attr\_ex set.

If the QP does not support all the requested work request types then QP creation will fail.

The mlx5dv\_qp\_ex is extracted from the IBV\_QP by ibv\_qp\_to\_qp\_ex() and mlx5dv\_qp\_ex\_from\_ibv\_qp\_ex(). This should be used to apply the mlx5 specific features on the posted WR.

A work request creation requires to use the ibv\_qp\_ex as described in the man for ibv\_wr\_post and mlx5dv\_qp with its available builders and setters.

### QP Specific builders

- *RC* QPs

  *mlx5dv\_wr\_mr\_interleaved()*

  registers an interleaved memory layout by using an indirect mkey and some interleaved data. The layout of the memory pointed by the mkey after its registration will be the *data* representation for the *num\_interleaved* entries. This single layout representation is repeated by *repeat\_count*. The *data* as described by struct mlx5dv\_mr\_interleaved will hold real data defined by *bytes\_count* and then a padding of *bytes\_skip*. Post a successful registration, RDMA operations can use this *mkey*. The hardware will scatter the data according to the pattern. The *mkey* should be used in a zero-based mode. The *addr* field in its *ibv\_sge* is an offset in the total data. To create this *mkey* mlx5dv\_create\_mkey() should be used. Current implementation requires the IBV\_SEND\_INLINE option to be on in *ibv\_qp\_ex-\>wr\_flags* field. To be able to have more than 3 *num\_interleaved* entries, the QP should be created with a larger WQE size that may fit it. This should be done using the *max\_inline\_data* attribute of *struct ibv\_qp\_cap* upon its creation. As one entry will be consumed for strided header, the *mkey* should be created with one more entry than the required *num\_interleaved*. In case *ibv\_qp\_ex-\>wr\_flags* turns on IBV\_SEND\_SIGNALED, the reported WC opcode will be MLX5DV\_WC\_UMR. Unregister the *mkey* to enable another pattern registration should be done via ibv\_post\_send with IBV\_WR\_LOCAL\_INV opcode. *mlx5dv\_wr\_mr\_list()* registers a memory layout based on list of ibv\_sge. The layout of the memory pointed by the *mkey* after its registration will be based on the list of *sge* counted by *num\_sges*. Post a successful registration RDMA operations can use this *mkey*, the hardware will scatter the data according to the pattern. The *mkey* should be used in a zero-based mode, the *addr* field in its *ibv\_sge* is an offset in the total data. Current implementation requires the IBV\_SEND\_INLINE option to be on in *ibv\_qp\_ex-\>wr\_flags* field. To be able to have more than 4 *num\_sge* entries, the QP should be created with a larger WQE size that may fit it. This should be done using the *max\_inline\_data* attribute of *struct ibv\_qp\_cap* upon its creation. In case *ibv\_qp\_ex-\>wr\_flags* turns on IBV\_SEND\_SIGNALED, the reported WC opcode will be MLX5DV\_WC\_UMR. Unregister the *mkey* to enable other pattern registration should be done via ibv\_post\_send with IBV\_WR\_LOCAL\_INV opcode.

- *RC* or *DCI* QPs

  *mlx5dv\_wr\_memcpy()*

  Builds a DMA memcpy work request to copy data of length *length* from *src\_addr* to *dest\_addr*. The copy operation will be done using the DMA MMO functionality of the device to copy data on PCI bus. The MLX5DV\_QP\_EX\_WITH\_MEMCPY flag in *mlx5dv\_qp\_init\_attr.send\_ops\_flags* needs to be set during QP creation. If the device or QP doesn’t support it then QP creation will fail. The maximum memcpy length that is supported by the device is reported in *mlx5dv\_context-\>max\_wr\_memcpy\_length*. A zero value in *mlx5dv\_context-\>max\_wr\_memcpy\_length* means the device doesn’t support memcpy operations. IBV\_SEND\_FENCE indicator should be used on a following send request which is dependent on *dest\_addr* of the memcpy operation. In case *ibv\_qp\_ex-\>wr\_flags* turns on IBV\_SEND\_SIGNALED, the reported WC opcode will be MLX5DV\_WC\_MEMCPY.

### Raw WQE builders

- *mlx5dv\_wr\_raw\_wqe()*

  It is used to build a custom work request (WQE) and post it on a normal QP. The caller needs to set all details of the WQE (except the “ctrl.wqe\_index” and “ctrl.signature” fields, which is the driver’s responsibility to set). The MLX5DV\_QP\_EX\_WITH\_RAW\_WQE flag in mlx5\_qp\_attr.send\_ops\_flags needs to be set.

  The wr\_flags are ignored as it’s the caller’s responsibility to set flags in WQE. No matter what the send opcode is, the work completion opcode for a raw WQE is IBV\_WC\_DRIVER2.

### QP Specific setters

- *DCI* QPs

  *mlx5dv\_wr\_set\_dc\_addr()* must be called to set the DCI WR properties. The destination address of the work is specified by *ah*, the remote DCT number is specified by *remote\_dctn* and the DC key is specified by *remote\_dc\_key*. This setter is available when the QP transport is DCI and send\_ops\_flags in struct ibv\_qp\_init\_attr\_ex is set. The available builders and setters for DCI QP are the same as RC QP. DCI QP created with MLX5DV\_QP\_INIT\_ATTR\_MASK\_DCI\_STREAMS can call *mlx5dv\_wr\_set\_dc\_addr\_stream()* to define the *stream\_id* of the operation to allow HW to choose one of the multiple concurrent DCI resources. Calls to *mlx5dv\_wr\_set\_dc\_addr()* are equivalent to using *stream\_id*=0

## EXAMPLE

-

  ```
  /* create DC QP type and specify the required send opcodes */
  attr_ex.qp_type = IBV_QPT_DRIVER;
  attr_ex.comp_mask |= IBV_QP_INIT_ATTR_SEND_OPS_FLAGS;
  attr_ex.send_ops_flags |= IBV_QP_EX_WITH_RDMA_WRITE;
  attr_dv.comp_mask |= MLX5DV_QP_INIT_ATTR_MASK_DC;
  attr_dv.dc_init_attr.dc_type = MLX5DV_DCTYPE_DCI;
  ibv_qp *qp = mlx5dv_create_qp(ctx, attr_ex, attr_dv);
  ibv_qp_ex *qpx = ibv_qp_to_qp_ex(qp);
  mlx5dv_qp_ex *mqpx = mlx5dv_qp_ex_from_ibv_qp_ex(qpx);
  ibv_wr_start(qpx);
  /* Use ibv_qp_ex object to set WR generic attributes */
  qpx->wr_id = my_wr_id_1;
  qpx->wr_flags = IBV_SEND_SIGNALED;
  ibv_wr_rdma_write(qpx, rkey, remote_addr_1);
  ibv_wr_set_sge(qpx, lkey, local_addr_1, length_1);
  /* Use mlx5 DC setter using mlx5dv_qp_ex object */
  mlx5dv_wr_set_wr_dc_addr(mqpx, ah, remote_dctn, remote_dc_key);
  ret = ibv_wr_complete(qpx);
  ```

## SEE ALSO

[**ibv\_post\_send**(3)](/ibv_post_send/3.md), **[ibv\_create\_qp\_ex(3)](/ibv_create_qp_ex/3.md)**, **[ibv\_wr\_post(3)](/ibv_wr_post/3.md)**, **[mlx5dv\_create\_mkey(3)](/mlx5dv_create_mkey/3.md)**.

## AUTHOR

Guy Levi ⟨guyle@mellanox.com⟩\\c

Mark Zhang ⟨markzhang@nvidia.com⟩\\c