--- name: "rte_ml_dev_stop" section: "3" platform: "linux" package: "dpdk-doc" version: "24.11.4-0+deb13u1" release: "Debian 13 (Trixie)" platforms: ["linux"] url: "https://new.carta.tech/rte_mldev.h" --- # rte\_ml\_dev\_stop(3) Part of [rte\_mldev.h(3)](/rte_mldev.h/3.md). ## NAME rte\_mldev.h ## SYNOPSIS `#include `\ `#include `\ `#include `\ ### Data Structures struct **rte\_ml\_dev\_info**\ struct **rte\_ml\_dev\_config**\ struct **rte\_ml\_dev\_qp\_conf**\ struct **rte\_ml\_buff\_seg**\ struct **rte\_ml\_op**\ struct **rte\_ml\_op\_error**\ struct **rte\_ml\_dev\_stats**\ struct **rte\_ml\_dev\_xstats\_map**\ struct **rte\_ml\_model\_params**\ struct **rte\_ml\_io\_info**\ struct **rte\_ml\_model\_info**\ ### Macros \#define **RTE\_ML\_STR\_MAX** 128\ ### Typedefs **typedef** void(\* **rte\_ml\_dev\_stop\_flush\_t**) (int16\_t dev\_id, uint16\_t qp\_id, struct **rte\_ml\_op** \*op)\ ### Enumerations enum **rte\_ml\_op\_status** { **RTE\_ML\_OP\_STATUS\_SUCCESS** = 0, **RTE\_ML\_OP\_STATUS\_NOT\_PROCESSED**, **RTE\_ML\_OP\_STATUS\_ERROR** }\ enum **rte\_ml\_dev\_xstats\_mode** { **RTE\_ML\_DEV\_XSTATS\_DEVICE**, **RTE\_ML\_DEV\_XSTATS\_MODEL** }\ enum **rte\_ml\_io\_type** { **RTE\_ML\_IO\_TYPE\_UNKNOWN** = 0, **RTE\_ML\_IO\_TYPE\_INT8**, **RTE\_ML\_IO\_TYPE\_UINT8**, **RTE\_ML\_IO\_TYPE\_INT16**, **RTE\_ML\_IO\_TYPE\_UINT16**, **RTE\_ML\_IO\_TYPE\_INT32**, **RTE\_ML\_IO\_TYPE\_UINT32**, **RTE\_ML\_IO\_TYPE\_INT64**, **RTE\_ML\_IO\_TYPE\_UINT64**, **RTE\_ML\_IO\_TYPE\_FP8**, **RTE\_ML\_IO\_TYPE\_FP16**, **RTE\_ML\_IO\_TYPE\_FP32**, **RTE\_ML\_IO\_TYPE\_BFLOAT16** }\ enum **rte\_ml\_io\_layout** { **RTE\_ML\_IO\_LAYOUT\_PACKED**, **RTE\_ML\_IO\_LAYOUT\_SPLIT** }\ ### Functions \_\_rte\_experimental int **rte\_ml\_dev\_init** (size\_t dev\_max)\ \_\_rte\_experimental uint16\_t **rte\_ml\_dev\_count** (void)\ \_\_rte\_experimental int **rte\_ml\_dev\_is\_valid\_dev** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_socket\_id** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_info\_get** (int16\_t dev\_id, struct **rte\_ml\_dev\_info** \*dev\_info)\ \_\_rte\_experimental int **rte\_ml\_dev\_configure** (int16\_t dev\_id, const struct **rte\_ml\_dev\_config** \*config)\ \_\_rte\_experimental uint16\_t **rte\_ml\_dev\_queue\_pair\_count** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_queue\_pair\_setup** (int16\_t dev\_id, uint16\_t queue\_pair\_id, const struct **rte\_ml\_dev\_qp\_conf** \*qp\_conf, int socket\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_start** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_stop** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_close** (int16\_t dev\_id)\ \_\_rte\_experimental uint16\_t **rte\_ml\_enqueue\_burst** (int16\_t dev\_id, uint16\_t qp\_id, struct **rte\_ml\_op** \*\*ops, uint16\_t nb\_ops)\ \_\_rte\_experimental uint16\_t **rte\_ml\_dequeue\_burst** (int16\_t dev\_id, uint16\_t qp\_id, struct **rte\_ml\_op** \*\*ops, uint16\_t nb\_ops)\ \_\_rte\_experimental int **rte\_ml\_op\_error\_get** (int16\_t dev\_id, struct **rte\_ml\_op** \*op, struct **rte\_ml\_op\_error** \*error)\ \_\_rte\_experimental int **rte\_ml\_dev\_stats\_get** (int16\_t dev\_id, struct **rte\_ml\_dev\_stats** \*stats)\ \_\_rte\_experimental void **rte\_ml\_dev\_stats\_reset** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_dev\_xstats\_names\_get** (int16\_t dev\_id, enum **rte\_ml\_dev\_xstats\_mode** mode, int32\_t model\_id, struct **rte\_ml\_dev\_xstats\_map** \*xstats\_map, uint32\_t size)\ \_\_rte\_experimental int **rte\_ml\_dev\_xstats\_by\_name\_get** (int16\_t dev\_id, const char \*name, uint16\_t \*stat\_id, uint64\_t \*value)\ \_\_rte\_experimental int **rte\_ml\_dev\_xstats\_get** (int16\_t dev\_id, enum **rte\_ml\_dev\_xstats\_mode** mode, int32\_t model\_id, const uint16\_t stat\_ids\[\], uint64\_t values\[\], uint16\_t nb\_ids)\ \_\_rte\_experimental int **rte\_ml\_dev\_xstats\_reset** (int16\_t dev\_id, enum **rte\_ml\_dev\_xstats\_mode** mode, int32\_t model\_id, const uint16\_t stat\_ids\[\], uint16\_t nb\_ids)\ \_\_rte\_experimental int **rte\_ml\_dev\_dump** (int16\_t dev\_id, FILE \*fd)\ \_\_rte\_experimental int **rte\_ml\_dev\_selftest** (int16\_t dev\_id)\ \_\_rte\_experimental int **rte\_ml\_model\_load** (int16\_t dev\_id, struct **rte\_ml\_model\_params** \*params, uint16\_t \*model\_id)\ \_\_rte\_experimental int **rte\_ml\_model\_unload** (int16\_t dev\_id, uint16\_t model\_id)\ \_\_rte\_experimental int **rte\_ml\_model\_start** (int16\_t dev\_id, uint16\_t model\_id)\ \_\_rte\_experimental int **rte\_ml\_model\_stop** (int16\_t dev\_id, uint16\_t model\_id)\ \_\_rte\_experimental int **rte\_ml\_model\_info\_get** (int16\_t dev\_id, uint16\_t model\_id, struct **rte\_ml\_model\_info** \*model\_info)\ \_\_rte\_experimental int **rte\_ml\_model\_params\_update** (int16\_t dev\_id, uint16\_t model\_id, void \*buffer)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_int8** (const void \*fp32, void \*i8, uint64\_t nb\_elements, float scale, int8\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_int8\_to\_float32** (const void \*i8, void \*fp32, uint64\_t nb\_elements, float scale, int8\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_uint8** (const void \*fp32, void \*ui8, uint64\_t nb\_elements, float scale, uint8\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_uint8\_to\_float32** (const void \*ui8, void \*fp32, uint64\_t nb\_elements, float scale, uint8\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_int16** (const void \*fp32, void \*i16, uint64\_t nb\_elements, float scale, int16\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_int16\_to\_float32** (const void \*i16, void \*fp32, uint64\_t nb\_elements, float scale, int16\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_uint16** (const void \*fp32, void \*ui16, uint64\_t nb\_elements, float scale, uint16\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_uint16\_to\_float32** (const void \*ui16, void \*fp32, uint64\_t nb\_elements, float scale, uint16\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_int32** (const void \*fp32, void \*i32, uint64\_t nb\_elements, float scale, int32\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_int32\_to\_float32** (const void \*i32, void \*fp32, uint64\_t nb\_elements, float scale, int32\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_uint32** (const void \*fp32, void \*ui32, uint64\_t nb\_elements, float scale, uint32\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_uint32\_to\_float32** (const void \*ui32, void \*fp32, uint64\_t nb\_elements, float scale, uint32\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_int64** (const void \*fp32, void \*i64, uint64\_t nb\_elements, float scale, int64\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_int64\_to\_float32** (const void \*i64, void \*fp32, uint64\_t nb\_elements, float scale, int64\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_uint64** (const void \*fp32, void \*ui64, uint64\_t nb\_elements, float scale, uint64\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_uint64\_to\_float32** (const void \*ui64, void \*fp32, uint64\_t nb\_elements, float scale, uint64\_t zero\_point)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_float16** (const void \*fp32, void \*fp16, uint64\_t nb\_elements)\ \_\_rte\_experimental int **rte\_ml\_io\_float16\_to\_float32** (const void \*fp16, void \*fp32, uint64\_t nb\_elements)\ \_\_rte\_experimental int **rte\_ml\_io\_float32\_to\_bfloat16** (const void \*fp32, void \*bf16, uint64\_t nb\_elements)\ \_\_rte\_experimental int **rte\_ml\_io\_bfloat16\_to\_float32** (const void \*bf16, void \*fp32, uint64\_t nb\_elements)\ \_\_rte\_experimental int **rte\_ml\_io\_quantize** (int16\_t dev\_id, uint16\_t model\_id, struct **rte\_ml\_buff\_seg** \*\*dbuffer, struct **rte\_ml\_buff\_seg** \*\*qbuffer)\ \_\_rte\_experimental int **rte\_ml\_io\_dequantize** (int16\_t dev\_id, uint16\_t model\_id, struct **rte\_ml\_buff\_seg** \*\*qbuffer, struct **rte\_ml\_buff\_seg** \*\*dbuffer)\ \_\_rte\_experimental struct **rte\_mempool** \* **rte\_ml\_op\_pool\_create** (const char \*name, unsigned int nb\_elts, unsigned int cache\_size, uint16\_t user\_size, int socket\_id)\ \_\_rte\_experimental void **rte\_ml\_op\_pool\_free** (struct **rte\_mempool** \*mempool)\ ## Detailed Description **Warning** **EXPERIMENTAL:** All functions in this file may be changed or removed without prior notice. ML (Machine Learning) device API. The ML framework is built on the following model: ``` +-----------------+ rte_ml_[en|de]queue_burst() | | | | Machine o------+ +--------+ | | Learning | | | queue | | +------+ | Inference o------+-----o |<===o===>|Core 0| | Engine | | | pair 0 | +------+ | o----+ | +--------+ | | | | +-----------------+ | | +--------+ ^ | | | queue | +------+ | | +-----o |<=======>|Core 1| | | | pair 1 | +------+ | | +--------+ +--------+--------+ | | +-------------+ | | +--------+ | | Model 0 | | | | queue | +------+ | +-------------+ | +-------o |<=======>|Core N| | +-------------+ | | pair N | +------+ | | Model 1 | | +--------+ | +-------------+ | | +-------------+ |<------> rte_ml_model_load() | | Model .. | |-------> rte_ml_model_info_get() | +-------------+ |<------- rte_ml_model_start() | +-------------+ |<------- rte_ml_model_stop() | | Model N | |<------- rte_ml_model_params_update() | +-------------+ |<------- rte_ml_model_unload() +-----------------+ ``` ML Device: A hardware or software-based implementation of ML device API for running inferences using a pre-trained ML model. ML Model: An ML model is an algorithm trained over a dataset. A model consists of procedure/algorithm and data/pattern required to make predictions on live data. Once the model is created and trained outside of the DPDK scope, the model can be loaded via **rte\_ml\_model\_load()** and then start it using **rte\_ml\_model\_start()** API. The **rte\_ml\_model\_params\_update()** can be used to update the model parameters such as weight and bias without unloading the model using **rte\_ml\_model\_unload()**. ML Inference: ML inference is the process of feeding data to the model via **rte\_ml\_enqueue\_burst()** API and use **rte\_ml\_dequeue\_burst()** API to get the calculated outputs/predictions from the started model. In all functions of the ML device API, the ML device is designated by an integer \>= 0 named as device identifier *dev\_id*. The functions exported by the ML device API to setup a device designated by its device identifier must be invoked in the following order: ``` - rte_ml_dev_configure() - rte_ml_dev_queue_pair_setup() - rte_ml_dev_start() ``` A model is required to run the inference operations with the user specified inputs. Application needs to invoke the ML model API in the following order before queueing inference jobs. ``` - rte_ml_model_load() - rte_ml_model_start() ``` A model can be loaded on a device only after the device has been configured and can be started or stopped only after a device has been started. The **rte\_ml\_model\_info\_get()** API is provided to retrieve the information related to the model. The information would include the shape and type of input and output required for the inference. Data quantization and dequantization is one of the main aspects in ML domain. This involves conversion of input data from a higher precision to a lower precision data type and vice-versa for the output. APIs are provided to enable quantization through **rte\_ml\_io\_quantize()** and dequantization through **rte\_ml\_io\_dequantize()**. These APIs have the capability to handle input and output buffers holding data for multiple batches. Two utility APIs rte\_ml\_io\_input\_size\_get() and rte\_ml\_io\_output\_size\_get() can used to get the size of quantized and de-quantized multi-batch input and output buffers. User can optionally update the model parameters with **rte\_ml\_model\_params\_update()** after invoking **rte\_ml\_model\_stop()** API on a given model ID. The application can invoke, in any order, the functions exported by the ML API to enqueue inference jobs and dequeue inference response. If the application wants to change the device configuration (i.e., call **rte\_ml\_dev\_configure()** or **rte\_ml\_dev\_queue\_pair\_setup()**), then application must stop the device using **rte\_ml\_dev\_stop()** API. Likewise, if model parameters need to be updated then the application must call **rte\_ml\_model\_stop()** followed by **rte\_ml\_model\_params\_update()** API for the given model. The application does not need to call **rte\_ml\_dev\_stop()** API for any model re-configuration such as **rte\_ml\_model\_params\_update()**, **rte\_ml\_model\_unload()** etc. Once the device is in the start state after invoking **rte\_ml\_dev\_start()** API and the model is in start state after invoking **rte\_ml\_model\_start()** API, then the application can call **rte\_ml\_enqueue\_burst()** and **rte\_ml\_dequeue\_burst()** API on the destined device and model ID. Finally, an application can close an ML device by invoking the **rte\_ml\_dev\_close()** function. Typical application utilisation of the ML API will follow the following programming flow. - • **rte\_ml\_dev\_configure()** - • **rte\_ml\_dev\_queue\_pair\_setup()** - • **rte\_ml\_model\_load()** - • **rte\_ml\_dev\_start()** - • **rte\_ml\_model\_start()** - • **rte\_ml\_model\_info\_get()** - • **rte\_ml\_enqueue\_burst()** - • **rte\_ml\_dequeue\_burst()** - • **rte\_ml\_model\_stop()** - • **rte\_ml\_model\_unload()** - • **rte\_ml\_dev\_stop()** - • **rte\_ml\_dev\_close()** Regarding multi-threading, by default, all the functions of the ML Device API exported by a PMD are lock-free functions which assume to not be invoked in parallel on different logical cores on the same target object. For instance, the dequeue function of a poll mode driver cannot be invoked in parallel on two logical cores to operate on same queue pair. Of course, this function can be invoked in parallel by different logical core on different queue pair. It is the responsibility of the user application to enforce this rule. Definition in file **rte\_mldev.h**. ## Macro Definition Documentation ### #define RTE\_ML\_STR\_MAX 128 Maximum length of name string Definition at line **153** of file **rte\_mldev.h**. ## Typedef Documentation ### typedef void(\* rte\_ml\_dev\_stop\_flush\_t) (int16\_t dev\_id, uint16\_t qp\_id, struct rte\_ml\_op \*op) Callback function called during **rte\_ml\_dev\_stop()**, invoked once per flushed ML op Definition at line **302** of file **rte\_mldev.h**. ## Enumeration Type Documentation ### enum rte\_ml\_op\_status Status of ML operation **Enumerator** - *RTE\_ML\_OP\_STATUS\_SUCCESS* *Operation completed successfully* - *RTE\_ML\_OP\_STATUS\_NOT\_PROCESSED* *Operation has not yet been processed by the device.* - *RTE\_ML\_OP\_STATUS\_ERROR* *Operation completed with error. Application can invoke* **rte\_ml\_op\_error\_get()** *to get PMD specific error code if needed.* *Definition at line* **404** *of file* **rte\_mldev.h***.* ### enum rte\_ml\_dev\_xstats\_mode *Selects the component of the mldev to retrieve statistics from.* **Enumerator** - *RTE\_ML\_DEV\_XSTATS\_DEVICE* *Device xstats* - *RTE\_ML\_DEV\_XSTATS\_MODEL* *Model xstats* *Definition at line* **632** *of file* **rte\_mldev.h***.* ### enum rte\_ml\_io\_type *Input and output data types. ML models can operate on reduced precision datatypes to achieve better power efficiency, lower network latency and lower memory footprint. This enum is used to represent the lower precision integer and floating point types used by ML models.* **Enumerator** - *RTE\_ML\_IO\_TYPE\_UNKNOWN* *Invalid or unknown type* - *RTE\_ML\_IO\_TYPE\_INT8* *8-bit integer* - *RTE\_ML\_IO\_TYPE\_UINT8* *8-bit unsigned integer* - *RTE\_ML\_IO\_TYPE\_INT16* *16-bit integer* - *RTE\_ML\_IO\_TYPE\_UINT16* *16-bit unsigned integer* - *RTE\_ML\_IO\_TYPE\_INT32* *32-bit integer* - *RTE\_ML\_IO\_TYPE\_UINT32* *32-bit unsigned integer* - *RTE\_ML\_IO\_TYPE\_INT64* *32-bit integer* - *RTE\_ML\_IO\_TYPE\_UINT64* *32-bit unsigned integer* - *RTE\_ML\_IO\_TYPE\_FP8* *8-bit floating point number* - *RTE\_ML\_IO\_TYPE\_FP16* *IEEE 754 16-bit floating point number* - *RTE\_ML\_IO\_TYPE\_FP32* *IEEE 754 32-bit floating point number* - *RTE\_ML\_IO\_TYPE\_BFLOAT16* *16-bit brain floating point number.* *Definition at line* **875** *of file* **rte\_mldev.h***.* ### enum rte\_ml\_io\_layout *ML I/O buffer layout* **Enumerator** - *RTE\_ML\_IO\_LAYOUT\_PACKED* *All inputs for the model should packed in a single buffer with no padding between individual inputs. The buffer is expected to be aligned to* **rte\_ml\_dev\_info::align\_size***.* *When I/O segmentation is supported by the device, the packed data can be split into multiple segments. In this case, each segment is expected to be aligned to* **rte\_ml\_dev\_info::align\_size** *Same applies to output.* **See also** *struct* **rte\_ml\_dev\_info::max\_segments** - *RTE\_ML\_IO\_LAYOUT\_SPLIT* *Each input for the model should be stored as separate buffers and each input should be aligned to* **rte\_ml\_dev\_info::align\_size***.* *When I/O segmentation is supported, each input can be split into multiple segments. In this case, each segment is expected to be aligned to* **rte\_ml\_dev\_info::align\_size** *Same applies to output.* **See also** *struct* **rte\_ml\_dev\_info::max\_segments** *Definition at line* **905** *of file* **rte\_mldev.h***.* ## Function Documentation ### \_\_rte\_experimental int rte\_ml\_dev\_init (size\_t dev\_max) *Maximum number of devices if* **rte\_ml\_dev\_init()** *is not called. Initialize the device array before probing devices. If not called, the first device probed would initialize the array to a size of RTE\_MLDEV\_DEFAULT\_MAX.* **Parameters** *dev\_max Maximum number of devices.* **Returns** *0 on success, -rte\_errno otherwise:* - • *ENOMEM if out of memory* - • *EINVAL if 0 size* - • *EBUSY if already initialized* ### \_\_rte\_experimental uint16\_t rte\_ml\_dev\_count (void) *Get the total number of ML devices that have been successfully initialised.* **Returns** - • *The total number of usable ML devices.* ### \_\_rte\_experimental int rte\_ml\_dev\_is\_valid\_dev (int16\_t dev\_id) *Check if the device is in ready state.* **Parameters** *dev\_id The identifier of the device.* **Returns** - • *0 if device state is not in ready state.* - • *1 if device state is ready state.* ### \_\_rte\_experimental int rte\_ml\_dev\_socket\_id (int16\_t dev\_id) *Return the NUMA socket to which a device is connected.* **Parameters** *dev\_id The identifier of the device.* **Returns** - • *The NUMA socket id to which the device is connected* - • *0 If the socket could not be determined.* - • *-EINVAL: if the dev\_id value is not valid.* ### \_\_rte\_experimental int rte\_ml\_dev\_info\_get (int16\_t dev\_id, struct rte\_ml\_dev\_info \* dev\_info) *Retrieve the information of the device.* **Parameters** *dev\_id The identifier of the device.* \ *dev\_info A pointer to a structure of type* **rte\_ml\_dev\_info to be filled with the info of the device.** **Returns** - • **0: Success, driver updates the information of the ML device** - • **\< 0: Error code returned by the driver info get function.** ### \_\_rte\_experimental int rte\_ml\_dev\_configure (int16\_t dev\_id, const struct rte\_ml\_dev\_config \* config) **Configure an ML device.** **This function must be invoked first before any other function in the API.** **ML Device can be re-configured, when in a stopped state. Device cannot be re-configured after rte\_ml\_dev\_close() is called.** **The caller may use rte\_ml\_dev\_info\_get() to get the capability of each resources available for this ML device.** **Parameters** *dev\_id* **The identifier of the device to configure.** \ *config* **The ML device configuration structure.** **Returns** - • **0: Success, device configured.** - • **\< 0: Error code returned by the driver configuration function.** ### \_\_rte\_experimental uint16\_t rte\_ml\_dev\_queue\_pair\_count (int16\_t dev\_id) **Get the number of queue pairs on a specific ML device.** **Parameters** *dev\_id* **The identifier of the device.** **Returns** - • **The number of configured queue pairs.** ### \_\_rte\_experimental int rte\_ml\_dev\_queue\_pair\_setup (int16\_t dev\_id, uint16\_t queue\_pair\_id, const struct rte\_ml\_dev\_qp\_conf \* qp\_conf, int socket\_id) **Set up a queue pair for a device. This should only be called when the device is stopped.** **Parameters** *dev\_id* **The identifier of the device.** \ *queue\_pair\_id* **The index of the queue pairs to set up. The value must be in the range \[0, nb\_queue\_pairs - 1\] previously supplied to rte\_ml\_dev\_configure().** \ *qp\_conf* **The pointer to the configuration data to be used for the queue pair.** \ *socket\_id* **The** *socket\_id* **argument is the socket identifier in case of NUMA. The value can be** *SOCKET\_ID\_ANY* **if there is no NUMA constraint for the memory allocated for the queue pair.** **Returns** - • **0: Success, queue pair correctly set up.** - • **\< 0: Queue pair configuration failed.** ### \_\_rte\_experimental int rte\_ml\_dev\_start (int16\_t dev\_id) **Start an ML device.** **The device start step consists of setting the configured features and enabling the ML device to accept inference jobs.** **Parameters** *dev\_id* **The identifier of the device.** **Returns** - • **0: Success, device started.** - • **\<0: Error code of the driver device start function.** ### \_\_rte\_experimental int rte\_ml\_dev\_stop (int16\_t dev\_id) **Stop an ML device. A stopped device cannot accept inference jobs. The device can be restarted with a call to rte\_ml\_dev\_start().** **Parameters** *dev\_id* **The identifier of the device.** **Returns** - • **0: Success, device stopped.** - • **\<0: Error code of the driver device stop function.** ### \_\_rte\_experimental int rte\_ml\_dev\_close (int16\_t dev\_id) **Close an ML device. The device cannot be restarted!** **Parameters** *dev\_id* **The identifier of the device.** **Returns** - • **0 on successfully closing device.** - • **\<0 on failure to close device.** ### \_\_rte\_experimental uint16\_t rte\_ml\_enqueue\_burst (int16\_t dev\_id, uint16\_t qp\_id, struct rte\_ml\_op \*\* ops, uint16\_t nb\_ops) **Enqueue a burst of ML inferences for processing on an ML device.** **The rte\_ml\_enqueue\_burst() function is invoked to place ML inference operations on the queue** *qp\_id* **of the device designated by its** *dev\_id***.** **The** *nb\_ops* **parameter is the number of inferences to process which are supplied in the** *ops* **array of rte\_ml\_op structures.** **The rte\_ml\_enqueue\_burst() function returns the number of inferences it actually enqueued for processing. A return value equal to** *nb\_ops* **means that all packets have been enqueued.** **Parameters** *dev\_id* **The identifier of the device.** \ *qp\_id* **The index of the queue pair which inferences are to be enqueued for processing. The value must be in the range \[0, nb\_queue\_pairs - 1\] previously supplied to** *rte\_ml\_dev\_configure***.** \ *ops* **The address of an array of** *nb\_ops* **pointers to rte\_ml\_op structures which contain the ML inferences to be processed.** \ *nb\_ops* **The number of operations to process.** **Returns** **The number of inference operations actually enqueued to the ML device. The return value can be less than the value of the** *nb\_ops* **parameter when the ML device queue is full or if invalid parameters are specified in a rte\_ml\_op.** ### \_\_rte\_experimental uint16\_t rte\_ml\_dequeue\_burst (int16\_t dev\_id, uint16\_t qp\_id, struct rte\_ml\_op \*\* ops, uint16\_t nb\_ops) **Dequeue a burst of processed ML inferences operations from a queue on the ML device. The dequeued operations are stored in rte\_ml\_op structures whose pointers are supplied in the** *ops* **array.** **The rte\_ml\_dequeue\_burst() function returns the number of inferences actually dequeued, which is the number of rte\_ml\_op data structures effectively supplied into the** *ops* **array.** **A return value equal to** *nb\_ops* **indicates that the queue contained at least nb\_ops\* operations, and this is likely to signify that other processed operations remain in the devices output queue. Application implementing a 'retrieve as many processed operations as possible' policy can check this specific case and keep invoking the rte\_ml\_dequeue\_burst() function until a value less than** *nb\_ops* **is returned.** **The rte\_ml\_dequeue\_burst() function does not provide any error notification to avoid the corresponding overhead.** **Parameters** *dev\_id* **The identifier of the device.** \ *qp\_id* **The index of the queue pair from which to retrieve processed packets. The value must be in the range \[0, nb\_queue\_pairs - 1\] previously supplied to rte\_ml\_dev\_configure().** \ *ops* **The address of an array of pointers to rte\_ml\_op structures that must be large enough to store** *nb\_ops* **pointers in it.** \ *nb\_ops* **The maximum number of inferences to dequeue.** **Returns** **The number of operations actually dequeued, which is the number of pointers to rte\_ml\_op structures effectively supplied to the** *ops* **array.** ### \_\_rte\_experimental int rte\_ml\_op\_error\_get (int16\_t dev\_id, struct rte\_ml\_op \* op, struct rte\_ml\_op\_error \* error) **Get PMD specific error information for an ML op.** **When an ML operation completed with RTE\_ML\_OP\_STATUS\_ERROR as status, This API allows to get PMD specific error details.** **Parameters** *dev\_id* **Device identifier** \ *op* **Handle of ML operation** \ *error* **Address of structure rte\_ml\_op\_error to be filled** **Returns** - • **Returns 0 on success** - • **Returns negative value on failure** ### \_\_rte\_experimental int rte\_ml\_dev\_stats\_get (int16\_t dev\_id, struct rte\_ml\_dev\_stats \* stats) **Retrieve the general I/O statistics of a device.** **Parameters** *dev\_id* **The identifier of the device.** \ *stats* **Pointer to structure to where statistics will be copied. On error, this location may or may not have been modified.** **Returns** - • **0 on success** - • **-EINVAL: If invalid parameter pointer is provided.** ### \_\_rte\_experimental void rte\_ml\_dev\_stats\_reset (int16\_t dev\_id) **Reset the statistics of a device.** **Parameters** *dev\_id* **The identifier of the device.** ### \_\_rte\_experimental int rte\_ml\_dev\_xstats\_names\_get (int16\_t dev\_id, enum rte\_ml\_dev\_xstats\_mode mode, int32\_t model\_id, struct rte\_ml\_dev\_xstats\_map \* xstats\_map, uint32\_t size) **Retrieve names of extended statistics of an ML device.** **Parameters** *dev\_id* **The identifier of the device.** \ *mode* **Mode of statistics to retrieve. Choices include the device statistics and model statistics.** \ *model\_id* **Used to specify the model number in model mode, and is ignored in device mode.** \ *xstats\_map* **Block of memory to insert names and ids into. Must be at least size in capacity. If set to NULL, function returns required capacity. The id values returned can be passed to** *rte\_ml\_dev\_xstats\_get* **to select statistics.** \ *size* **Capacity of xstats\_names (number of xstats\_map).** **Returns** - • **Positive value lower or equal to size: success. The return value is the number of entries filled in the stats table.** - • **Positive value higher than size: error, the given statistics table is too small. The return value corresponds to the size that should be given to succeed. The entries in the table are not valid and shall not be used by the caller.** - • **Negative value on error: -ENODEV for invalid** *dev\_id***. -EINVAL for invalid mode, model parameters. -ENOTSUP if the device doesn't support this function.** ### \_\_rte\_experimental int rte\_ml\_dev\_xstats\_by\_name\_get (int16\_t dev\_id, const char \* name, uint16\_t \* stat\_id, uint64\_t \* value) **Retrieve the value of a single stat by requesting it by name.** **Parameters** *dev\_id* **The identifier of the device.** \ *name* **Name of stat name to retrieve.** \ *stat\_id* **If non-NULL, the numerical id of the stat will be returned, so that further requests for the stat can be got using rte\_ml\_dev\_xstats\_get, which will be faster as it doesn't need to scan a list of names for the stat. If the stat cannot be found, the id returned will be (unsigned)-1.** \ *value* **Value of the stat to be returned.** **Returns** - • **Zero: No error.** - • **Negative value: -EINVAL if stat not found, -ENOTSUP if not supported.** ### \_\_rte\_experimental int rte\_ml\_dev\_xstats\_get (int16\_t dev\_id, enum rte\_ml\_dev\_xstats\_mode mode, int32\_t model\_id, const uint16\_t stat\_ids\[\], uint64\_t values\[\], uint16\_t nb\_ids) **Retrieve extended statistics of an ML device.** **Parameters** *dev\_id* **The identifier of the device.** \ *mode* **Mode of statistics to retrieve. Choices include the device statistics and model statistics.** \ *model\_id* **Used to specify the model id in model mode, and is ignored in device mode.** \ *stat\_ids* **ID numbers of the stats to get. The ids can be got from the stat position in the stat list from rte\_ml\_dev\_xstats\_names\_get(), or by using rte\_ml\_dev\_xstats\_by\_name\_get().** \ *values* **Values for each stats request by ID.** \ *nb\_ids* **Number of stats requested.** **Returns** - • **Positive value: number of stat entries filled into the values array** - • **Negative value on error: -ENODEV for invalid** *dev\_id***. -EINVAL for invalid mode, model id or stat id parameters. -ENOTSUP if the device doesn't support this function.** ### \_\_rte\_experimental int rte\_ml\_dev\_xstats\_reset (int16\_t dev\_id, enum rte\_ml\_dev\_xstats\_mode mode, int32\_t model\_id, const uint16\_t stat\_ids\[\], uint16\_t nb\_ids) **Reset the values of the xstats of the selected component in the device.** **Parameters** *dev\_id* **The identifier of the device.** \ *mode* **Mode of the statistics to reset. Choose from device or model.** \ *model\_id* **Model stats to reset. 0 and positive values select models, while -1 indicates all models.** \ *stat\_ids* **Selects specific statistics to be reset. When NULL, all statistics selected by** *mode* **will be reset. If non-NULL, must point to array of at least** *nb\_ids* **size.** \ *nb\_ids* **The number of ids available from the** *ids* **array. Ignored when ids is NULL.** **Returns** - • **Zero: successfully reset the statistics.** - • **Negative value: -EINVAL invalid parameters, -ENOTSUP if not supported.** ### \_\_rte\_experimental int rte\_ml\_dev\_dump (int16\_t dev\_id, FILE \* fd) **Dump internal information about** *dev\_id* **to the FILE\* provided in** *fd***.** **Parameters** *dev\_id* **The identifier of the device.** \ *fd* **A pointer to a file for output.** **Returns** - • **0: on success.** - • **\<0: on failure.** ### \_\_rte\_experimental int rte\_ml\_dev\_selftest (int16\_t dev\_id) **Trigger the ML device self test.** **Parameters** *dev\_id* **The identifier of the device.** **Returns** - • **0: Selftest successful.** - • **-ENOTSUP: if the device doesn't support selftest.** - • **other values \< 0 on failure.** ### \_\_rte\_experimental int rte\_ml\_model\_load (int16\_t dev\_id, struct rte\_ml\_model\_params \* params, uint16\_t \* model\_id) **Load an ML model to the device.** **Load an ML model to the device with parameters requested in the structure rte\_ml\_model\_params.** **Parameters** *dev\_id* **The identifier of the device.** \ *params* **Parameters for the model to be loaded.** \ *model\_id* **Identifier of the model loaded.** **Returns** - • **0: Success, Model loaded.** - • **\< 0: Failure, Error code of the model load driver function.** ### \_\_rte\_experimental int rte\_ml\_model\_unload (int16\_t dev\_id, uint16\_t model\_id) **Unload an ML model from the device.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier of the model to be unloaded.** **Returns** - • **0: Success, Model unloaded.** - • **\< 0: Failure, Error code of the model unload driver function.** ### \_\_rte\_experimental int rte\_ml\_model\_start (int16\_t dev\_id, uint16\_t model\_id) **Start an ML model for the given device ID.** **Start an ML model to accept inference requests.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier of the model to be started.** **Returns** - • **0: Success, Model loaded.** - • **\< 0: Failure, Error code of the model start driver function.** ### \_\_rte\_experimental int rte\_ml\_model\_stop (int16\_t dev\_id, uint16\_t model\_id) **Stop an ML model for the given device ID.** **Model stop would disable the ML model to be used for inference jobs. All inference jobs must have been completed before model stop is attempted.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier of the model to be stopped.** **Returns** - • **0: Success, Model unloaded.** - • **\< 0: Failure, Error code of the model stop driver function.** ### \_\_rte\_experimental int rte\_ml\_model\_info\_get (int16\_t dev\_id, uint16\_t model\_id, struct rte\_ml\_model\_info \* model\_info) **Get ML model information.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier for the model created** \ *model\_info* **Pointer to a model info structure** **Returns** - • **Returns 0 on success** - • **Returns negative value on failure** ### \_\_rte\_experimental int rte\_ml\_model\_params\_update (int16\_t dev\_id, uint16\_t model\_id, void \* buffer) **Update the model parameters without unloading model.** **Update model parameters such as weights and bias without unloading the model. rte\_ml\_model\_stop() must be called before invoking this API.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier for the model created** \ *buffer* **Pointer to the model weights and bias buffer. Size of the buffer is equal to wb\_size returned in rte\_ml\_model\_info.** **Returns** - • **Returns 0 on success** - • **Returns negative value on failure** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_int8 (const void \* fp32, void \* i8, uint64\_t nb\_elements, float scale, int8\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to signed 8-bit integer format (INT8).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *i8* **Output buffer to store INT8 numbers. Size of buffer is equal to (nb\_elements \* 1) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_int8\_to\_float32 (const void \* i8, void \* fp32, uint64\_t nb\_elements, float scale, int8\_t zero\_point) **Convert a buffer containing numbers in signed 8-bit integer format (INT8) to single precision floating format (float32).** **Parameters** *i8* **Input buffer containing INT8 numbers. Size of buffer is equal to (nb\_elements \* 1) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_uint8 (const void \* fp32, void \* ui8, uint64\_t nb\_elements, float scale, uint8\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to unsigned 8-bit integer format (UINT8).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *ui8* **Output buffer to store UINT8 numbers. Size of buffer is equal to (nb\_elements \* 1) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_uint8\_to\_float32 (const void \* ui8, void \* fp32, uint64\_t nb\_elements, float scale, uint8\_t zero\_point) **Convert a buffer containing numbers in unsigned 8-bit integer format (UINT8) to single precision floating format (float32).** **Parameters** *ui8* **Input buffer containing UINT8 numbers. Size of buffer is equal to (nb\_elements \* 1) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_int16 (const void \* fp32, void \* i16, uint64\_t nb\_elements, float scale, int16\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to signed 16-bit integer format (INT16).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *i16* **Output buffer to store INT16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_int16\_to\_float32 (const void \* i16, void \* fp32, uint64\_t nb\_elements, float scale, int16\_t zero\_point) **Convert a buffer containing numbers in signed 16-bit integer format (INT16) to single precision floating format (float32).** **Parameters** *i16* **Input buffer containing INT16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_uint16 (const void \* fp32, void \* ui16, uint64\_t nb\_elements, float scale, uint16\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to unsigned 16-bit integer format (UINT16).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *ui16* **Output buffer to store UINT16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_uint16\_to\_float32 (const void \* ui16, void \* fp32, uint64\_t nb\_elements, float scale, uint16\_t zero\_point) **Convert a buffer containing numbers in unsigned 16-bit integer format (UINT16) to single precision floating format (float32).** **Parameters** *ui16* **Input buffer containing UINT16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_int32 (const void \* fp32, void \* i32, uint64\_t nb\_elements, float scale, int32\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to signed 32-bit integer format (INT32).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *i32* **Output buffer to store INT32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_int32\_to\_float32 (const void \* i32, void \* fp32, uint64\_t nb\_elements, float scale, int32\_t zero\_point) **Convert a buffer containing numbers in signed 32-bit integer format (INT32) to single precision floating format (float32).** **Parameters** *i32* **Input buffer containing INT32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_uint32 (const void \* fp32, void \* ui32, uint64\_t nb\_elements, float scale, uint32\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to unsigned 32-bit integer format (UINT32).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *ui32* **Output buffer to store UINT32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_uint32\_to\_float32 (const void \* ui32, void \* fp32, uint64\_t nb\_elements, float scale, uint32\_t zero\_point) **Convert a buffer containing numbers in unsigned 32-bit integer format (UINT32) to single precision floating format (float32).** **Parameters** *ui32* **Input buffer containing UINT32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_int64 (const void \* fp32, void \* i64, uint64\_t nb\_elements, float scale, int64\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to signed 64-bit integer format (INT64).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *i64* **Output buffer to store INT64 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_int64\_to\_float32 (const void \* i64, void \* fp32, uint64\_t nb\_elements, float scale, int64\_t zero\_point) **Convert a buffer containing numbers in signed 64-bit integer format (INT64) to single precision floating format (float32).** **Parameters** *i64* **Input buffer containing INT64 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_uint64 (const void \* fp32, void \* ui64, uint64\_t nb\_elements, float scale, uint64\_t zero\_point) **Convert a buffer containing numbers in single precision floating format (float32) to unsigned 64-bit integer format (UINT64).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *ui64* **Output buffer to store UINT64 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_uint64\_to\_float32 (const void \* ui64, void \* fp32, uint64\_t nb\_elements, float scale, uint64\_t zero\_point) **Convert a buffer containing numbers in unsigned 64-bit integer format (UINT64) to single precision floating format (float32).** **Parameters** *ui64* **Input buffer containing UINT64 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** \ *scale* **Scale factor for conversion.** \ *zero\_point* **Zero point for conversion.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_float16 (const void \* fp32, void \* fp16, uint64\_t nb\_elements) **Convert a buffer containing numbers in single precision floating format (float32) to half precision floating point format (FP16).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \*4) bytes.** \ *fp16* **Output buffer to store float16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *nb\_elements* **Number of elements in the buffer.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float16\_to\_float32 (const void \* fp16, void \* fp32, uint64\_t nb\_elements) **Convert a buffer containing numbers in half precision floating format (FP16) to single precision floating point format (float32).** **Parameters** *fp16* **Input buffer containing float16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_float32\_to\_bfloat16 (const void \* fp32, void \* bf16, uint64\_t nb\_elements) **Convert a buffer containing numbers in single precision floating format (float32) to brain floating point format (bfloat16).** **Parameters** *fp32* **Input buffer containing float32 numbers. Size of buffer is equal to (nb\_elements \*4) bytes.** \ *bf16* **Output buffer to store bfloat16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *nb\_elements* **Number of elements in the buffer.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_bfloat16\_to\_float32 (const void \* bf16, void \* fp32, uint64\_t nb\_elements) **Convert a buffer containing numbers in brain floating point format (bfloat16) to single precision floating point format (float32).** **Parameters** *bf16* **Input buffer containing bfloat16 numbers. Size of buffer is equal to (nb\_elements \* 2) bytes.** \ *fp32* **Output buffer to store float32 numbers. Size of buffer is equal to (nb\_elements \* 4) bytes.** \ *nb\_elements* **Number of elements in the buffer.** **Returns** - • **0, Success.** - • **\< 0, Error code on failure.** ### \_\_rte\_experimental int rte\_ml\_io\_quantize (int16\_t dev\_id, uint16\_t model\_id, struct rte\_ml\_buff\_seg \*\* dbuffer, struct rte\_ml\_buff\_seg \*\* qbuffer) **Quantize input data.** **Quantization converts data from a higher precision types to a lower precision types to improve the throughput and efficiency of the model execution with minimal loss of accuracy. Types of dequantized data and quantized data are specified by the model.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier for the model** \ *dbuffer* **Address of dequantized input data** \ *qbuffer* **Address of quantized input data** **Returns** - • **Returns 0 on success** - • **Returns negative value on failure** ### \_\_rte\_experimental int rte\_ml\_io\_dequantize (int16\_t dev\_id, uint16\_t model\_id, struct rte\_ml\_buff\_seg \*\* qbuffer, struct rte\_ml\_buff\_seg \*\* dbuffer) **Dequantize output data.** **Dequantization converts data from a lower precision type to a higher precision type. Types of quantized data and dequantized are specified by the model.** **Parameters** *dev\_id* **The identifier of the device.** \ *model\_id* **Identifier for the model** \ *qbuffer* **Address of quantized output data** \ *dbuffer* **Address of dequantized output data** **Returns** - • **Returns 0 on success** - • **Returns negative value on failure** ### \_\_rte\_experimental struct rte\_mempool \* rte\_ml\_op\_pool\_create (const char \* name, unsigned int nb\_elts, unsigned int cache\_size, uint16\_t user\_size, int socket\_id) **Create an ML operation pool** **Parameters** *name* **ML operations pool name** \ *nb\_elts* **Number of elements in pool** \ *cache\_size* **Number of elements to cache on lcore, see** *rte\_mempool\_create* **for further details about cache size** \ *user\_size* **Size of private data to allocate for user with each operation** \ *socket\_id* **Socket to identifier allocate memory on** **Returns** - • **On success pointer to mempool** - • **On failure NULL** ### \_\_rte\_experimental void rte\_ml\_op\_pool\_free (struct rte\_mempool \* mempool) **Free an ML operation pool** **Parameters** *mempool* **A pointer to the mempool structure. If NULL then, the function does nothing.** ## Author **Generated automatically by Doxygen for DPDK from the source code.**