--- name: "libpmemkv" section: "3" platform: "linux" package: "libpmemkv-dev" version: "1.4-1" release: "Debian 11 (Bullseye)" platforms: ["linux"] url: "https://new.carta.tech/libpmemkv" --- # libpmemkv(3) Key/Value Datastore for Persistent Memory ## NAME **pmemkv** - Key/Value Datastore for Persistent Memory ## SYNOPSIS - ``` #include typedef int pmemkv_get_kv_callback(const char *key, size_t keybytes, const char *value, size_t valuebytes, void *arg); typedef void pmemkv_get_v_callback(const char *value, size_t valuebytes, void *arg); int pmemkv_open(const char *engine, pmemkv_config *config, pmemkv_db **db); void pmemkv_close(pmemkv_db *kv); int pmemkv_count_all(pmemkv_db *db, size_t *cnt); int pmemkv_count_above(pmemkv_db *db, const char *k, size_t kb, size_t *cnt); int pmemkv_count_below(pmemkv_db *db, const char *k, size_t kb, size_t *cnt); int pmemkv_count_between(pmemkv_db *db, const char *k1, size_t kb1, const char *k2, size_t kb2, size_t *cnt); int pmemkv_get_all(pmemkv_db *db, pmemkv_get_kv_callback *c, void *arg); int pmemkv_get_above(pmemkv_db *db, const char *k, size_t kb, pmemkv_get_kv_callback *c, void *arg); int pmemkv_get_below(pmemkv_db *db, const char *k, size_t kb, pmemkv_get_kv_callback *c, void *arg); int pmemkv_get_between(pmemkv_db *db, const char *k1, size_t kb1, const char *k2, size_t kb2, pmemkv_get_kv_callback *c, void *arg); int pmemkv_exists(pmemkv_db *db, const char *k, size_t kb); int pmemkv_get(pmemkv_db *db, const char *k, size_t kb, pmemkv_get_v_callback *c, void *arg); int pmemkv_get_copy(pmemkv_db *db, const char *k, size_t kb, char *buffer, size_t buffer_size, size_t *value_size); int pmemkv_put(pmemkv_db *db, const char *k, size_t kb, const char *v, size_t vb); int pmemkv_remove(pmemkv_db *db, const char *k, size_t kb); int pmemkv_defrag(pmemkv_db *db, double start_percent, double amount_percent); const char *pmemkv_errormsg(void); ``` For pmemkv configuration API description see [**libpmemkv\_config**(3)](/libpmemkv_config/3.md). For pmemkv iterator API description see [**libpmemkv\_iterator**(3)](/libpmemkv_iterator/3.md). For general pmemkv information, engine descriptions and bindings details see [**libpmemkv**(7)](/libpmemkv/7.md). ## DESCRIPTION Keys and values stored in a pmemkv database can be arbitrary binary data and can contain multiple null characters. Every function which accepts key expects `const char *k` pointer to data and its size as `size_t`. Some of the functions (mainly range-query API) are not guaranteed to be implemented by all engines. If an engine does not support a certain function, it will return PMEMKV\_STATUS\_NOT\_SUPPORTED. Note: There are no explicit upper\_bound/lower\_bound functions. If you want to obtain an element(s) above or below the selected key, you can use pmemkv\_get\_above() or pmemkv\_get\_below(). See descriptions of these functions for details. - **int pmemkv\_open(const char \*engine, pmemkv\_config \*config, pmemkv\_db \*\*db);** Opens the pmemkv database and stores a pointer to a *pmemkv\_db* instance in `*db`. The `engine` parameter specifies the engine name (see [**libpmemkv**(7)](/libpmemkv/7.md) for the list of available engines). The `config` parameter specifies configuration (see [**libpmemkv\_config**(3)](/libpmemkv_config/3.md) for details). Pmemkv takes ownership of the config parameter - this means that pmemkv\_config\_delete() must NOT be called after open (successful or failed). - **void pmemkv\_close(pmemkv\_db \*kv);** Closes pmemkv database. - **int pmemkv\_count\_all(pmemkv\_db \*db, size\_t \*cnt);** Stores in `*cnt` the number of records in `db`. - **int pmemkv\_count\_above(pmemkv\_db \*db, const char \*k, size\_t kb, size\_t \*cnt);** Stores in `*cnt` the number of records in `db` whose keys are greater than the key `k` of length `kb`. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_count\_below(pmemkv\_db \*db, const char \*k, size\_t kb, size\_t \*cnt);** Stores in `*cnt` the number of records in `db` whose keys are less than the key `k` of length `kb`. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_count\_between(pmemkv\_db \*db, const char \*k1, size\_t kb1, const char \*k2, size\_t kb2, size\_t \*cnt);** Stores in `*cnt` the number of records in `db` whose keys are greater than key `k1` (of length `kb1`) and less than key `k2` (of length `kb2`). Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_get\_all(pmemkv\_db \*db, pmemkv\_get\_kv\_callback \*c, void \*arg);** Executes function `c` for every record stored in `db`. Arguments passed to the function are: pointer to a key, size of the key, pointer to a value, size of the value and `arg` specified by the user. Function `c` can stop iteration by returning non-zero value. In that case *pmemkv\_get\_all()* returns PMEMKV\_STATUS\_STOPPED\_BY\_CB. Returning 0 continues iteration. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_get\_above(pmemkv\_db \*db, const char \*k, size\_t kb, pmemkv\_get\_kv\_callback \*c, void \*arg);** Executes function `c` for every record stored in `db` whose keys are greater than key `k` (of length `kb`). Arguments passed to `c` are: pointer to a key, size of the key, pointer to a value, size of the value and `arg` specified by the user. Function `c` can stop iteration by returning non-zero value. In that case *pmemkv\_get\_above()* returns PMEMKV\_STATUS\_STOPPED\_BY\_CB. Returning 0 continues iteration. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_get\_below(pmemkv\_db \*db, const char \*k, size\_t kb, pmemkv\_get\_kv\_callback \*c, void \*arg);** Executes function `c` for every record stored in `db` whose keys are less than key `k` (of length `kb`). Arguments passed to `c` are: pointer to a key, size of the key, pointer to a value, size of the value and `arg` specified by the user. Function `c` can stop iteration by returning non-zero value. In that case *pmemkv\_get\_below()* returns PMEMKV\_STATUS\_STOPPED\_BY\_CB. Returning 0 continues iteration. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_get\_between(pmemkv\_db \*db, const char \*k1, size\_t kb1, const char \*k2, size\_t kb2, pmemkv\_get\_kv\_callback \*c, void \*arg);** Executes function `c` for every record stored in `db` whose keys are greater than key `k1` (of length `kb1`) and less than key `k2` (of length `kb2`). Arguments passed to `c` are: pointer to a key, size of the key, pointer to a value, size of the value and `arg` specified by the user. Function `c` can stop iteration by returning non-zero value. In that case *pmemkv\_get\_between()* returns PMEMKV\_STATUS\_STOPPED\_BY\_CB. Returning 0 continues iteration. Order of the elements is specified by a comparator (see [**libpmemkv**(7)](/libpmemkv/7.md)). - **int pmemkv\_exists(pmemkv\_db \*db, const char \*k, size\_t kb);** Checks existence of record with key `k` of length `kb`. If record is present PMEMKV\_STATUS\_OK is returned, otherwise PMEMKV\_STATUS\_NOT\_FOUND is returned. Other possible return values are described in the *ERRORS* section. - **int pmemkv\_get(pmemkv\_db \*db, const char \*k, size\_t kb, pmemkv\_get\_v\_callback \*c, void \*arg);** Executes function `c` on record with key `k` (of length `kb`). If record is present and no error occurred the function returns PMEMKV\_STATUS\_OK. If record does not exist PMEMKV\_STATUS\_NOT\_FOUND is returned. Other possible return values are described in the *ERRORS* section. Function `c` is called with the following parameters: pointer to a value, size of the value and `arg` specified by the user. `Value` points to the location where data is actually stored (no copy occurs). This function is guaranteed to be implemented by all engines. - **int pmemkv\_get\_copy(pmemkv\_db \*db, const char \*k, size\_t kb, char \*buffer, size\_t buffer\_size, size\_t \*value\_size);** Copies value of record with key `k` of length `kb` to user provided buffer. `buffer` points to the value buffer, `buffer_size` specifies its size and `*value_size` is filled in by this function. If the value doesn’t fit in the provided buffer then this function returns PMEMKV\_STATUS\_UNKNOWN\_ERROR. Otherwise, in absence of any errors, PMEMKV\_STATUS\_OK is returned. Other possible return values are described in the *ERRORS* section. This function is guaranteed to be implemented by all engines. - **int pmemkv\_put(pmemkv\_db \*db, const char \*k, size\_t kb, const char \*v, size\_t vb);** Inserts a key-value pair into pmemkv database. `kb` is the length of key `k` and `vb` is the length of value `v`. When this function returns, caller is free to reuse both buffers. This function is guaranteed to be implemented by all engines. - **int pmemkv\_remove(pmemkv\_db \*db, const char \*k, size\_t kb);** Removes record with key `k` of length `kb`. This function is guaranteed to be implemented by all engines. - **int pmemkv\_defrag(pmemkv\_db \*db, double start\_percent, double amount\_percent);** Defragments approximately \`amount\_percent' percent of elements in the database starting from \`start\_percent' percent of elements. - **const char \*pmemkv\_errormsg(void);** Returns a human readable string describing the last error. ### ERRORS Each function, except for *pmemkv\_close()* and *pmemkv\_errormsg()*, returns one of the following status codes: - • **PMEMKV\_STATUS\_OK** – no error - • **PMEMKV\_STATUS\_UNKNOWN\_ERROR** – unknown error - • **PMEMKV\_STATUS\_NOT\_FOUND** – record not found - • **PMEMKV\_STATUS\_NOT\_SUPPORTED** – function is not implemented by current engine - • **PMEMKV\_STATUS\_INVALID\_ARGUMENT** – argument to function has wrong value - • **PMEMKV\_STATUS\_CONFIG\_PARSING\_ERROR** – parsing data to config failed - • **PMEMKV\_STATUS\_CONFIG\_TYPE\_ERROR** – config item has different type than expected - • **PMEMKV\_STATUS\_STOPPED\_BY\_CB** – iteration was stopped by user’s callback - • **PMEMKV\_STATUS\_OUT\_OF\_MEMORY** – operation failed because there is not enough memory (or space on the device) - • **PMEMKV\_STATUS\_WRONG\_ENGINE\_NAME** – engine name does not match any available engine - • **PMEMKV\_STATUS\_TRANSACTION\_SCOPE\_ERROR** – an error with the scope of the libpmemobj transaction - • **PMEMKV\_STATUS\_DEFRAG\_ERROR** – the defragmentation process failed (possibly in the middle of a run) Status returned from a function can change in a future version of a library to a more specific one. For example, if a function returns PMEMKV\_STATUS\_UNKNOWN\_ERROR, it is possible that in future versions it will return PMEMKV\_STATUS\_INVALID\_ARGUMENT. Recommended way to check for an error is to compare status with PMEMKV\_STATUS\_OK. ## EXAMPLE The following example is taken from `examples/pmemkv_basic_c` directory. Basic pmemkv usage in C: - ``` #include #include #include #include #include #define ASSERT(expr) \ do { \ if (!(expr)) \ puts(pmemkv_errormsg()); \ assert(expr); \ } while (0) #define LOG(msg) puts(msg) #define MAX_VAL_LEN 64 static const uint64_t SIZE = 1024UL * 1024UL * 1024UL; int get_kv_callback(const char *k, size_t kb, const char *value, size_t value_bytes, void *arg) { printf(" visited: %s\n", k); return 0; } int main(int argc, char *argv[]) { if (argc < 2) { fprintf(stderr, "Usage: %s file\n", argv[0]); exit(1); } /* See libpmemkv_config(3) for more detailed example of config creation */ LOG("Creating config"); pmemkv_config *cfg = pmemkv_config_new(); ASSERT(cfg != NULL); int s = pmemkv_config_put_path(cfg, argv[1]); ASSERT(s == PMEMKV_STATUS_OK); s = pmemkv_config_put_size(cfg, SIZE); ASSERT(s == PMEMKV_STATUS_OK); s = pmemkv_config_put_force_create(cfg, true); ASSERT(s == PMEMKV_STATUS_OK); LOG("Opening pmemkv database with 'cmap' engine"); pmemkv_db *db = NULL; s = pmemkv_open("cmap", cfg, &db); ASSERT(s == PMEMKV_STATUS_OK); ASSERT(db != NULL); LOG("Putting new key"); const char *key1 = "key1"; const char *value1 = "value1"; s = pmemkv_put(db, key1, strlen(key1), value1, strlen(value1)); ASSERT(s == PMEMKV_STATUS_OK); size_t cnt; s = pmemkv_count_all(db, &cnt); ASSERT(s == PMEMKV_STATUS_OK); ASSERT(cnt == 1); LOG("Reading key back"); char val[MAX_VAL_LEN]; s = pmemkv_get_copy(db, key1, strlen(key1), val, MAX_VAL_LEN, NULL); ASSERT(s == PMEMKV_STATUS_OK); ASSERT(!strcmp(val, "value1")); LOG("Iterating existing keys"); const char *key2 = "key2"; const char *value2 = "value2"; const char *key3 = "key3"; const char *value3 = "value3"; pmemkv_put(db, key2, strlen(key2), value2, strlen(value2)); pmemkv_put(db, key3, strlen(key3), value3, strlen(value3)); pmemkv_get_all(db, &get_kv_callback, NULL); LOG("Removing existing key"); s = pmemkv_remove(db, key1, strlen(key1)); ASSERT(s == PMEMKV_STATUS_OK); ASSERT(pmemkv_exists(db, key1, strlen(key1)) == PMEMKV_STATUS_NOT_FOUND); LOG("Defragmenting the database"); s = pmemkv_defrag(db, 0, 100); ASSERT(s == PMEMKV_STATUS_OK); LOG("Closing database"); pmemkv_close(db); return 0; } ``` ### Common usage mistake Common mistake in pmemkv API usage (especially when using C++ API) is to dereference pointer to the data stored in pmemkv outside of a callback function scope. - ``` std::string value; const char* ptr; size_t sz; kv->get("key1", [&](string_view v) { /* Save pointer to the data to use it later outside of a callback scope */ ptr = v.data(); sz = v.size(); }); kv->remove("key"); /* ERROR! * Using this pointer outside of a callback function may cause access to some random data * or a segmentation fault. At that point, ptr should be considered as invalid. */ value.append(ptr, sz); ``` ## SEE ALSO [**libpmemkv**(7)](/libpmemkv/7.md), [**libpmemkv\_config**(3)](/libpmemkv_config/3.md), [**libpmemkv\_iterator**(3)](/libpmemkv_iterator/3.md) and **\**