nbdkit-memory-plugin1

Usage

Overview

"nbdkit-memory-plugin" is a plugin for nbdkit(1) which stores a single disk image in virtual memory, and discards it when nbdkit exits. This plugin can be used for testing or where you don't care about the final content of the disk image.

All nbdkit clients will see the same disk content, initially all zeroes.

By default the disk image is stored in memory using a sparse array. The allocated parts of the disk image cannot be larger than physical RAM plus swap, less whatever is being used by the rest of the system. Other allocators are available, see "ALLOCATORS" below. All allocators store the image in memory. If you want to allocate more space than this use nbdkit-file-plugin(1) backed by a temporary file instead.

Using the sparse allocator the virtual size can be as large as you like, up to the maximum supported by nbdkit (2⁶³-1 bytes). This limit is tested when nbdkit is compiled, and it should work on all platforms and architectures supported by nbdkit.

Examples

Create a one gigabyte sparse RAM disk:

 nbdkit memory 1G

If you want to loop mount the above disk, see nbdkit-loop(1).

Create the largest possible RAM disk:

 nbdkit memory $(( 2**63 - 1 ))

Parameters

[size=]SIZE

Specify the virtual size of the disk image.

This parameter is required.

"size=" prefix may be omitted in most cases. See "Magic parameters" in nbdkit(1).

allocator=sparse

allocator=malloc[,mlock=true]

allocator=zstd

(nbdkit ≥ 1.22)

Select the backend allocation strategy. See "ALLOCATORS" below. The default is sparse.

Notes

 $ rm -f pid
 $ nbdkit -P pid memory 10G

 $ while [ ! -f pid ]; do sleep 1; done

 $ virt-builder fedora-28 --size=10G
 $ qemu-img convert -p -n fedora-28.img nbd:localhost:10809

 $ nbdcopy -p fedora-28.img nbd://localhost

Allocators

Since nbdkit ≥ 1.22 several allocation strategies are available using the "allocator" parameter.

allocator=sparse

The disk image is stored in memory using a sparse array. The sparse array uses a simple two level page table with a fixed page size. The allocated parts of the disk image cannot be larger than physical RAM plus swap, less whatever is being used by the rest of the system. The aim of the sparse array implementation is to support extremely large images for testing, although it won't necessarily be efficient for that use case. However it should also be reasonably efficient for normal disk sizes.

The virtual size of the disk can be as large as you like, up to the maximum supported by nbdkit (2⁶³-1 bytes).

This is the default, and was the only allocator available before nbdkit 1.22.

allocator=malloc

allocator=malloc,mlock=true

The disk image is stored directly in memory allocated using malloc(3) on the heap. No sparseness is possible: you must have enough memory for the whole disk. Very large virtual sizes will usually fail. However this can be faster because the implementation is simpler and the locking strategy allows more concurrency.

If "mlock=true" is added then additionally the array is locked into RAM using mlock(2) (so it should never be swapped out). This usually requires you to adjust the ulimit(1) associated with the process and on some operating systems may require you to run nbdkit as root. (See also the nbdkit(1) --swap option).

The "mlock=true" feature is only supported on some platforms. Use "nbdkit memory --dump-plugin" and check that the output contains "mlock=yes".

allocator=zstd

The disk image is stored in a sparse array where each page is compressed using zstd compression. Assuming a typical 2:1 compression ratio, this allows you to store twice as much real data as "allocator=sparse", with the trade-off that the plugin is slightly slower because it has to compress and decompress each page. Aside from compression, the implementation of this allocator is similar to "allocator=sparse", so in other respects (such as supporting huge virtual disk sizes) it is the same.

This allocator is only supported if nbdkit was compiled with zstd support. Use "nbdkit memory --dump-plugin" and check that the output contains "zstd=yes".

Files

$plugindir/nbdkit-memory-plugin.so

The plugin.

Use "nbdkit --dump-config" to find the location of $plugindir.

Version

"nbdkit-memory-plugin" first appeared in nbdkit 1.2.