--- name: "dtrace_fbt" section: "4" platform: "freebsd" package: "base" version: "16.0-CURRENT-20261027-61c70dc16a79" release: "FreeBSD 16.0-CURRENT (2026-10-27)" platforms: ["freebsd"] url: "https://new.carta.tech/dtrace_fbt" --- # dtrace\_fbt(4) a DTrace provider for dynamic kernel tracing based on function boundaries ## NAME **dtrace\_fbt** – a DTrace provider for dynamic kernel tracing based on function boundaries ## SYNOPSIS **fbt****:***module***:***function***:entry** **fbt****:***module***:***function***:return** ## DESCRIPTION The Function Boundary Tracing (**fbt**) provider instruments the entry and return of almost every kernel function corresponding to an [elf(5)](/elf/5.md) symbol in the kernel and loaded kernel modules. **fbt****:***module***:***function***:entry** fires whenever the *function* is called. **fbt****:***module***:***function***:return** fires when the *function* returns. The *module* in the probe description is either the name of the loaded kernel module or ‘kernel’ for functions compiled into the kernel. ### Function Boundary Instrumentation The **fbt** will always instrument a function's entry, but its return will be intsrumented so long as it can find a ‘ret’ instruction. In some cases, **fbt** cannot instrument a function's entry and/or return. Refer to subsection Frame Pointer for more details. ### Probe Arguments The arguments of the entry probe (**fbt****:***module***:***function***:entry**) are the arguments of the traced function call. | | | | --- | --- | | **Entry** **Probe** **Argument** | **Definition** | | *args\[0\]* | Function's first argument, typed | | *args\[1\]* | Function's second argument, typed | | *args\[2\]* | Function's third argument, typed | | *...* | ... | The arguments of the return probe (**fbt****:***module***:***function***:return**) are *args\[0\]*( the offset of the firing return instruction within the function; useful to tell apart two different return statements in a single function) and *args\[1\]* (the return value, if any). | | | | --- | --- | | **Return** **Probe** **Argument** | **Definition** | | *args\[0\]* | Offset of the traced return instruction | | *args\[1\]* | Function's return value | Subsection Example 2: Getting Details About Probe's Arguments shows how to get probe's argument count and types directly with [dtrace(1)](/dtrace/1.md) without having to resort to the reading function's source code or documentation. ## EXAMPLES ### Example 1 : Listing Available FBT Probes The following example shows how to list all the available **fbt** probes. ``` # dtrace -l -P fbt ID PROVIDER MODULE FUNCTION NAME [...] 31868 fbt kernel hammer_time entry 31869 fbt kernel hammer_time return [...] ``` Since **hammer\_time**() is a part of the kernel and not a separate loaded module, the *module* column displays ‘kernel’. ### Example 2 : Getting Details About Probe's Arguments The following example shows how to generate a program stability report of [malloc(9)](/malloc/9.md)'s entry and return probes. Those reports are useful to view the probe's number of arguments and their types. ``` # dtrace -l -v -n fbt::malloc:entry [...] Argument Types args[0]: size_t args[1]: struct malloc_type * args[2]: int ``` The count and types of **fbt****::malloc:entry** arguments match the function signature of [malloc(9)](/malloc/9.md): *args\[0\]* is *size\_t*, *args\[1\]* is *struct malloc\_type \**, and *args\[2\]* is *int*. ``` # dtrace -l -v -n fbt::malloc:return [...] Argument Types args[0]: int args[1]: void * ``` The **return** probe reports two arguments and their types: the return instruction offset (the usual *int*) and the function's return value, which in this case is *void* *\**, as [malloc(9)](/malloc/9.md) returns a kernel virtual address. ### Example 3 : Counting Kernel Slab Memory Allocation by Function ``` # dtrace -n 'fbt::kmem*:entry { @[probefunc] = count(); }' dtrace: description 'fbt::kmem*:entry ' matched 47 probes ^C kmem_alloc_contig 1 kmem_alloc_contig_domainset 1 kmem_cache_reap_active 1 kmem_alloc_contig_pages 2 kmem_free 2 kmem_std_destructor 19 kmem_std_constructor 26 kmem_cache_free 151 kmem_cache_alloc 181 ``` ### Example 4 : Counting Kernel Slab Memory Allocation by Calling Function ``` # dtrace -q -n 'fbt::kmem*:entry { @[caller] = count(); } END { printa("%40a %@16d\n", @); }' ^C kernel`contigmalloc+0x33 1 kernel`free+0xd3 1 kernel`kmem_alloc_contig+0x29 1 kernel`kmem_alloc_contig_domainset+0x19a 1 zfs.ko`arc_reap_cb_check+0x16 1 ``` ### Example 5 : Counting Kernel malloc()'s by Calling Function ``` # dtrace -q -n 'fbt::malloc:entry { @[caller] = count(); } END { printa("%45a %@16d\n", @); }' ^C kernel`devclass_get_devices+0xa8 1 kernel`sys_ioctl+0xb7 1 dtrace.ko`dtrace_ioctl+0x15c1 1 dtrace.ko`dtrace_ioctl+0x972 2 dtrace.ko`dtrace_dof_create+0x35 2 kernel`kern_poll_kfds+0x2f0 4 kernel`kern_poll_kfds+0x28a 19 ``` ### Example 6 : Counting Kernel malloc()'s by Kernel Stack Trace ``` # dtrace -q -n 'fbt::malloc:entry { @[stack()] = count(); }' ^C dtrace.ko`dtrace_dof_create+0x35 dtrace.ko`dtrace_ioctl+0x827 kernel`devfs_ioctl+0xd1 kernel`VOP_IOCTL_APV+0x2a kernel`vn_ioctl+0xb6 kernel`devfs_ioctl_f+0x1e kernel`kern_ioctl+0x286 kernel`sys_ioctl+0x12f kernel`amd64_syscall+0x169 kernel`0xffffffff81092b0b 2 ``` ### Example 7 : Summarizing vmem\_alloc()'s by Arena Name and Size Distribution ``` # dtrace -q -n 'fbt::vmem_alloc:entry { @[args[0]->vm_name] = quantize(arg1); }' ^C kernel arena dom value ------------- Distribution ------------- count 2048 | 0 4096 |@@@@@@@@@@@@@@@@@@@@@@@@@@@ 4 8192 |@@@@@@@@@@@@@ 2 16384 | 0 ``` ### Example 8 : Measuring Total Time Spent Executing a Function This DTrace script measures the total time spent in **vm\_page\***() kernel functions. The **quantize**() aggregation organizes the measurements into power-of-two buckets, providing a time distribution in nanoseconds for each function. ``` fbt::vm_page*:entry { self->start = timestamp; } fbt::vm_page*:return /self->start/ { @[probefunc] = quantize(timestamp - self->start); self->start = 0; } ``` ## SEE ALSO [dtrace(1)](/dtrace/1.md), [dtrace\_kinst(4)](/dtrace_kinst/4.md), [tracing(7)](/tracing/7.md) Brendan Gregg and Jim Mauro, *DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X and FreeBSD*, Prentice Hall, pp. 898–903, 2011, https://www.brendangregg.com/dtracebook/. *The illumos Dynamic Tracing Guide*, 2008, Chapter fbt Provider, https://illumos.org/books/dtrace/chp-fbt.html#chp-fbt. ## AUTHORS This manual page was written by Mateusz Piotrowski ⟨*0mp@FreeBSD.org*⟩. ## CAVEATS ### Stability and Portability **fbt** probes are by definition tightly coupled to kernel code; if the code underlying a script changes, the script may fail to run or may produce incorrect results. Scripts written for one version of FreeBSD might not work on others, and almost certainly will not work on other operating systems. Individual **fbt** probes often do not correspond nicely to logical system events. For example, consider a DTrace script which prints the destination address of every IP packet as the kernel hands them over to the network card driver (NIC). An **fbt**-based implementation of such a script is a discouragingly difficult task: it involves instrumenting at least four different functions in different parts of the IPv4 and IPv6 code. At the same time, with the [dtrace\_ip(4)](/dtrace_ip/4.md) provider the script is a simple one-liner: ``` dtrace -n 'ip:::send {printf("%s", args[2]->ip_daddr);}' ``` Make sure to review available [dtrace(1)](/dtrace/1.md) providers first before implementing a custom script with the **fbt** provider. If none of the DTrace providers offer the desired probes, consider adding new statically-defined tracing probes ([SDT(9)](/SDT/9.md)). ### Frame Pointer Inline functions are not instrumentable by **fbt** as they lack a frame pointer. A developer might explicitly disable inlining by adding the ‘\_\_noinline’ attribute to a function definition, but of course this requires a recompilation of the kernel. Building the kernel with **-fno-omit-frame-pointer** is another way of preserving frame pointers. Note, that sometimes compilers will omit the frame pointer in leaf functions, even when configured with **-fno-omit-frame-pointer**. Function returns via a tail call are also not instrumentable by **fbt**. As a result, a function might have an entry probe and a mix of instrumented and uninstrumentable returns. Use [dtrace\_kinst(4)](/dtrace_kinst/4.md) to trace arbitrary instructions inside kernel functions and work around some of the limitations of **fbt**. ### Tracing DTrace The **fbt** provider cannot attach to functions inside DTrace provider kernel modules.