Reference

tcp_bblog(4)

TCP Black Box Logging facility

NAME

tcp_bblog – TCP Black Box Logging facility

SYNOPSIS

options TCP_BLACKBOX

DESCRIPTION

The tcp_bblog facility provides structured logging of TCP stack activity for debugging and performance analysis. It is implemented in the kernel and allows per-connection tracing of TCP events with low overhead. Each TCP endpoint may maintain an internal log buffer recording events such as packet transmission, reception, retransmissions, and state transitions. Unlike earlier debugging mechanisms, tcp_bblog records structured metadata without capturing TCP payload data. Logs are exported to userland via the device /dev/tcp_log. If no process is actively reading from this device, log records may be discarded. Each log entry includes:

  • Connection identifiers, including local and remote addresses and ports

  • Timestamp information

  • TCP state and sequence variables

  • Event-specific metadata

Logging may be enabled globally using sysctl(8) or on a per-connection basis using setsockopt(2) or tcpsso(8).

MODES

The following logging modes are supported:

TCP_LOG_STATE_OFF

Disable logging.

TCP_LOG_STATE_TAIL

Maintain a ring buffer of recent events.

TCP_LOG_STATE_HEAD

Log only the initial events of a connection.

TCP_LOG_STATE_HEAD_AUTO

Log initial events and automatically export them.

TCP_LOG_STATE_CONTINUAL

Continuously log and export events as buffers fill.

TCP_LOG_STATE_TAIL_AUTO

Maintain a tail buffer and export on overflow.

CONFIGURATION

Global configuration is available via sysctl(8) under the net.inet.tcp.bb namespace. Notable variables include:

net.inet.tcp.bb.log_auto_all

Enable logging consideration for all TCP connections.

net.inet.tcp.bb.log_auto_ratio

Select one out of N connections for logging.

net.inet.tcp.bb.log_auto_mode

Default logging mode applied automatically. When set to 1 the last net.inet.tcp.bb.log_session_limit entries are stored at the TCP endpoint. This is a good way to capture information when debugging TCP related panics.

net.inet.tcp.bb.log_session_limit

Maximum number of log entries per connection.

net.inet.tcp.bb.log_global_limit

Global limit on allocated log entries.

FILES

/dev/tcp_log

TCP BBLog device interface

/var/log/tcplog_dumps

Default directory for stored log output

EXAMPLES

Enable continual logging for all TCP connections:

sysctl net.inet.tcp.bb.log_auto_all=1
sysctl net.inet.tcp.bb.log_auto_ratio=1
sysctl net.inet.tcp.bb.log_auto_mode=4

Enable logging on a specific socket:

int mode = TCP_LOG_STATE_CONTINUAL;
setsockopt(sd, IPPROTO_TCP, TCP_LOG, &mode, sizeof(mode));

SEE ALSO

tcpdump(1), setsockopt(2), tcp(4), sysctl(8), tcpsso(8) Randall Stewart and Michael Tüxen, “Adventures in TCP/IP: TCP Black Box Logging”, FreeBSD Journal, May / June 2024, https://freebsdfoundation.org/adventures-in-tcp-ip-tcp-black-box-logging/.

HISTORY

The tcp_bblog facility first appeared in FreeBSD 13.0 and replaced earlier TCP debugging mechanisms.

AUTHORS

The tcp_bblog facility was written by Randall Stewart ⟨[email protected]⟩ and Jonathan Looney ⟨[email protected]⟩ and sponsored by Netflix, Inc. This manual page was written by Gordon Bergling ⟨[email protected]⟩.

CAVEATS

Log records may be dropped if userland does not drain /dev/tcp_log fast enough. TCP payload data is not recorded.

BUGS

Logging may introduce measurable overhead under high connection rates.