--- name: "nfdump" section: "1" platform: "linux" package: "nfdump" version: "1.7.5-2+deb13u1" release: "Debian 13 (Trixie)" platforms: ["linux"] url: "https://new.carta.tech/nfdump" --- # nfdump(1) flow display and analysis program ## NAME **nfdump** – flow display and analysis program ## SYNOPSIS **nfdump** **-r** *flowpath* \[**-w** *outfile*\] \[**-f** *filterfile*\] \[**-C** *config*\] \[**-R** *filelist*\] \[**-M** *dirlist*\] \[**-O** *order*\] \[**-t** *timewin*\] \[**-c** *num*\] \[**-a**\] \[**-A** *aggregation*\] \[**-b**\] \[**-B**\] \[**-I**\] \[**-D** *nameserver*\] \[**-G** *geoDB*\] \[**-H** *torDB*\] \[**-s** *statistic*\] \[**-n** *num*\] \[**-o** *format*\] \[**-6**\] \[**-q**\] \[**-N**\] \[**-i** *ident*\] \[**-v** *flowfile*\] \[**-E** *flowfile*\] \[**-x** *flowfile*\] \[**-W** *workers*\] \[**-z=\**\] \[**-J** *compress*\] \[**-X**\] \[**-Z**\] \[**-T**\] \[**-V**\] \[**-h**\] \[filter\] ## DESCRIPTION **nfdump** reads the flow data from one or more binary files, created by any nfdump collector *nfcapd,* *nfpcapd* and *sfcapd*. It processes and lists the flows in many different output formats and can create a wide range of statistics. **nfdump** has a very powerful flow filter to process flows. The filter syntax is very similar to tcpdump, but adapted and extended for flow filtering. A flow filter may also contain arrays of many thousand IP addresses etc. to search for specific records. **nfdump** can aggreagte flows according to a user defined number of elements. This masks certain elements and allows to sum up flow records matching the same values. The combination of flow filtering and aggregation as input for any flow statistics allows complex flow processing. Pre-filtered and aggregated flow data may also be written back into a binary flow file, which again may be processed with **nfdump** **nfdump** can enrich the listing of flows with geo location information and AS information, unless AS information is already available in the flow records. IP addresses can be tagged with a two letter country code, or with a longer location label containing the geographic region, country and city. The geo location and AS information is retrieved from the optional *geoDB* database, created by the *geolookup* program from the nfdump tools. *geolookup* uses the *Maxmind* database *GeoDB* or *GeoLite2* to create a binary lookup database for **nfdump** Please check the *geolooup(1)* man page for more details. The options are as follows: - **-r** *flowpath* Reads flow records from this path. *flowpath* may be a single file, or a directory containing any number of flow files or sub directories. All files are processed in the order, as listed by the OS. - **-w** *outfile* Writes all processed records into *outfile* instead of printing. The *flowfile* is a binary flow file and may be processed again with **nfdump** This can be useful to limit flows according to a flow filter and/or specific flow aggregation. - **-f** *filterfile* Reads the flow filter from *filterfile.* This can be useful for very long or structured filters, with comments and long lists. *Note:* Any filter specified directly on the command line takes precedence over the *filterfile.* - **-C** *config* Read more options from file *config.* **nfdump** tries to read by default *%prefix/etc/nfdump.config.* This may be overwritten by the environment valiable *NFCONF* which again may be overwritten by this option **-C.** In order to prevent reading any config file, even if it would exist set **-C** **none.** A config file is not required, but may be handy for often used output formats etc. - **-O** *order* Sets an output order for records to be printed as text output. This order applies after all records processing, such as filtering, and aggregation and before printing. - **flows** Sort according to the number of flows - **packets** Sort according to (in)packets - **ipkg** Same as packets - **opkg** Sort according to output packets - **bytes** Sort according to (in)bytes - **ibyte** Same as bytes - **obyte** Sort according to output bytes - **pps** Sort according to (in)packets per second - **ipps** Same as ipps - **opps** Sort according to out packets per second - **bps** Sort according to (in)bytes per second - **ibps** Same as bps - **obps** Sort according to output bytes per second - **bpp** Sort according to (in)bytes per packet - **ibpp** Same as bpp - **obpp** Sort according to output packets - **tstart** Sort according to start time of flow - former -m - **tend** Sort according to end time of flows - **duration** Sort according to duration of flows - **-t** *timewin* Set time window to process flows. This option is considered legacy andmay be replaced with a *filter* primitive in future rleases. The time window is specified as: YYYY/MM/dd.hh:mm:ss\[-YYYY/MM/dd.hh:mm:ss\]. Any parts of the time spec may be omitted e.g YYYY/MM/dd expands to YYYY/MM/dd.00:00:00-infinity and processes all flow from a given day onwards. The time window may also be specified as +/- n. In this case it is relative to the beginning or end of all flows. +10 means the first 10 seconds of all flows, -10 means the last 10 seconds of all flows. - **-c** *num* Limit the number of records to be processed to the first *num* records, which passwd the *filter.* - **-a** Aggregate flow records. The default aggregation is done at connection level by taking the 5-tuple *protocol,* *srcip,* *dstip,* *srcport* and *dstport.* This way of aggregation may be overwritten by option **-A** - **-A** *aggregation* Sets the list of elements in a flow record to be aggregated. *aggregation* is a ',' separated list of any number of v9/ipfix elements. The following elements are accepted: - **proto** IP protocol - **srcip** Source IP address - **dstip** Destination IP address - **srcip4/net** IPv4 source IP address with applied netmask - **srcip6/net** IPv6 source IP address with applied netmask - **dstip4/net** IPv4 destination IP address with applied netmask - **dstip6/net** IPv6 destination IP address with applied netmask - **srcnet** Apply netmask srcmask in netflow record for source IP - **dstnet** Apply netmask dstmask in netflow record for dest IP - **srcport** Source port - **dstport** Destination port - **srcmask** Source mask - **dstmask** Destination mask - **srcvlan** Source vlan label - **dstvlan** Destination vlan label - **srcas** Source AS number - **dstas** Destination AS number - **nextas** BGP Next AS - **prevas** BGP Previous AS - **inif** SNMP input interface number - **outif** SNMP output interface number - **next** IP next hop - **bgpnext** BGP next hop - **insrcmac** In source MAC address - **outdstmac** out destination MAC address - **indstmac** In destination MAC address - **outsrcmac** Out source MAC address - **tos** Source type of service - **srctos** Source type of Service - **dsttos** Destination type of Service - **mpls1** MPLS label 1 - **mpls2** MPLS label 2 - **mpls3** MPLS label 3 - **mpls4** MPLS label 4 - **mpls5** MPLS label 5 - **mpls6** MPLS label 6 - **mpls7** MPLS label 7 - **mpls8** MPLS label 8 - **mpls9** MPLS label 9 - **mpls10** MPLS label 10 - **router** IP address of exporting router - **odid** observation domain ID - **opid** observation point ID - **xsrcip** X-late source IP address, if compiled with NSEL support - **xdstip** X-late destination IP address, if compiled with NSEL support - **xsrcport** X-late source port, if compiled with NSEL support - **xdstport** X-late destination port, if compiled with NSEL support **nfdump** automatically compiles the appropriate output format for the selected aggregation elements unless an explicit output format **-o** is given. The automatic output format is identical to ``` -o 'fmt:%ts %td %pkt %byt %bps %bpp %fl' ``` where \ represents the selected aggregation tags. - **-b** Aggregate flow records as bidirectional flows. This automatically implies -a. Aggregation is done on connection level by taking the 5-tuple *protocol,* *srcip,* *dstip,* *srcport* and *dstport* The reverse order applies for the corresponding reverse flow. Input and output packets/bytes are counted and reported separately. Both flows are merged into a single record with corresponding input and output counters. An appropriate output format is selected automatically, which may be overwritten by any **-o** *format* option. - **-B** Similar to option **-b** but tries to guess the correct client to server direction. Automagically swaps flows if src port is \< dst port for TCP and UDP flows and src port \< 1024 and dst port \> 1024. Some exporters do not really care sending the flows in proper order. It's considered to be a conveniency option. - **-I** Print flow statistics of a single file or the summary of all the files specified by **-r** *flowpath.* - **-g** Print for each flow file given by **-r** *flowpath* a one line summary, which can be easily used by gnu plot. - **-D** *nameserver* Sets the *nameserver* to translate hostnames into IP addresses in filter expressions. See *filter* below for more details. - **-G** *geoDB* Use *geoDB* as geo lookup DB for geo location and AS lookups. **nfdump** tries to read the environment variable *NFGEODB* for the path of *geoDB.* The option **-G** overwrites *NFGEODB* or *geodb.path* in *nfdump.conf.* In order to prevent reading any *geoDB* file, even if it would exist set **-G** **none.** See also *geolookup(1)* - **-H** *torDB* Use *torDB* as tor lookup DB for tor exit node lookups. **nfdump** tries to read the environment variable *NFTORDB* for the path of *torDB.* The option **-H** overwrites *NFTORDB* or *tordb.path* in *nfdump.conf.* In order to prevent reading any *torDB* file, even if it would exist set **-H** **none.** See also *torlookup(1)* - **-s** *statistic* \[*:p* \[*/orderby*\]\] Generate the Top N flow record or flow element statistic. By optionally adding **:p** to *statistic,* the statistic is additionally split up into the transport layer protocols. By default the statistic is transport protocol independent. Each *statistic* may be ordered by the optional parameter *orderby* This can be **flows,** **packets,** **bytes,** **pps,** **bps** or **bpp.** You may specify more than one *orderby* option, which results in the same statistic but ordered differently. If no orderby is given, the statistic is ordered by flows. You can specify as many -s flow element statistics as needed on the command line for the same run. *statistic* can be: - **record** aggregated netflow records. - **srcip** source IP addresses - **dstip** destination IP addresses - **ip** any (src or dst) IP addresses - **nhip** next hop IP addresses - **nhbip** BGP next hop IP addresses - **router** exporting router IP address - **srcport** source ports - **dstport** destination ports - **port** any (source or destination) ports - **tos** type of service - default src - **srctos** src type of service - **dsttos** dst type of service - **dir** flow directions ingress/egress - **srcas** source AS numbers - **dstas** destination AS numbers - **srcasn** source AS organisations and numbers - **dstasn** destination AS organisations and numbers - **srcgeo** 2 letter geo source country code - **dstgeo** 2 letter geo destination country code - **as** any (source or destination) AS numbers - **asn** any (source or destination) AS org and numbers - **inif** input interface - **outif** output interface - **if** any interface - **inam** input interface name - **onam** output interface name - **srcmask** src mask - **dstmask** dst mask - **srcvlan** src vlan label - **dstvlan** dst vlan label - **vlan** any vlan label - **insrcmac** input src MAC address - **outdstmac** output dst MAC address - **indstmac** input dst MAC address - **outsrcmac** output src MAC address - **srcmac** any src MAC address - **dstmac** any dst MAC address - **inmac** any input MAC address - **outmac** any output MAC address - **mask** any mask - **proto** IP protocols - **mpls1** MPLS label 1 - **mpls2** MPLS label 2 - **mpls3** MPLS label 3 - **mpls4** MPLS label 4 - **mpls5** MPLS label 5 - **mpls6** MPLS label 6 - **mpls7** MPLS label 7 - **mpls8** MPLS label 8 - **mpls9** MPLS label 9 - **mpls10** MPLS label 10 - **sysid** Internal SysID of exporter - **nbar** nbar ID - **ja3** ja3 hashes - **odid** observation domain ID - **opid** observation point ID - **vrf/ivrf** ingress vrf - **evrf** egress vrf - **ivrfnam** ingress vrf name - **evrfnam** egress vrf name - NSEL/ASA statistics - **event** NSEL/ASA event - **xevent** NSEL/ASA extended event - **xsrcip** NSEL/ASA translated src IP address - **xsrcport** NSEL/ASA translated src port - **xdstip** NSEL/ASA translated dst IP address - **xdstport** NSEL/ASA translated dst port - **iacl** NSEL/ASA ingress ACL - **iace** NSEL/ASA ingress ACE - **ixace** NSEL/ASA ingress xACE - **eacl** NSEL/ASA egress ACL - **eace** NSEL/ASA egress ACE - **exace** NSEL/ASA egress xACE - NAT statistics - **nevent** NAT event - **nsrcip** NAT src IP address - **nsrcport** NAT src port - **ndstip** NAT dst IP address - **ndstport** NAT dst port ``` % nfdump -s srcip -s ip/flows/bytes -s record/bytes ``` - **-n** *num* Set the number of records to be printed to *num.* This option applies to **-s** statistics as well as to ordered output **-O** **-or** **-aggregated** **-records** **-a** The default is set to 10 for statistics and unlimited for the other use cases. To disable the limit, set *num* to 0. - **-o** *format* Sets the output format to print flow records. **has** **many** **different** **output** **formats** **already** **predefined.** *format* *may* *be* *one* *of* *the* *options* *below:* - **raw** Print the full flow record on multiple lines. This prints all available information. - **fmt:** *user* Print the flow records according the format *user.* This is a very flexible and powerful way to format flow records. See the section **OUTPUT** below for more details on how to compile your own format. - **csv:** *user* Print the flow records as user defined csv format. Use a ',' separated list of *user* defined output token compatible with *fmt* format. See the section **OUTPUT** below for more details on how to compile your own csv format. - **json** Print full record as a separate json object. - **ndjson** Print full record as a one line json object, sepatated by newline. Suitable for log processors such as logstash. - **csv** Print reocrd in csv format - format compatible to fmt **line** format. - **csv-fast** Replaces old pipe format. Basic record information only. Fast implementation. Already predefined fmt formats: - **line** Print each flow on one line. Default format. - **long** Print each flow on one line with more details - **biline** Same as *line,* but for bi-directional flows - **bilong** Same as *long,* but for bi-directional flows - **gline** Same as *line,* but add country code to IPs. If a geoDB file is supplied this is the default output format - **glong** Same as *long,* but add country code to IPs - **extended** Print each flow on one line with even more details. - **nsel** Print format for NSEL event records. Default format if NSEL/NAT support has been compiled in. - **nel** Print format for NAT event records. The **nfdump** config file may contain additional formats. If you want to add new formats or change existing ones, check the config file. IPv6 addresses are printed condensed in any **fmt** defined format to prevent cluttering the output with large blank blocks. A condensed IPV6 uses max 16 characters. If it is longer, then the middle part of the IP is cut out and replaced be "..". For previewing an output, this fits most needs. For a listing with the full IPV6 addresses add option **-6.** - **-6** Print full length of IPv6 addresses in output instead of condensed. - **-q** Quiet mode. Suppress the header line and the statistics at the bottom of text outputs. - **-N** Print plain numbers in output without scaling. Easier for output parsing with 3rd party tools. - **-i** *ident* Change the ident label in the file, specified by **-r** to *ident* - **-v** *flowfile* Verify the consistency of *flowfile* and print the file parameters and number of records. - **-E** *flowfile* Print the exporter and sampler list if found in *flowfile.* Additional statistics per exporter are printed with number of flows, packets and sequence errors. - **-x** *flowfile* This options works on nfdump version 1.6.x files only and may get removed in future. Scans and prints extension maps located in *flowfile* - **-z=lzo** Compress flow files with LZO1X-1 compression. Fastest compression. - **-z=bz2** Compress flow files with bz2 compression. Slow but most efficient. May be used for archiving files or if you are really short of spce. - **-z=lz4\[:level\]** Compress flow files with LZ4 compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels \> 5 may need more workers. See -W. - **-z=zstd\[:level\]** Compress flow files with ZSTD compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels \> 5 may need more workers. See -W. - **-W** *num* Sets the number of workers to compress flows. Defaults to 4. Must not be greater than the number of cores online. Useful for higher levels of compression for lz4 or zstd and large amount of flows per second. Please not, -W affects only writing flows. - **-J** *compress* Change compression for any number of files given by option **-r** *flowpath* Set *compress* to 0 for no compression or to any of: 1 or LZO, 2 or BZ2, 3 or LZ4. This option may be used for archiving flow files and changing the compression to use less disk space. - **-X** Compiles the *filter* syntax and dumps the filter engine table to stdout. This is for debugging purpose only. - **-Z** Check *filter* syntax and exit. Sets the return value accordingly. - **-R** *filelist* Select a range of files. This option is mainly used by old NfSen and documented here as legacy option. - /any/dir Read recursively all files in directory dir. - /dir/file Read all files beginning with file. - /dir/file1:file2 Read all files from file1 to file2. When using in combination with a sub hierarchy: /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from sub1/sub2/file1 sub3/sub4/file2 iterating over all required hierarchy levels. Note: files are read in alphabetical order. - **-M** *dirlist* Read the same file hierarchy from multiple directories. This option is mainly used by old NfSen and documented here as legacy option. Example: /any/path/to/dir1:dir2:dir3 etc. and will be expanded to the directories: /any/path/to/dir1, /any/path/to/dir2 and /any/path/to/dir3. Any number of colon separated directories may be given. A path ending with a wildcard '@' such as /any/path/to@ will expand automatically in a list of all existing sub directories /any/path/to/dir1:dir2:dir3. The files to read are specified by -r or -R and are expected to exist in all the given directories. The options -r and -R must not contain any directories when used in combination with -M. - **-T** Tag IP addresses with a prepending cntrl-A character, to allow output parsers to hook in. This option is mainly used by old NfSen and documented here as legacy option. - **-V** Print **nfdump** version and exit. - **-h** Print help text on stdout with all options and exit. *filter* selects, which records will be further processed. If no filter is given, all records will be processed. Otherwise, only those flows matching the filter will be processed. Any IP address in a filter may be specified as IPv4 or IPv6. The filter syntax is similar to tcpdump but adapted and extended for flow records. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case insensitive. A single filter primitive filters a single element of a flow record. A filter consists of one or more primitives, which are linked together: ``` expr and expr ``` ``` expr or expr ``` ``` not expr and (expr) ``` In all expressions, where a *number* is a valid argument, the *number* may be given as a normal decimal number or as a hex number prefixed by **0x** such as **0x22.** A decimal number may also contain a multiplication factor such as *K,* *M,* *G,* *T* which multiplies the number by the corresponding factor. For example *1K,* *2G* *etc.* String arguments may be single or double quoted or not quoted at all, if the string is not a reserved filter key word - **src** **geo** **CH** but **src** **geo** **'IN'** Possible filter primitives: - **@include** *file* Expands the content of *file* into the current filter - **count** *comp* *number* True if the comparison with the record counter matches *number* Each record gets assigned a record number at the time it is read from file. Therefore this record number is not unique and may change, depending on the order files are read. - **ident** *string* True if the record ident field matches *string.* This filter can be used to filter out different sources. - **inet** - **ipv4** True if source and destination IP of a record are IPv4 IPs. - **inet6** - **ipv6** True if source and destination IP of a record are IPv6 IPs. - **ttl** *comp* *num* True if IP ttl matches comparison. - **proto** *protocol* True if the record protocol field matches *protocol.* *protocol* can be a string such as **tcp**, **udp**, **icmp**, **ah**, **esp**, **ipip**, and many more or a protocol number, such as 6, 17 for protocol **tcp** and **udp**. - **tun** **proto** *protocol* True if the record tunnel protocol field matches *protocol.* *protocol* *may* *be* *a* *string* *or* *protocol* *number.* - **ip** *ipaddr* - **src** **ip** *ipaddr* - **dst** **ip** *ipaddr* True if the respective IP field of the record matches *ipaddr*. *ipaddr* may be an IPv4 or IPv6 address or a symbolic hostname. In this case a DNS lookup resolves the hostname to one or more IP addresses. If more than one IP results, all IPs are chained together in an **or** chain. (IP or IP or IP). If **ip** is not specified with **src** or **dst** the source or destination IP may match. If *ipaddr* is set to **tor** then flows are listed, if the respective ip addr is a to tor exit not. For this filter to work, you need a working nftordb. - **host** *ipaddr* **host** is just a synonym for **ip** (See above) - **ip** **in** \[*iplist*\] - **src** **in** **ip** \[*iplist*\] - **dst** **ip** \[*iplist*\] True if the respective IP field of the record is in *iplist.* *iplist* is a space or ',' separated list of IP addresses or networks in CIDR notation. This is the preferred way to search in large list of IP addresses and networks and is much more efficient than to chain all IP addresses together. (IP1 or IP2 or IP3). The *iplist* may contain several hundreds to thousand IPs and/or networks. For just a few IPs use an **or** chain, otherwise use an *iplist* If **ip** is not specified with **src** or **dst** the source or destination IP may match. - **net** *network* *netmask* - **src** **net** *network* *netmask* - **dst** **net** *network* *netmask* - **net** *network/netbits* - **src** **net** *network/netbits* - **dst** **net** *network/netbits* True if the respective IP field of the record matches the *network* if the corresponding *netmask* or *netbits* are applied to the IP address. If **net** is not specified with **src** or **dst** the source or destination IP may match. - **geo** *string* - **src** **geo** *string* - **dst** **geo** *string* True, if the 2-letter country code resolved by geolookup of the source or destination IP address matches *string.* This filter works only, if a valid geoDB is specified. See geo location option above. The 2-letter country code corresponds to the maxmind DB definitions. if **geo** is not specified with **src** or **dst** the source or destination geo location code may match. Please note: country codes, which match **nfdump** filter language reserved words such as IN, LT etc must be explicitly quoted to be recoginzed as string. - **tunip** *ipaddr* - **src** **tunip** *ipaddr* - **dst** **tunip** *ipaddr* True if the respective tunnel IP field of the record matches *ipaddr*. If **tunip** is not specified with **src** or **dst** the source or destination tunnel IP may match. - **port** *comp* *num* - **src** **port** *comp* *num* - **dst** **port** *comp* *num* True if the comparison of the respective port field matches *num* See *comp* for the comparator details. If **port** is not specified with **src** or **dst** the source or destination port may match. - **port** **in** \[*portlist*\] - **src** **port** **in** \[*portlist*\] - **dst** **port** **in** \[*portlist*\] True if the respective port field of the record is in *portlist.* *portlist* is a space or ',' separated list of port numbers. This is the preferred way to search in large list of port numbers and is much more efficient than to chain all ports together. (PORT1 or PORT2 or PORT3). *portlist* may contain several hundreds to thousand of port numbers. If **port** is not specified with **src** or **dst** the source or destination port may match. - **icmp** **type** *num* - **icmp** **code** *num* True if the respective icmp field of the record matches *num.* This automatically implies **proto** **icmp.** - **engine** **type** *num* - **engine** **id** *num* - **sysid** *num* True if the respective fields of the record matches *num* engine type and ID are set by the exporting device, sysid refers to the **nfdump** collector internal assigned number. See also option **-E** above. - **if** *num* - **in** **if** *num* - **out** **if** *num* True if the respective interface fields of the record matches *num.* This ID may correspond to the SNMP ID of the interface but depends on the exporter. If **if** is not specified with **in** or **out** the input or output interface may match. - **as** *comp* *num* - **src** **as** *comp* *num* - **dst** **as** *comp* *num* - **prev** **as** *comp* *num* - **next** **as** *comp* *num* True if the comparison of the respective AS fields matches **nfdump** supports 32-bit AS numbers every where. Without or the source or destination AS may match. See *comp* for the comparator details. - **as** **in** \[*aslist*\] - **src** **as** **in** \[*aslist*\] - **dst** **as** **in** \[*aslist*\] - **prev** **as** **in** \[*aslist*\] - **next** **as** **in** \[*aslist*\] True if the respective AS field of the record is in *aslist.* *aslist* is a space or ',' separated list of AS numbers. This is the preferred way to search in large list of AS numbers and is much more efficient than to chain all ports together. *aslist* may contain several hundreds to thousand of AS numbers. If **as** is not specified with **src,** **dst,** **prev** or **next** the source or destination AS may match. - **mask** *bits* - **src** **mask** *bits* - **dst** **mask** *bits* True if the respective mask bit field of the record matches *bits* If **mask** is not specified with **src** or **dst** the source or destination mask bits may match. - **vlan** *num* - **src** **vlan** *num* - **dst** **vlan** *num* True if the respective vlan field of the record matches *num* If **vlan** is not specified with **src** or **dst** the source or destination vlan may match. - **flags** *tcpflags* True if the respective tcp flags field of the record matches any of the given *tcpflags.* *tcpflags* is a string combination of all flags to be tested: - **A** ACK. - **S** SYN. - **F** FIN. - **R** Reset. - **P** Push. - **U** Urgent. - **X** All flags on. The order of the flags within *tcpflags* is not relevant. Flags not mentioned are treated as don't care. In order to get those flows with only the SYN flag set, use the syntax ``` flags S and not flags AFRPU ``` - **router** **ip** *ipaddr* True if the ip address of the sending router matches *ipaddr* as valid IPv4/IPv6 address. - **next** **ip** *ipaddr* True if the field next-ip of the record matches *ipaddr* as valid IPv4/IPv6 address. - **bgp** **next** **ip** *ipaddr* True if the field bgpnext-ip of the record matches *ipaddr* as valid IPv4/IPv6 address. - **mac** *macaddr* - **in** **mac** *macaddr* - **in** **src** **mac** *macaddr* - **in** **dst** **mac** *macaddr* - **out** **mac** *macaddr* - **out** **src** **mac** *macaddr* - **out** **dst** **mac** *macaddr* True if the respective mac address field of the record matches *macaddr* By prepending **mac** with any combination of a direction specifier as defined by CISCO v9 the test is limited to those **mac** addresses only. Otherwise multiple matches are possible. Without any specifiers any **mac** address is tested against *macaddr* - **mpls** **labelN** *comp* *number* True if the comparison of the mpls label **N** with **N** as mpls label number 1..10 matches *number* Filters according a specific number in the mpls label stack. - **mpls** **eos** *comp* *number* True if the comparison of the end of stack mpls label matches *number* - **mpls** **expN** *comp* *number* True if the comparison of the experimental bits 0..7 of mpls label **N** with **N** as mpls label number 1..10 matches *number* - **packets** *comp* *num* - **in** **packets** *comp* *num* - **out** **packets** *comp* *num* True if the comparison of the packet counter in the flow record matches *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* Example: packets \> 1k. For a single flow **packets** and **in** **packets** is equivalent and describes the number of packets from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option **--B** ) the packet counter for the reverse flow can be tested with **out** **packet** - **bytes** *comp* *num* - **in** **bytes** *comp* *num* - **out** **bytes** *comp* *num* True if the comparison of the byte counter in the flow record matches *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* Example: bytes \> 1k **bytes** and **in** **bytes** is equivalent and describes the number of bytes from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option **--B** ) the byte counter for the reverse flow can be tested with **out** **bytes** - **flows** *comp* *num* True if the comparison of the flow counter in the flow record matches *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* For each received flow, the flow counter is set to 1, unless the exporter sends this information. If multiple flows are aggregated, this counter is increased respectively. - **tos** *num* True if the type of service field of the flow record matches *num* - **flowdir** *direction* True, if the flow direction field in the flow record matches *direction.* *direction* may be *ingress,* *egress,* *0* for ingress, or *1* for egress - **duration** *comp* *time* True if the calculated duration of a flow (tend - tstart) compares to *time.* The duration is specified in msec (milliseconds) - **pps** *comp* *num* True if the calculated value of in-packets/duration (packets per second) compares with the number *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* - **bps** *comp* *num* True if the calculated value of 8\*in-bytes/duration (bits per second) compares with the number *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* - **bpp** *comp* *num* True if the calculated value of in-bytes/in-packets (bytes per packet) compares with the number *num.* *num* may contain any valid scaling factor such as *k,* *m,* *g* - **observation** **domain** **id** *comp* *number* - **observation** **point** **id** *comp* *number* True if the comparison of the observation domain ID or point ID field respectively matches *number* - payload filters Some exporters, such as **yaf** or the nfdump collector **nfpcap** **can** **send** **payload** **data** **along** **the** **netflow** **information.** **If** **such** payloads are sent it can be filtered according the filter primitives below: - **payload** **content** *'string'* True if the string *string* is found in the payload data. *string* must be quoted with single or double quotes: 'string', “string” - **payload** **regex** *'regex'* - **payload** **regex** *'regex'* *flags* True if *regex* matches the payload data. **regex** searches over the full payload length. A ' ' byte does not stop the match process. *regex* must be quoted with single or double quotes: 'regex' or “regex” The regex engine understands the following reduced syntax: - **(...)** subexpressions/capture ranges - \| the "or" operator - **^and** **$** anchors - - **\[...\]** and **\[^...\]** character classes - **?,** **\*,** **+,** simple quantifiers - **\*?,** **+?,** **??** lazy quantifiers - **{\},** **{\,\}** complex quantifiers *flags* are optional can be: - **m** multiline - **i** case insensitive matching - **s** - **payload** **ssl** **defined** True, if the payload contains the start of a valid SSL/TLS handshake - **payload** **ssl** **version** *version* True, if the payload contains the start of a valid SSL handshake and the SSL/TLS version matches *version* Valid versions are 1.0, 2.0, 3.0 - **payload** **tls** **version** *version* True, if the payload contains the start of a valid TLS handshake and the TLS version matches *version* Valid versions are 1.0, 1.1, 1.2, 1.3 - **payload** **tls** **sni** *sniname* True, if the payload contains the start of a valid TLS handshake and the TLS sni name contains the string *sniname* - **payload** **ja3** *md5string* True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja3 value of the handshake matches *md5string* Depending on client or server SSL handshake, either ja3 or ja3s is calculated. - **payload** **ja3** **defined** True, if the payload contains the start of a valid SSL/TLS handshake and a valid ja3 value can be calculated. Useful to mask out all flow records with no SSL/TLS traffic in order to generate a **-s** **ja3** statistic - **payload** **ja4** *ja4string* True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja4 value of the client TLS handshake matches *ja4string* - **payload** **jas4** *ja4Sstring* True, if the payload contains the start of a valid SSL/TLS handshake and the calculated ja4 value of the server TLS handshake matches *ja4Sstring* - **payload** **ja4** **defined** True, if the payload contains the start of a valid SSL/TLS handshake and a valid ja4/ja4s value can be calculated. Useful to mask out all flow records with no SSL/TLS traffic in order to generate a **-s** **ja3** or a **-s** **ja3s** statistic - OpenBSD pflog implemented elements - **pf** **action** *action* True, if the respective pflog action field compares to one of *pass,* *block,* *scrub,* *noscrub,* *nat,* *nonat,* *binat,* *nobinat,* *rdr,* *nordr,* *synblock,* *defer,* *match,* *divert,* *rt,* *afrt* - **pf** **reason** *reason* True, if the respective pflog reason field compares to one of *match,* *bad-offset,* *fragment,* *short,* *normalize,* *memory,* *bad-timestamp,* *congestion,* *ip-option,* *proto-cksum,* *state-mismatch,* *state-insert,* *state-limit,* *src-limit,* *synproxy,* *translate,* *no-route* - **pf** **rule** *ruleNr* True, if the respective pflog rule number field matches *ruleNr* - **pf** **dir** *in\|out* True, if the respective pflog rule direction field matches *in* or *out* - **pf** **interface** *interfaceName* True, if the respective pflog rule interface name field matches the string *interfaceName* - nprobe implemented elements - **client** **latency** *comp* *time* - **server** **latency** *comp* *time* True, if the respective latency field in the flow record compares to *time.* *time* is specified in msec. - CISCO ASA, network security event logging (NSEL) and NAT event logging (NEL) specific filters: - NSEL specific filters: - **asa** **event** *event* True if the NSEL event type of an event record matches *event* which may be: *ignore,* *create,* *term,* *delete,* *deny* - **asa** **event** *comp* *number* True if the comparison of the NSEL event type of an event records matches *number* as a number. - **asa** **event** **denied** *reason* True if the event denied type of an event records matches *reason* which may be *ingress,* *egress,* *interface,* *nosyn* - **asa** **xevent** *comp* *num* True, if the comparison of the extended event field of the event record matches *num* - **xip** *ipaddr* - **src** **xip** *ipaddr* - **dst** **xip** *ipaddr* True, if the field of the translated source or destination IP address matches *ipaddr* if **xip** is specified without **src** or **dst** both IP addresses may match. - **xport** *ipaddr* - **src** **xport** *ipaddr* - **dst** **xport** *ipaddr* True, if the field of the translated source or destination IP address matches *ipaddr* if **xport** is specified without **src** or **dst** both ports may match. - **xnet** *network/mask* - **src** **xnet** *network/mask* - **dst** **xnet** *network/mask* True if the translated source or destination IP address matches *network* if mask *mask* is applied. if **xnet** is specified without **src** or **dst** both IP addresses may match. - **ingress** **ACL** *comp* *number* - **ingress** **ACE** *comp* *number* - **ingress** **XACE** *comp* *number* True if the comparison of the respective ingress field matches *number* - **egress** **ACL** *comp* *number* True if the comparison of the egress field matches *number* - NEL specific filters: - **nat** **event** **event** True if the NEL event type of an event record matches *event.* *event* may be *add,* *delete* - **nat** **event** *comp* *number* True if the comparison of the NEL event type of an event records matches *number* as a number. - **nip** *ipaddr* - **src** **nip** *ipaddr* - **dst** **nip** *ipaddr* True, if the field of the nat source or destination IP address matches *ipaddr* if **nip** is specified without **src** or **dst** both IP addresses may match. It Cm nport Ar number - **src** **nport** *number* - **dst** **nport** *number* True, if the field of the nat source or destination port matches *number* if **nip** is specified without **src** or **dst** both ports may match. - **ingress** **vrf** *number* True, if the field of the ingess vrf field of the event record matches *number* - **pblock** **start** *comp* *number* - **pblock** **step** *comp* *number* - **pblock** **end** *comp* *number* True if the comparison of the start, step or end of the NAT port block in the event record matches *number* - **port** **in** **pblock** - **src** **port** **in** **pblock** - **dst** **port** **in** **pblock** True, if the source or destination port field matches the NAT port block range - *comp* Many filter elements support the comparison with a number. The following comparators are supported for each of those filters: **=,** **==,** **\>,** **\<,** **\>=,** **\<=** To prevent collisions with bash interpretation, alternative comparators are available: **EQ,** **LT,** **GT,** **LE,** **GE** If comp is omitted, '==' is assumed. ## OUTPUT FORMAT This section describes how output formats are compiled. **nfdump** has a lot of already pre-defined output formats such as *raw,* *json,* *ndjson,* *csv* etc. One line formats supplied with option **-o** can be compiled from various elements of a flow record. As a flow record contains many different elements it is often useful to compile an output format for specific needs. ### Format description The output format is specified by **-o** “fmt: *string*” for line output or **-o** “csv: *string*” for csv output. *String* contains the field *tags* to be printed as well as other characters if needed. A *tag* starts with a **%** sign followed by the field name. *tags* are separated by spaces for line format or by ',' for csv format. Characters or other strings, not starting with a **%** sign are copied literally to the output. Example: ``` -o “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl” ``` ``` -o “csv:%ts,%td,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl” ``` The difference between *fmt* and *csv* is the printing of the header line. Furthermore *csv* does not have a flow summary at the end. *csv* output can be processed by any other post processing tool, which understands csv data. The examples above show the definitions of the predined formats **line** and **csv.** It adds the elements *tstart* *duration* *protocol* *source* *IP* *address/port* followed by the literal characters -\> and *destination* *ip* *address/port* *packets,* *bytes,* *flows* counter. Depending on the task, different output formats are required to see the required fields of a flow record. You can either extend a predefined format or specify a new one at the command line. Example: Extend the predefined format **long** with the the IP address of the sending router ``` -o “fmt:%long %ra” ``` Predefined formats can be extended by simply add their name with a **%** sign somewhere in the format string. As described under the output option **-o** ### Format definition **nfdump** has already many formats predefined. Most of the time, these format are good enough. Sometimes you may need different formats, which can be compiled as described above. In order to prevent adding the same often used output format each time you run **nfdump** a new output format may be define in the config file *nfdump.conf* The file nfdump.conf.dist contains the definition of the already hard coded formats. These may be uncommented and changed according to the specific needs. New formats may be added using the following syntax: ``` fmt.newname = “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl” ``` with *newname* any new or existing definition of output formats. Existing formats are overwritten with the new definition. You may also define a custom csv output format such as: ``` csv.newname = “csv:%tsr,%ter,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl” ``` Please note, that *newname* must be unique for all formats in *fmt* as well as *csv* ### Tag definition The following list contains all tags, which are available to compile the output format: - **%\** Inserts the predefined format at this position. e.g. %line - **%cnt** Record counter. record numbers are assigned dynamically assigned while reading read from file. - **%nfv** Netflow version. - **%ts** Start Time - first seen - **%tfs** First seen - identical to %ts - **%tsr** Start Time, but in fractional seconds since the epoch (1970-01-01) UNIX format. - **%tsg** Start Time GMT - first seem - **%te** End Time - last seen - **%ter** End Time, in fractional seconds - **%teg** End Time GMT - last seen - **%tr** Time the flow was received by the collector - **%trr** Time the flow was received, in fractional seconds - **%trg** Time GMT the flow was received by the collector - **%td** Duration of flow. Displayed in ddHHMMSS.msec - **%pr** Transort protocol - **%exp** Exporter ID - **%eng** Engine Type/ID - **%lbl** Flowlabel - **%sa** Source Address - **%da** Destination Address - **%sap** Source Address:Port - **%dap** Destination Address:Port - **%gsap** Source Address(country code):Port - **%gdap** Destination Address(country code):Port - **%sp** Source Port - **%dp** Destination Port - **%ttl** IP ttl - **%it** ICMP-type - **%ic** ICMP-code - **%sn** Source Network, mask applied - **%dn** Destination Network, mask applied - **%nh** Next-hop IP Address - **%nhb** BGP Next-hop IP Address - **%ra** Router IP Address - **%sas** Source AS - **%das** Destination AS - **%nas** Next AS - **%pas** Previous AS - **%in** Input Interface num - **%out** Output Interface num - **%pkt** Packets - default input - **%ipkt** Input Packets - **%opkt** Output Packets - **%byt** Bytes - default input - **%ibyt** Input Bytes - **%obyt** Output Bytes - **%fl** Flows - **%flg** TCP Flags - **%tos** Tos - default src - **%stos** Src Tos - **%dtos** Dst Tos - **%dir** Direction: ingress, egress - **%smk** Src mask - **%dmk** Dst mask - **%fwd** Forwarding Status - **%svln** Src vlan label - **%dvln** Dst vlan label - **%scvln** Src customer vlan label - **%dcvln** Dst customer vlan label - **%ismc** Input Src Mac Addr - **%odmc** Output Dst Mac Addr - **%idmc** Input Dst Mac Addr - **%osmc** Output Src Mac Addr - **%mpls1** MPLS label 1 - **%mpls2** MPLS label 2 - **%mpls3** MPLS label 3 - **%mpls4** MPLS label 4 - **%mpls5** MPLS label 5 - **%mpls6** MPLS label 6 - **%mpls7** MPLS label 7 - **%mpls8** MPLS label 8 - **%mpls9** MPLS label 9 - **%mpls10** MPLS label 10 - **%mpls** MPLS labels 1-10 - **%bps** bps - bits per second - **%pps** pps - packets per second - **%bpp** bps - Bytes per package - **%sc** src IP 2 letter country code - **%dc** dst IP 2 letter country code - **%sloc** src IP geo location info - **%dloc** dst IP geo location info - **%sasn** src AS organisation name - **%dasn** dst AS organisation name - **%stor** src IP 2 letter tor exit info: TX tor exit node - **%dtor** dst IP 2 letter tor exit info: TX tor exit node - **%n** new line char \\n - **%ipl** input payload - **%opl** output payload - **%nbid** nbar ID - **%ja3** ja3 hash - **%sni** sni name in tls handshake - **%nbnam** nbar name - **%odid** observation domainID - **%opid** observation pointID - OpenBSD pflog specific formats - **%pfifn** pflog interface name - **%pfact** pflog action - **%pfrea** pflog reason - **%pfdir** pflog direction - **%pfrule** pflog rule nr - NSEL specific formats - **%nfc** NSEL connection ID - **%evt** NSEL event - **%xevt** NSEL extended event - **%sgt** NSEL Source security group tag - **%msec** NSEL event time in msec - **%iacl** NSEL ingress ACL - **%eacl** NSEL egress ACL - **%xsa** NSEL XLATE src IP address - **%xda** NSEL XLATE dst IP address - **%xsp** NSEL XLATE src port - **%xdp** NSEL SLATE dst port - **%xsap** Xlate Source Address:Port - **%xdap** Xlate Destination Address:Port - **%uname** NSEL user name - NEL/NAT specific formats - **%nevt** NAT event - same as %evt - **%ivrf** NAT ingress VRF ID - **%evrf** NAT egress VRF ID - **%nsa** NAT src IP address - **%nda** NAT dst IP address - **%nsp** NAT src port - **%ndp** NAT dst port - **%pbstart** NAT pool block start - **%pbend** NAT pool block end - **%pbstep** NAT pool block step - **%pbsize** NAT pool block size - **%flid** Flow ID - **%isid** Nokia NAT in service ID - **%osid** Nokia NAT out service ID - **%nats** Nokia NAT string - Nprobe formats - **%cl** Client latency - **%sl** Server latency - **%al** Application latency ## EXAMPLES **nfdump** processes files created by any previous version of nfdump 1.6.x with some limitations for versions \< 1.6.17. In order to convert flow files to the new 1.7.x binary format use the following command to read//write files: ``` % nfdump -r oldfile -w newfile ``` Print a statistic about the top 20 IP addresses, once sorted by flows and once by bytes ``` % nfdump -r flowfile -s ip/flows/bytes -n 20 ``` Print two statistics, one about the source IP and one about the destination IP address limited to flow with either source or destination port 443 ``` % nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port 443' ``` Print a statistic about the IP pairs, which exchanged most traffic. ``` % nfdump -r flowfile -s record/bytes -A srcip,dstip ``` Print all flows in raw format with a HTTP header in the payload even if flow is not on port 80. ``` % nfdump -r flowfile -o raw “payload regex 'GET|POST'” ``` Print a statistic about all ja3 md5 sums for those flows, which a valid ja3 can be calculated ``` % nfdump -r flowfile -s ja5 -n 0 'payload ja3 defined' ``` Aggregate all flows and write the result back to a binary file, sorted by the start time ``` % nfdump -r flowfile -a -Otstart -w newfile ``` ## RETURN VALUES **nfdump** returns 0 on success and 255 if processing failed. ## SEE ALSO https://www.iana.org/assignments/ipfix/ipfix.xhtml https://www.cisco.com/en/US/technologies/tk648/tk362/technologies\_white\_paper09186a00800a3db9.html [nfcapd(1)](/nfcapd/1.md) [nfpcapd(1)](/nfpcapd/1.md) [sfcapd(1)](/sfcapd/1.md) geolookup(1) ## BUGS No software without bugs! Please report any bugs back to me.