--- name: "pmc.cmn-600" section: "3" platform: "freebsd" package: "base" version: "16.0-CURRENT-20261027-61c70dc16a79" release: "FreeBSD 16.0-CURRENT (2026-10-27)" platforms: ["freebsd"] url: "https://new.carta.tech/pmc.cmn-600" --- # pmc.cmn-600(3) Library for accessing the Arm CoreLink CMN-600 Coherent Mesh Network Controller performance counter events ## NAME **pmc.cmn-600** – Library for accessing the Arm CoreLink CMN-600 Coherent Mesh Network Controller performance counter events ## LIBRARY library “libpmc” ## SYNOPSIS **#include \** ## DESCRIPTION CMN-600 PMU counters may be configured to count any one of a defined set of hardware events. Unlike other performance counters, counters for the CMN-600 require the node ID to set up. Node ID information currently can be obtained one of two ways. Using bootverbose, for example set sysctl debug.bootverbose=1 and then load the [hwpmc(4)](/hwpmc/4.md) KLD module. The cmn600 module will be loaded automatically as a dependency: ``` $ sysctl debug.bootverbose=1 ``` ``` $ kldload hwpmc ``` Another way is to use sysctl to trigger dump of nodes tree to system console: ``` $ sysctl dev.cmn600.0.dump_nodes=1 ``` Some BIOS versions of dual-socket machines have no NUMA domain information in ACPI. In such cases, to get more accurate events statistics, set the kernel environment variable hint.cmn600.{unit}.domain={value}. Where {unit} is a cmn600 device unit number and {value} is the NUMA domain of the CPU package containing that CMN-600 controller. Example: ``` $ kenv hint.cmn600.0.domain=0 ``` ``` $ kenv hint.cmn600.1.domain=1 ``` ``` $ kldunload hwpmc cmn600 ``` ``` $ kldload hwpmc ``` Arm CoreLink CMN-600 Coherent Mesh Network Controller performance counters are documented in “Revision: r3p2”, *Arm CoreLink CMN-600 Coherent Mesh Network Technical Reference Manual*, ARM Limited, 2020. ### PMC Capabilities CMN-600 PMU counters support the following capabilities: | | | | --- | --- | | **Capability** | *Support* | | PMC\_CAP\_CASCADE | No | | PMC\_CAP\_EDGE | No | | PMC\_CAP\_INTERRUPT | Yes | | PMC\_CAP\_INVERT | No | | PMC\_CAP\_READ | Yes | | PMC\_CAP\_PRECISE | No | | PMC\_CAP\_SYSTEM | Yes | | PMC\_CAP\_TAGGING | No | | PMC\_CAP\_THRESHOLD | Yes | | PMC\_CAP\_USER | No | | PMC\_CAP\_WRITE | Yes | ### Event Qualifiers Event specifiers for these PMCs support the following common qualifiers: - nodeid=*nodeid* Request counting for specific event at node *nodeid*. - occupancy=*value* Filtering by occupancy type. - xpport=*port* Count only events matched by *port*. (East, West, North, South, devport0, devport1 or numeric 0 to 5) - xpchannel=*channel* Filter events by XP node channel. (REQ, RSP, SNP, DAT or 0, 1, 2, 3) ### Class Name Prefix These PMCs are named using a class name prefix of “CMN600\_PMU\_”. ### Event Specifiers The following list of PMC events are available: ### DVM node events - **dn\_rxreq\_dvmop** Number of DVMOP requests received. This includes all the sub-types include TLB invalidate, Branch predictor invalidate, instruction cache (physical and virtual) invalidate. - **dn\_rxreq\_dvmsync** Number of DVM Sync requests received. - **dn\_rxreq\_dvmop\_vmid\_filtered** Number of incoming DVMOP requests that are subject to VMID based filtering. This is a measure of the effectiveness of VMID based filtering and potential reduction in DVM snoops. - **dn\_rxreq\_retried** Number of incoming requests that are retried. This is a measure of the retry rate. - **dn\_rxreq\_trk\_occupancy** Counts the tracker occupancy in DN. "occupancy": All, dvmop, dvmsync - **dn\_rxreq\_tlbi\_dvmop** Number of DVMOP TLB invalidate requests received. - **dn\_rxreq\_bpi\_dvmop** Number of DVMOP Branch predictor invalidate requests received. - **dn\_rxreq\_pici\_dvmop** Number of DVMOP physical instruction cache invalidate requests received. - **dn\_rxreq\_vivi\_dvmop** Number of DVMOP virtual instruction cache invalidate requests received. - **dn\_rxreq\_dvmop\_other\_filtered** Number of DVM op requests to RNDs, BPI or PICI/VICI, that were filtered - **dn\_rxreq\_snp\_sent** Number of SNPs sent to RNs - **dn\_rxreq\_snp\_stalled** Number of SNPs stalled to RNs due to lack of Crds - **dn\_rxreq\_trk\_full** DVM tracker full counter ### HN-F node events - **hnf\_cache\_miss** Counts total cache misses in first lookup result (high priority) - **hnf\_slc\_sf\_cache\_access** Counts number of cache accesses in first access (high priority) - **hnf\_cache\_fill** Counts total allocations in HN SLC (all cache line allocations to SLC) - **hnf\_pocq\_retry** Counts number of retried requests - **hnf\_pocq\_reqs\_recvd** Counts number of requests received by HN - **hnf\_sf\_hit** Counts number of SF hits - **hnf\_sf\_evictions** Counts number of SF eviction cache invalidations initiated - **hnf\_dir\_snoops\_sent** Counts number of directed snoops sent (not including SF back invalidation) - **hnf\_brd\_snoops\_sent** Counts number of multicast snoops sent (not including SF back invalidation) - **hnf\_slc\_eviction** Counts number of SLC evictions (dirty only) - **hnf\_slc\_fill\_invalid\_way** Counts number of SLC fills to an invalid way - **hnf\_mc\_retries** Counts number of retried transactions by the MC - **hnf\_mc\_reqs** Counts number of requests sent to MC - **hnf\_qos\_hh\_retry** Counts number of times a HighHigh priority request is protocol-retried at the HN-F - **hnf\_qos\_pocq** Counts the POCQ occupancy in HN-F. Support argument "occupancy". Accept: All, Read, Write, Atomic, Stash. Default: All. - **hnf\_pocq\_addrhaz** Counts number of POCQ address hazards upon allocation - **hnf\_pocq\_atomic\_addrhaz** Counts number of POCQ address hazards upon allocation for atomic operations - **hnf\_ld\_st\_swp\_adq\_full** Counts number of times ADQ is full for Ld/St/SWP type atomic operations while POCQ has pending operations - **hnf\_cmp\_adq\_full** Counts number of times ADQ is full for CMP type atomic operations while POCQ has pending operations - **hnf\_txdat\_stall** Counts number of times HN-F has a pending TXDAT flit but no credits to upload - **hnf\_txrsp\_stall** Counts number of times HN-F has a pending TXRSP flit but no credits to upload - **hnf\_seq\_full** Counts number of times requests are replayed in SLC pipe due to SEQ being full - **hnf\_seq\_hit** Counts number of times a request in SLC hit a pending SF eviction in SEQ - **hnf\_snp\_sent** Counts number of snoops sent including directed, multicast, and SF back invalidation - **hnf\_sfbi\_dir\_snp\_sent** Counts number of times directed snoops were sent due to SF back invalidation - **hnf\_sfbi\_brd\_snp\_sent** Counts number of times multicast snoops were sent due to SF back invalidation - **hnf\_snp\_sent\_untrk** Counts number of times snooped were sent due to untracked RNF's - **hnf\_intv\_dirty** Counts number of times SF back invalidation resulted in dirty line intervention from the RN - **hnf\_stash\_snp\_sent** Counts number of times stash snoops were sent - **hnf\_stash\_data\_pull** Counts number of times stash snoops resulted in data pull from the RN - **hnf\_snp\_fwded** Counts number of times data forward snoops were sent ### HN-I node events - **hni\_rrt\_rd\_occ\_cnt\_ovfl** RRT read occupancy count overflow - **hni\_rrt\_wr\_occ\_cnt\_ovfl** RRT write occupancy count overflow - **hni\_rdt\_rd\_occ\_cnt\_ovfl** RDT read occupancy count overflow - **hni\_rdt\_wr\_occ\_cnt\_ovfl** RDT write occupancy count overflow - **hni\_wdb\_occ\_cnt\_ovfl** WDB occupancy count overflow - **hni\_rrt\_rd\_alloc** RRT read allocation - **hni\_rrt\_wr\_alloc** RRT write allocation - **hni\_rdt\_rd\_alloc** RDT read allocation - **hni\_rdt\_wr\_alloc** RDT write allocation - **hni\_wdb\_alloc** WDB allocation - **hni\_txrsp\_retryack** RETRYACK TXRSP flit sent - **hni\_arvalid\_no\_arready** ARVALID set without ARREADY event - **hni\_arready\_no\_arvalid** ARREADY set without ARVALID event - **hni\_awvalid\_no\_awready** AWVALID set without AWREADY event - **hni\_awready\_no\_awvalid** AWREADY set without AWVALID event - **hni\_wvalid\_no\_wready** WVALID set without WREADY event - **hni\_txdat\_stall** TXDAT stall (TXDAT valid but no link credit available) - **hni\_nonpcie\_serialization** Non-PCIe serialization event - **hni\_pcie\_serialization** PCIe serialization event ### XP node events - **xp\_txflit\_valid** Number of flits transmitted on a specified port and CHI channel. This is a measure of the flit transfer bandwidth from an XP. Note: On device ports, this event also includes link flit transfers. - **xp\_txflit\_stall** Number of cycles when a flit is stalled at an XP waiting for link credits at a specified port and CHI channel. This is a measure of the flit traffic congestion on the mesh and at the flit download ports. - **xp\_partial\_dat\_flit** Number of times when a partial DAT flit is uploaded onto the mesh from a RN-F\_CHIA port. Partial DAT flit transmission occurs when XP is not able to combine two 128b DAT flits and send them over the 256b DAT channel. This can happen under 2 circumstances: 1. Only one 128b DAT flit is received within a transmission time window. 2. Two 128b DAT flits are received but they are not two halves of a\ single 256b word. ### SBSX node events - **sbsx\_rd\_req** Read request - **sbsx\_wr\_req** Write request - **sbsx\_cmo\_req** CMO request - **sbsx\_txrsp\_retryack** RETRYACK TXRSP flit sent - **sbsx\_txdat\_flitv** TXDAT flit seen - **sbsx\_txrsp\_flitv** TXRSP flit seen - **sbsx\_rd\_req\_trkr\_occ\_cnt\_ovfl** Read request tracker occupancy count overflow - **sbsx\_wr\_req\_trkr\_occ\_cnt\_ovfl** Write request tracker occupancy count overflow - **sbsx\_cmo\_req\_trkr\_occ\_cnt\_ovfl** CMO request tracker occupancy count overflow - **sbsx\_wdb\_occ\_cnt\_ovfl** WDB occupancy count overflow - **sbsx\_rd\_axi\_trkr\_occ\_cnt\_ovfl** Read AXI pending tracker occupancy count overflow - **sbsx\_cmo\_axi\_trkr\_occ\_cnt\_ovfl** CMO AXI pending tracker occupancy count overflow - **sbsx\_arvalid\_no\_arready** ARVALID set without ARREADY - **sbsx\_awvalid\_no\_awready** AWVALID set without AWREADY - **sbsx\_wvalid\_no\_wready** WVALID set without WREADY - **sbsx\_txdat\_stall** TXDAT stall (TXDAT valid but no link credit available) - **sbsx\_txrsp\_stall** TXRSP stall (TXRSP valid but no link credit available) ### RN-D node events - **rnd\_s0\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 0. This is a measure of the read bandwidth, including CMO responses. - **rnd\_s1\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 1. This is a measure of the read bandwidth, including CMO responses. - **rnd\_s2\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 2. This is a measure of the read bandwidth, including CMO responses. - **rnd\_rxdat\_flits** Number of RXDAT flits received. This is a measure of the true read data bandwidth, excluding CMOs. - **rnd\_txdat\_flits** Number of TXDAT flits dispatched. This is a measure of the write bandwidth. - **rnd\_txreq\_flits\_total** Number of TXREQ flits dispatched. This is a measure of the total request bandwidth. - **rnd\_txreq\_flits\_retried** Number of retried TXREQ flits dispatched. This is a measure of the retry rate. - **rnd\_rrt\_occ\_ovfl** All entries in the read request tracker are occupied. This is a measure of oversubscription in the read request tracker. - **rnd\_wrt\_occ\_ovfl** All entries in the write request tracker are occupied. This is a measure of oversubscription in the write request tracker. - **rnd\_txreq\_flits\_replayed** Number of replayed TXREQ flits. This is the measure of replay rate. - **rnd\_wrcancel\_sent** Number of write data cancels sent. This is the measure of write cancel rate. - **rnd\_s0\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 0. This is a measure of write bandwidth on AXI port 0. - **rnd\_s1\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 1. This is a measure of write bandwidth on AXI port 1. - **rnd\_s2\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 2. This is a measure of write bandwidth on AXI port 2. - **rnd\_rrt\_alloc** Number of allocations in the read request tracker. This is a measure of read transaction count. - **rnd\_wrt\_alloc** Number of allocations in the write request tracker. This is a measure of write transaction count. - **rnd\_rdb\_unord** Number of cycles for which Read Data Buffer state machine is in Unordered Mode. - **rnd\_rdb\_replay** Number of cycles for which Read Data Buffer state machine is in Replay mode - **rnd\_rdb\_hybrid** Number of cycles for which Read Data Buffer state machine is in hybrid mode. Hybrid mode is where there is mix of ordered/unordered traffic. - **rnd\_rdb\_ord** Number of cycles for which Read Data Buffer state machine is in ordered Mode. ### RN-I node events - **rni\_s0\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 0. This is a measure of the read bandwidth, including CMO responses. - **rni\_s1\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 1. This is a measure of the read bandwidth, including CMO responses. - **rni\_s2\_rdata\_beats** Number of RData beats, RVALID and RREADY, dispatched on port 2. This is a measure of the read bandwidth, including CMO responses. - **rni\_rxdat\_flits** Number of RXDAT flits received. This is a measure of the true read data bandwidth, excluding CMOs. - **rni\_txdat\_flits** Number of TXDAT flits dispatched. This is a measure of the write bandwidth. - **rni\_txreq\_flits\_total** Number of TXREQ flits dispatched. This is a measure of the total request bandwidth. - **rni\_txreq\_flits\_retried** Number of retried TXREQ flits dispatched. This is a measure of the retry rate. - **rni\_rrt\_occ\_ovfl** All entries in the read request tracker are occupied. This is a measure of oversubscription in the read request tracker. - **rni\_wrt\_occ\_ovfl** All entries in the write request tracker are occupied. This is a measure of oversubscription in the write request tracker. - **rni\_txreq\_flits\_replayed** Number of replayed TXREQ flits. This is the measure of replay rate. - **rni\_wrcancel\_sent** Number of write data cancels sent. This is the measure of write cancel rate - **rni\_s0\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 0. This is a measure of write bandwidth on AXI port 0. - **rni\_s1\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 1. This is a measure of write bandwidth on AXI port 1. - **rni\_s2\_wdata\_beats** Number of WData beats, WVALID and WREADY, dispatched on port 2. This is a measure of write bandwidth on AXI port 2. - **rni\_rrt\_alloc** Number of allocations in the read request tracker. This is a measure of read transaction count. - **rni\_wrt\_alloc** Number of allocations in the write request tracker. This is a measure of write transaction count - **rni\_rdb\_unord** Number of cycles for which Read Data Buffer state machine is in Unordered Mode. - **rni\_rdb\_replay** Number of cycles for which Read Data Buffer state machine is in Replay mode - **rni\_rdb\_hybrid** Number of cycles for which Read Data Buffer state machine is in hybrid mode. Hybrid mode is where there is mix of ordered/unordered traffic. - **rni\_rdb\_ord** Number of cycles for which Read Data Buffer state machine is in ordered Mode. ### CXHA node events (Note: CXHA events descriptions are guessed) - **cxha\_rddatbyp** Number of Read DAT Bypass - **cxha\_chirsp\_up\_stall** Number of CHI RSP up Stall - **cxha\_chidat\_up\_stall** Number of CHI DAT up Stall - **cxha\_snppcrd\_lnk0\_stall** Number of Snoop Pcrd Stall on Link 0 - **cxha\_snppcrd\_lnk1\_stall** Number of Snoop Pcrd Stall on Link 1 - **cxha\_snppcrd\_lnk2\_stall** Number of Snoop Pcrd Stall on Link 2 - **cxha\_reqtrk\_occ** Request Tracker Occupancy - **cxha\_rdb\_occ** Read Data Buffer Occupancy - **cxha\_rdbbyp\_occ** Read Data Buffer Bypass Occupancy - **cxha\_wdb\_occ** Write Data Buffer Occupancy - **cxha\_snptrk\_occ** Snoop Tracker Occupancy - **cxha\_sdb\_occ** SDB Occupancy - **cxha\_snphaz\_occ** Snoop Hazard Occupancy ### CXRA node events - **cxra\_req\_trk\_occ** Request tracker occupancy - **cxra\_snp\_trk\_occ** Snoop tracker occupancy - **cxra\_rd\_dat\_buf\_occ** Read data buffer occupancy - **cxra\_wr\_dat\_buf\_occ** Write data buffer occupancy - **cxra\_snp\_sink\_buf\_occ** Snoop sink buffer occupancy - **cxra\_snp\_bcasts** Snoop broadcasts - **cxra\_req\_chains** Number of request chains formed larger than one - **cxra\_req\_chain\_avg\_len** Average size of request chains, only for chain sizes larger than one - **cxra\_chi\_rsp\_upload\_stalls** Local RA upload stalls to CHI because of contention with HA - **cxra\_chi\_dat\_upload\_stalls** Local RA upload stalls to CHI because of contention with HA - **cxra\_dat\_pcrd\_stalls\_lnk0** Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 0 - **cxra\_dat\_pcrd\_stalls\_lnk1** Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 1 - **cxra\_dat\_pcrd\_stalls\_lnk2** Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 2 - **cxra\_req\_pcrd\_stalls\_lnk0** Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 0 - **cxra\_req\_pcrd\_stalls\_lnk1** Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 1 - **cxra\_req\_pcrd\_stalls\_lnk2** Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 2 - **cxra\_ext\_rsp\_stall** CHI external RSP stall - **cxra\_ext\_dat\_stall** CHI external DAT stall ### CXLA node events - **cxla\_rx\_tlp\_link0** RX TLP for Link 0 - **cxla\_rx\_tlp\_link1** RX TLP for Link 1 - **cxla\_rx\_tlp\_link2** RX TLP for Link 2 - **cxla\_tx\_tlp\_link0** TX TLP for Link 0 - **cxla\_tx\_tlp\_link1** TX TLP for Link 1 - **cxla\_tx\_tlp\_link2** TX TLP for Link 2 - **cxla\_rx\_cxs\_link0** RX CXS for Link 0 - **cxla\_rx\_cxs\_link1** RX CXS for Link 1 - **cxla\_rx\_cxs\_link2** RX CXS for Link 2 - **cxla\_tx\_cxs\_link0** TX CXS for Link 0 - **cxla\_tx\_cxs\_link1** TX CXS for Link 1 - **cxla\_tx\_cxs\_link2** TX CXS for Link 2 - **cxla\_avg\_rx\_tlp\_sz\_dws** Average RX TLP size in DWs - **cxla\_avg\_tx\_tlp\_sz\_dws** Average TX TLP size in DWs - **cxla\_avg\_rx\_tlp\_sz\_ccix\_msg** Average RX TLP size in CCIX messages - **cxla\_avg\_tx\_tlp\_sz\_ccix\_msg** Average TX TLP size in CCIX messages - **cxla\_avg\_sz\_rx\_cxs\_dw\_beat** Average size of RX CXS in DWs within a beat - **cxla\_avg\_sz\_tx\_cxs\_dw\_beat** Average size of TX CXS in DWs within a beat - **cxla\_tx\_cxs\_link\_credit\_backpressure** TX CXS link credit backpressure - **cxla\_rx\_tlp\_buffer\_full** RX TLP buffer full and backpressured - **cxla\_tx\_tlp\_buffer\_full** TX TLP buffer full and backpressured - **cxla\_avg\_latency\_process\_rx\_tlp** Average latency to process an RX TLP - **cxla\_avg\_latency\_form\_tx\_tlp** Average latency to form a TX TLP ## SEE ALSO [pmc(3)](/pmc/3.md), [pmc.amd(3)](/pmc.amd/3.md), [pmc.atom(3)](/pmc.atom/3.md), [pmc.core(3)](/pmc.core/3.md), [pmc.core2(3)](/pmc.core2/3.md), [pmc.corei7(3)](/pmc.corei7/3.md), [pmc.corei7uc(3)](/pmc.corei7uc/3.md), [pmc.iaf(3)](/pmc.iaf/3.md), [pmc.soft(3)](/pmc.soft/3.md), [pmc.tsc(3)](/pmc.tsc/3.md), [pmc.westmere(3)](/pmc.westmere/3.md), [pmc.westmereuc(3)](/pmc.westmereuc/3.md), [pmc\_cpuinfo(3)](/pmc_cpuinfo/3.md), [pmclog(3)](/pmclog/3.md), [hwpmc(4)](/hwpmc/4.md) ## HISTORY The **pmc** library first appeared in FreeBSD 6.0.\ The **pmc.cmn-600** driver was added in FreeBSD 14.0. ## AUTHORS The library “libpmc” library was written by Joseph Koshy ⟨*jkoshy@FreeBSD.org*⟩, Oleksandr Rybalko ⟨*ray@FreeBSD.org*⟩.\ The CMN-600 PMU driver was sponsored by ARM Ltd. This manual page was written by Oleksandr Rybalko ⟨*ray@FreeBSD.org*⟩.