NAME
dist2 - rdist wrapper
SYNOPSIS
dist2 [-o rdist_opts] [-c cluster] [-v [-n]] [-i] [node spec] [package.[fileset]...]
dist2 [-o rdist_opts] [-c cluster] [-v [-n]] [-i] [node spec] -f Distfile
DESCRIPTION
Dist2 is a wrapper for USC rdist which adds a powerful macro capability based on the genders file, and abstracts a collection of Distfiles into an rdist repository made up of packages and filesets. It operates as follows:
-
1) Chdir to /var/dist (the rdist repository) and verify that the dummy file VAR_DIST_IS_MOUNTED exists.
-
2) Determine the appropriate set of Distfiles corresponding to packages specified on the command line. If packages are specified, select Distfile.{package list}, otherwise Distfile.{all packages}.
-
3) Scan the Distfiles for macro references of the form ${attr_xyz} and prepend macro definitions based on the content of the genders file. In the example above, all nodes carrying the genders attribute xyz would be listed in the attr_xyz definition.
-
4) Send the resulting concatenation of Distfiles and macro definitions to the rdist program.
Node specification options are -w node1,node2,node3 or -g genders_attribute or -l In the absense of these options, all nodes targeted by the preprocessed Distfile (potentially all nodes appearing in the genders file) are updated.
-w limits the update to the listed nodes. Always specify the name as it appears in the genders file; generally the primary hostname as reported by the hostname command.
-g similarly selects the set of nodes that have the specified genders attribute.
-l selects the local node.
-i selects canonical hostnames (the left hand side of a genders file entry). The default on the SP is to convert these "initial_hostname" values to "reliable_hostname" via the SDR. Elsewhere, the conversion is based on taking the value of an "altname" genders attribute, if any. This option defeats this conversion.
-v is used for debugging Distfile problems. When rdist reports a syntax error, it refers to line numbers in the preprocessed Distfile. This option provides a means for the Distfile author to see the preprocessed version. If -n is also specified, line numbers will be prepended to each line.
/var/dist is actually used to manage rdisted files across several MPP systems. We refer to each MPP as a cluster, with the file /etc/clusters mapping hostnames to cluster names. Distfile macros of the form ${attr_xyz} implicitly refer to the local cluster. Specific clusters can be targeted by specifying a macro of the form ${clustername!attr_xyz}. These definitions expand to empty lists on clusters other than the one specified.
Put another way, ${attr_xyz} refers all nodes in all clusters with the xyz attribute, while ${clustername!attr_xyz} only refers to the nodes in clustername with the xyz attribute.
BUGS
When expanding macros, we actually refer to /var/dist/genders/genders.cluster instead of the installed version, /etc/genders. Since the genders file is under rdist control, referring directly to the source file eliminates a potential bootstrapping problem. On the other hand, it's ugly to hardcode the source path, and we refer to other files, such as /etc/clusters, in their installed location.
NFS mounting /var/dist simplifies our lives as long as the switch is up and the NFS servers are fast. When one of these assumptions fails to be true, our old method of rsyncing /var/dist to head nodes, and dshing rdists on the head nodes to the frames is probably superior. Even on a good day, sending out everything in /var/dist to all nodes of a 488-node SP is probably slower than the old method. Updates of small parts of /var/dist, however, really scream now.
There should be some way to run dist2 -l on a node before the switch comes up. Maybe a script that mounts /var/dist over the ethernet temporarily, performs the update, and then unmounts it? For this to work we would need a NetApp 100bT interface connected directly to the SP ethernet router. If invoked from an rc file on reboot, care should be taken so it does not execute on a whole machine reboot to avoid saturating the ethernet.