--- name: "xrootd" section: "1" platform: "linux" package: "root-system-xrootd" version: "5.18.00-2.3~lenny1" release: "Debian 5.0 (Lenny)" platforms: ["linux"] url: "https://new.carta.tech/xrootd" --- # xrootd(1) The eXtended ROOT file server daemon ## NAME xrootd - The eXtended ROOT file server daemon ## SYNOPSIS **xrootd** *\[options\] \[exported directories\]* ## DESCRIPTION This manual page documents briefly the **xrootd** program. **xrootd** is a **ROOT** remote file server daemon. ## STARTING XROOTD The simplest possible setup is one server and one or more clients: ``` +-- kan001 -----------+ +-----------+ +--------+ +---------+ | | Client |------->| Server | | Storage | | | TXNetFile | | xrootd |--| /data |-+ +-----------+ +--------+ +---------+ ``` As with the original [**rootd**(1)](/rootd/1.md) server, this is relatively easy to setup. You can start it as: ``` kan001> xrootd ``` Unlike the **rootd**server**(1),** this doesn't immediately go into background, but in fact produces some amount of logging information. Using some **xrootd** client application, you should be able to access a file on the server using the same syntax as with **TNetFile**/**rootd**, for example: ``` root://kan001//tmp/mydatafile.root ``` From a client-side application such as a ROOT interactive session, you should be able to open the file and perform operations on it: ``` root [1] myfile = TFile::Open("root://kan001//tmp/mydatafile.root"); root [2] myfile->ls(); XTNetFile** root://kan001//tmp/mydatafile.root XTNetFile* root://kan001//tmp/mydatafile.root KEY: TF1 fooHist;1 gaus KEY: TF1 barHist;1 gaus root [3] myfile->Close(); root [4] .q ``` ## ACCESSING AREAS OTHER THAN /tmp One important difference from [**rootd**(1)](/rootd/1.md) is that **xrootd** by default only allows one to access files in */tmp*. This is for security reasons so that a running **xrootd** does not give remote clients access to read (or write to!) everything visible as **xrootd**. A client accessing an area which is not explicitly exported by **xrootd e.g.,** *root://kan001//data/mydatafile.root* will get an error. You must specify explicitly any directories you wish to export via **xrootd**. This can be done on the command line when starting **xrootd**, for example: ``` kan001> xrootd /data ``` will export the */data* directory. ## MULTIPLE SERVERS Often it is the case that the data that an application wants to access is spread over multiple data servers. The client application thus needs to know (or determine from some external source) on which server any given data file is located. It can then access those files directly with URL's like: ``` root://kan001//data/aaaFile.root root://kan002//data/bbbFile.root ``` While this may work for small systems with few users, it has some disadvantages: - If users know statically where files are located it doesn't easily permit load balancing by moving files between servers or by replicating them to spread the load. - If an external catalog is read by the client in order to determine which server has the file, any changes to that catalog must be synchronized with what actually happens to the files on disk. - If the file isn't actually on disk, but is retrieved from somewhere else when the client asks for it (e.g. mass storage, discussed in later examples), there is no easy way to manage the disk space to insure that it is used efficiently and uniformly. The **xrootd** system provides a method for solving these problems via its *open load balancer* component. This provides a method for providing transparent access data servers. ``` +-- kan001 ----------------+ | +--------+ +---------+ | +--->| Server | | Storage | | + kanrdr + | | | olbd | | /data | | +-----------+ +--------+ | | +--| xrootd |--| |-+ | Client |--->| Server |------+ +--------+ +---------+ | TXNetFile | | xrootd | | | +-----------+ | olbd |---+ | +-- kan002 ----------------+ +--------+ | | +--------+ +---------+ | +--->| Server | | Storage | | | | | olbd | | /data | | | +--| xrootd |--| |-+ | +--------+ +---------+ | ... ``` In addition to the dataservers an additional machine (the *redirector* - **kanrdr** above) is added. It does not need to have any disk space itself as its sole role is to redirect the client dynamically to an actual data server machine which can serve the requested file. Thus client applications open all files via the redirector **kanrdr**: ``` root://kanrdr//data/aaaFile.root root://kanrdr//data/bbbFile.root ``` The redirection to the appropriate dataserver will take place automatically (with the proper server configuration, discussed below) and hence the user running the application doesn't need to know anything about the actual layout of the data over the data servers. Note also that after redirection the client is reading directly from the data server to which it was redirected. It does need the redirector any longer (at least until it needs to open a different file). To accomplish this a second daemon, the **olbd**, must be started on both the redirector and each of the dataservers as show in the figure above. Configuration of an **xrootd***/***olbd** system ## OPTIONS - **-l** *logfile* The primary advantage of using this option as opposed to simply redirecting the output to a file is that there is also a feature which does automatic log file rotation. At midnight each night **xrootd** will move the current *logfile* to *logfile.yyyymmdd* and then continue writing to *logfile*. - **-p** *port* As with [**rootd**(1)](/rootd/1.md), **xrootd** will started automatically on the IANA assigned port number 1094. If you need to start it on another port, that can be done using this option. Clients will need to specify this port when accessing the server, using an URL like *root://kan001:2525//tmp/mydatafile.root*. In general it is probably simplest to run xrootd on the standard port 1094. - **-c** *configfile* **xrootd** can be configured via a config file as well as via command line options. There are in fact many more options available via the config file and hence this is the recommended way of starting **xrootd** is to use this option. The config file *configfile* can have configuration directives as described below. - **-r** Flag this server as a redirector. ## CONFIGURATION FILES A configuration file for **xrootd** looks like ``` # Start the server on port 2525 xrd.port 2525 # Export files in /data only xrootd.export /data ``` in order to achieve (for example) what was done with command line options in the sections above. Note that lines beginning with **#** are taken to be comments and ignored. For the moment we just note that all config lines are of the form: ``` configclass.option [optval1] [optval2] [...] ``` It is also important to note that spelling errors in either *configclass* or *option* will result in the entire line being (silently) ignored. **Read-only data access**\ An important thing to note about **xrootd** access to files is that by default it allows any accessed file to be opened in update mode (and/or new files to be created). For many situations this is undesirable: the goal is simply to make data files available for others to read. The easiest thing to do in that case is simply to disallow all write access. To do this you need to add two lines to your config file: ``` xrootd.fslib /opt/xrootd/lib/libXrdOfs.so oss.readonly ``` in your configuration file. The first of these will cause the shared library for the optional ofs/oss file layer to be loaded (the exact path the *lib* area will depend on how you installed **xrootd**). This shared library provides a number of filesystem extensions. The second line disallows all write/update access using one of these options. More fine-grained configurations are possible, but globally disallowing writes like this should cover a common use case. **Multiple servers**\ As an example we take a redirector (called **kanrdr**) and two data servers (called **kan001** and **kan002**) with files **aaaFile.root** and **bbbFile.root** as above. For the redirector we use a config file: ``` # # redirector.cf # # xrootd xrootd.fslib /opt/xrootd/lib/libXrdOfs.so xrootd.export /data odc.manager kanrdr 3121 odc.trace redirect # olbd olb.port 3121 olb.allow host kanrdr.slac.stanford.edu olb.allow host kan001.slac.stanford.edu olb.allow host kan002.slac.stanford.edu ``` and for the data servers we use: ``` # # dataserver.cf # # xrootd xrootd.fslib /opt/xrootd/lib/libXrdOfs.so xrootd.export /data oss.readonly odc.manager kanrdr 3121 # olbd olb.port 3121 olb.subscribe kanrdr 3121 ``` As in the earlier examples, the actual path the to the **libXrdOfs.so** may depend on your installation. (And you should obviously substitute your machine names for **kanrdr**/**kan001**/**kan002**.) It is then possible to start the **xrootd** and **olbd** servers on each of the machines. On dataserver kan001: ``` xrootd -l kan001.xrdlog -c dataserver.cf & olbd -s -l kan001.olblog -c dataserver.cf & ``` On dataserver kan002: ``` xrootd -l kan002.xrdlog -c dataserver.cf & olbd -s -l kan002.olblog -c dataserver.cf & ``` And on the redirector kanrdr: ``` xrootd -r -l kanrdr.xrdlog -c redirector.cf & olbd -m -l kanrdr.olblog -c redirector.cf & ``` This should give you a working system capable of providing (readonly) access to data files on two servers transparently masqueraded behind a single redirector machine. **Testing the example load balanced system**\ Once the system is running we can try to open one of the files, for example: ``` root [1] myfile = TFile::Open("root://kanrdr//data/aaafile.root"); root [2] myfile->ls(); XTNetFile** root://kanrdr//data/aaafile.root XTNetFile* root://kanrdr//data/aaafile.root KEY: TF1 fooHist;1 gaus KEY: TF1 barHist;1 gaus ``` Your client will be redirected and connect itself to **kan001**, the actual location of the requested file. ## SEE ALSO [*rootd*(1)](/rootd/1.md), [*proofd*(1)](/proofd/1.md), [*system.rootdaemonrc*(1)](/system.rootdaemonrc/1.md) See also *http://xrootd.slac.stanford.edu* for more on the extended **ROOT** server. For more information on the **ROOT** system, please refer to *http://root.cern.ch* ⟨http://root.cern.ch/⟩ ## ORIGINAL AUTHORS The ROOT team (see web page above): **Rene Brun** and **Fons Rademakers** ## COPYRIGHT This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA ## AUTHOR This manual page was originally written by Christian Holm Christensen \, for the Debian GNU/Linux system and ROOT version 3.