--- name: "mkls-lR" section: "8" platform: "linux" package: "mirror" version: "2.9-15" release: "Debian 2.2 (Potato)" platforms: ["linux"] url: "https://new.carta.tech/mkls-lR/8" --- # mkls-lR(8) Make ls-lR files on FTP server for mirror use ## NAME mkls-lR - Make ls-lR files on FTP server for mirror use ## SYNOPSIS **mkls-lR** **-d** *dirpath* \[**-a**\] \[**-n** *basename*\] \[**-t** *tod*\] \[**-h**\] \[**--help**\] \[**-V**\] \[**--version**\] \[**"**\] \[**-I***path* ...\] \[*path* ...\] \[**"**\] ## DESCRIPTION **mkls-lR** creates or updates the four files **timezone**, **ls-lR.gz**, **ls-lR.patch.gz** amd **ls-lR.times** in the directory *dirpath*. With the **-a** option a fifth file **ls-lR** is included. More than the disk space of **ls-lR** is briefly used in **/tmp**. The command **ls -lR \>ls-lR** stores a long recursive directory listing in the file **ls-lR**. FTP servers make these files once a day so a directory listing is ready to download whenever a mirror logs in. For gigabyte servers these files are several megabytes and are usually compressed to **ls-lR.gz**. Many mirrors still download **ls-lR.gz** daily, compare it with their own directory and then request the new files they want. Many large archives now have **ls-lR.gz** about a megabyte and the average total size of the daily new files is similar. The bandwidth and (horror) charges for downloading the new files are increased by the size of the daily **ls-lR.gz**. **ls-lR.patch.gz** is a compressed unified difference of the previous and the current **ls-lR** file. Mirror scripts and programs can download this file and patch a local copy of the remote server's **ls-lR** daily. **ls-lR.patch.gz** is sometimes only 1% of the size of **ls-lR.gz.** **ls-lR.times** contains just the modification times of the previous and current **ls-lR** expressed as decimal seconds since the beginning of 1st January 1970 in the server's time zone. A mirror can download this 20 byte file first to check whether its local copy of the server's **ls-lR** can be patched with the **ls-lR.patch.gz** currently available or whether it needs to download a fresh **ls-lR.gz**. **mkls-lR** makes the above files with careful attention to preserving and recording names and times. Mirrors can check the names and times against their local copy of the server's **ls-lR** before applying **ls-lR.patch.gz** as well as checking whether the patch fits. This ensures the mirror's **ls-lR** is aligned with the server's. **mkls-lR** does not make new **ls-lR** files unless there is a difference in the directories further saving in unnecessary mirror downloads. Heavy server load increases **ls** runtime delaying **ls-lR** files until mirrors miss downloading them. **mkls-lR** can store new **ls-lR** files in **/tmp** for up to an hour before release at a precise time of day *tod* set with the **-t** option. The time **mkls-lR** starts should under normal load allow making of the final file **ls-lR.times** in the middle of the interval before *tod*. This prevents mirrors missing **ls-lR** files when server load is heavy even if mirroring starts only 1 minute after *tod*. A mirror using the **timezone** file with *tod* can set its start time precisely when appropriate. **mkls-lR** makes and places the files as follows. If the file **/etc/timezone** is readable **mkls-lR** copies it to **timezone** in *dirpath*. It then makes **ls-lR** and **ls-lR.patch.gz** in the **/tmp** directory but relative to *dirpath*. If **ls-lR** is made before time of day *tod* its modification time is set to *tod*. After compressing **ls-lR** to **ls-lR.gz**, **mkls-lR** sets the modification time of **ls-lR.gz** equal to **ls-lR**. By default **mkls-lR** then deletes **ls-lR** but for the **-a** option it moves **ls-lR** to *dirpath* instead. **mkls-lR** makes **ls-lR.times** last. If *tod* has passed or **at** is unavailable, **mkls-lR** moves **ls-lR.times**, **ls-lR.patch.gz** and **ls-lR.gz** into *dirpath* immediately but otherwise asks **at** to move them at *tod* precisely. **mkls-lR** is a **bash** script using *GNU* utilities intended for use on FTP servers with archives over 100 MB in size. ## EXAMPLES **mkls-lR** would normally be started by **cron** shortly before it is desired mirrors should log in. For these examples to work **cron** must have write permission to the *dirpath* subdirectory path argument. If you have more than one of these commands you should put them in a shell script and start the script with **cron**. - **50 20 \* \* \* /usr/sbin/mkls-lR -d /home/ftp/pub -t 21:00** At 8:50 pm local time start making the **ls-lR** files in **/tmp**. Move them to **/home/ftp/pub** where mirrors can download them at 9:00 pm precisely. - **mkls-lR -d /home/ftp/pub "-Ils-lR.\*"** Make the **ls-lR** files in **/home/ftp/pub** and do not include them in the listing. This stops any badly configured mirrors from downloading the **ls-lR** files twice. - **mkls-lR -d /home/ftp/tex-archive "-Ils-lR.\* -IFILES.by\*"** Include in the listing neither **ls-lR** files nor **FILES.bydate** nor **FILES.byname** nor **FILES.bysize**. A common fault of CTAN mirrors is downloading daily the 12 MB of index files mentioned. A mirror using these **ls-lR** files cannot make that mistake. - **mkls-lR -d /home/ftp/pub/debian -n ls-lR\_stable\\ "dists/slink/\*"** The *basename*, **ls-lR\_stable**, replaces **ls-lR** so the output filenames become **ls-lR\_stable.gz**, **ls-lR\_stable.patch.gz** and **ls-lR\_stable.times**. Only subdirectories matching **dists/slink/\*** and their contents are listed so a mirror using the above files will only download these. This example may be obsolete when you read this manpage. ## OPTIONS - **-a** Move **ls-lR** to directory *dirpath* immediately after compression. Default: Delete after compression. - **-d** *dirpath* Copy any **/etc/timezone** into directory *dirpath*. Make the files **ls-lR.gz**, **ls-lR.patch.gz** and **ls-lR.times** relative to directory *dirpath*. Move them into *dirpath*. Write permission to *dirpath* is necessary. - **-n** *basename* Substitute *basename* for **ls-lR** in each of the output file names **ls-lR** occurs in. Default: **ls-lR**. - **-t** *tod* Move **ls-lR.times**, **ls-lR.patch.gz**, and **ls-lR.gz** from **/tmp** to *dirpath* at a time of day *tod* not more than 1 hour after **mkls-lR** starts. The modification times of **ls-lR** and **ls-lR.gz** are set to *tod*. If **-a** requires **ls-lR**, it moves when created. *tod* must be a single word **date** understands. Usually *tod* is in the format **hh:mm**, for example **21:00**. An example for testing is **now+1min**. To release files at 5 minutes past midnight tomorrow use **23:65**. Default: **now**. - **-h, --help** Display help message then exit. - **"** expand special pattern characters **\***, **?**, **\[** and **\]** in *path* relative to directory *dirpath*. Default: Expand relative to current default directory. - **-I***path* *path* is a shell pattern of files or directories not to appear in **ls-lR**, **ls-lR.gz** or **ls-lR.patch.gz**. - *path* *path* is a shell pattern of files or directories to appear in **ls-lR**, **ls-lR.gz** or **ls-lR.patch.gz**. Default: All files in and below *dirpath.* - **-V, --version** Display version and copyright message then exit. ## SEE ALSO [**cron**(8)](/cron/8.md), [**crontab**(1)](/crontab/1.md), [**mirror**(1)](/mirror/1.md), [**ftp**(1)](/ftp/1.md), [**ftpd**(8)](/ftpd/8.md), [**ls**(1)](/ls/1.md), [**diff**(1)](/diff/1.md), [**gzip**(1)](/gzip/1.md), [**date**(1)](/date/1.md), [**at**(1)](/at/1.md), [**tzset**(3)](/tzset/3.md), [**wu-ftpd**(8)](/wu-ftpd/8.md). **README.ls-lR** describes how Lee McLoughlin's Perl script **mirror** can use **ls-lR.patch.gz** and **ls-lR.times**. ## AUTHOR Ian Maclaine-cross 19th August 1999