--- name: "pslogin" section: "1" platform: "linux" package: "mirrordir" version: "0.10.49-intl-4" release: "Debian 5.0 (Lenny)" platforms: ["linux"] url: "https://new.carta.tech/mirrordir" --- # pslogin(1) secure remote tcp login using strong stream cipher encryption and diffie-hellman key exchanges. forward-socket - arbitrary tcp socket forwarding over a secure/encrypted channel. copydir, mirrordir - copy, mirror directory trees via a minimal set of changes, locally or over FTP, or over a secure tcp connection. recursdir - recurse through local or remote directories to command/find files or create tar files. Part of [mirrordir(1)](/mirrordir/1.md). ## NAME pslogin - secure remote tcp login using strong stream cipher encryption and diffie-hellman key exchanges. forward-socket - arbitrary tcp socket forwarding over a secure/encrypted channel. copydir, mirrordir - copy, mirror directory trees via a minimal set of changes, locally or over FTP, or over a secure tcp connection. recursdir - recurse through local or remote directories to command/find files or create tar files. ## SYNOPSIS **mirrordir**\ \[**-a**, **--access-times**\]\ \[**-m**, **--strict-mtimes**\]\ \[**--no-mtimes**\]\ \[**--ignore-size**\]\ \[**-A**, **--mtime-threshold** *ext*\]\ \[**--time-offset** \[\[*+*\]\|*-*\]\[*H*\]*H*\[*:MM*\]\]\ \[**-A**, **--always-write**\]\ \[**-r**, **--restore-access**\]\ \[**--no-chown**\]\ \[**--no-chmod**\]\ \[**-D**, **--only-delete**\]\ \[**-b**, **-S**, **--backup-extension**, **--suffix** *ext*\]\ \[**-N**, **--num-backups** *num*\]\ \[**-O**, **--backup-outdate** *sec*\]\ \[**-B**, **--block-size** *bytes*\]\ \[**-M**, **--max-bytes** *num*\[**K**\|**M**\|**G**\]\]\ \[**-s**, **--starting-file** *path*\]\ \[**-i**, **--ignore-next-exclude**\]\ \[\[**-i**\] **-X**, **--exclude** *path*\] \[\[**-i**\] **-X**, **--exclude** *path*\] ...\ \[\[**-i**\] **-F**, **--exclude-from** *file*\] ...\ \[\[**-i**\] **-G**, **--exclude-glob** *expr*\] \[\[**-i**\] **-G**, **--exclude-glob** *expr*\] ...\ \[\[**-i**\] **-R**, **--exclude-regexp** *expr*\] \[\[**-i**\] **-R**, **--exclude-regexp** *expr*\] ...\ \[**-C**, **--exclude-script** \[*expr*\|*file*\]\] \[**-C**, **--exclude-script** \[*expr*\|*file*\]\] ...\ \[**-h**, **--help**\]\ \[**-v**, **--verbose**\] \[**-v**, **--verbose**\] ...\ \[**-V**, **--version**\]\ \[**-k**, **--keep-files**\]\ \[**-l**, **--no-hard-links**\]\ \[**--follow-symlinks**\]\ \[**-L**, **--strict-locking**\]\ \[**-p**, **--password** *password*\]\ \[**-P**, **--password-exact** *password*\]\ \[**--test-login**\]\ \[**--no-warn-first-login**\]\ \[**--read-password-from-stdin**\]\ \[**--allow-empty-ftp-dirs**\]\ \[**--no-allow-empty-ftp-dirs**\]\ \[**--netrc**\]\ \[**--no-netrc**\]\ \[**--proxy-host** *host*\]\ \[**--secure**\]\ \[**-z**, **--gzip**\]\ \[**--gzip-backups**\]\ \[**--case-insensitive**\]\ \[**--to-lower**\]\ \[**--to-upper**\]\ \[**--no-use-passive-connections**\]\ \[**-K**, **--key-size** *bits*\]\ \[**--download-scripts**\]\ \[**--tar-file** *filename*\]\ \[**--tar-block-size** *N*\]\ \[**-t**, **--dry-run**, **--test-only**\]\ \[**--nice** *num*\] *control mirror* **mirrordir** \[**-c** \| **--copy-mode** \| **--recurs-mode** \] -\[**abBCdDFGhklMmNOopRrstvVX**\] *src* \[*src* ...\] *dest* **copydir** -\[**abBCdeFGhklMmNOopRrstvVX**\] *src* \[*src* ...\] *dest* **recursdir** -\[**abBCdeFGhklMmNOopRrstvVX**\] *src* \[*src* ...\] **pslogin** \[**--key-size** *bits*\] \[**mc://**\]\[*username***@**\]*hostname*\[**:***portnumber*\]\[**/***path*\] But usually just\ **mirrordir** \[**--exclude** *path*\] ***control mirror***\ **copydir** *src* \[*src* ...\] *dest*\ **recursdir** *src* \[*src* ...\] \[**-C** *program*\]\ **pslogin** \[*username***@**\]*hostname*\[**:***portnumber*\] ## DESCRIPTION **mirrordir** is a set of useful utilities for manipulating and mirroring directories. Included is also the command **pslogin** - an alternative to [**ssh**(1)](/ssh/1.md), and [**forward-socket**(1)](/forward-socket/1.md) for forwarding arbitrary TCP socket connections over encrypted secure channels. **mirrordir** copies files that are different between the directories *control* and *mirror* to the directory *mirror*. Files whose modification times or sizes differ are copied. File permissions, ownerships, modification times, access times (only if --access-times is used), sticky bits, and device types are duplicated. Symlinks are duplicated without any translation. Symlink modification and access times (of the symlink itself, not the file it points to) are *not* preserved. Hard linked files are merely copied. Creation times cannot be set with Unix as far as I can see. **mirrordir** is a **DANGEROUS** command because files or directories that exist in *mirror* that don't exist in *control* are deleted. If *control* is entirely empty, then all files and directories in *mirror* will be deleted. If *mirror* is entirely empty, then all files and directories in *control* will be copied. In short, **mirrordir** forces *mirror* to be an exact replica of the directory tree *control* in every possible detail suitable for purposes of timed backup. It naturally descends into subdirectories to all their depths. **mirrordir** tries to be as efficient as possible by making the minimal set of changes necessary to mirror the directory. Access time duplication is not usually required and creates unnecessary load. Hence it is given as an option. The directory *control* is left untouched. If --restore-access is given then access times are reset to their original with each read. If the the --strict-locking option is on, files in *control* that are copied are locked for \`shared reading´. This will ensure, if another process is busy writing to that file, that the file is not copied in its incomplete or corrupted state. Usually **mirrordir** will not exit, but will give error messages to stderr to report any problems, and then will continue. The directory *mirror* or *dest* must exist, even if it is empty. Before erasing all the files in a directory, **mirrordir** checks for the file **\*--keep-me** (where **\*** is zero or one characters). If this file is present it will abort with an error message. Hence such a file can be created in all directories that you are fearful of being recursively erased. **copydir** is equivalent to **mirrordir** *-ck* *--no-erase-directories* ...\ (although -c implies -k anyway), so **copydir** is very much like a rigorous version of [**cp**(1)](/cp/1.md) where filenames can also be URLs, and only outdated files are replaced. Use **copydir** instead of **mirrordir** for most file transfers. Only use **mirrordir**, when you really want to delete things. **recursdir** is a further program that does nothing but descend into the directories on the command line. It is equivalent to **mirrordir** *--recurs-mode* ...\ It was born after the **-C** option was added, and can be used as a more rigorous version of [**find**(1)](/find/1.md) and can also pack all the files it finds into a tar file. **pslogin** is yet a further program which has almost nothing to do with the previous three. It envokes a secure login session using **secure-mcserv**. It is equivalent to **mirrordir** *--login-mode* *--secure* ...\ **pslogin** should be called **logindir**. See **--login-mode** below. **forward-socket** is yet a further program which has almost nothing to do with first three. It can do forwarding of arbitrary services over a secure channel. See [**forward-socket**(1)](/forward-socket/1.md) for details. The importance of this package is that you can use URL's instead of normal filenames, and hence manipulate files over a network. The URL types currently supported are **ftp://** and **mc://** (**http://** is not a filesystem and therefore is not supported). **mc://** is the **Midnight Commander** filesystem and is served by the **secure-mcserv** daemon. It has the advantage of serving cryptographically strong secure file transfers and logins. You can also use glob expressions in filenames for the **recursdir** and **copydir** commands. These will be recursively expanded. ## SECURITY AND ENCRYPTION **mirrordir** supports strong stream cipher encryption and Diffie-Hellman key exchanges with several possible key sizes. Secure connections work with **mc://** type connections. See the options **--secure**, **--key-size**, **--download-scripts**. See the **EXAMPLES** section for demo's and the **FILES** section for where public/private keys are stored. ## OPTIONS - **--help** Print out detailed help, then exit. - **--verbose** Specifies verbose output of file modifications made to *mirror*. This option can be given multiple times for greater verbosity. Output is written to stdout. - **--restore-access** Restore the access times of *control* with each read. - **--access-times** Duplicate even the access times of *control*. - **--always-write** Force rewrite of every file regardless of whether they appear to be identical or not. - **--recurs-mode** This is set by default with **recursdir**. Listed directories are read recursively and nothing is done to them. This option is useful with **-C** in order to execute shell commands and search for files. Note the **system()**, **exec()** and **popen()** functions available to the C interpretor. - **--login-mode** This option is set by default with **pslogin**. This approximates [**rlogin**(1)](/rlogin/1.md) using **secure-mcserv** as a server. It is a secure encrypted connection intended as a replacement to [**ssh**(1)](/ssh/1.md). **pslogin** implies **--secure** as well. When using this option or when using **pslogin**, only one path must be present on the command-line. The path is of the form \[**mc://**\]\[*username***@**\]*hostname*\[**:***portnumber*\]\[**/***path*\].\ Immediately after logging in, a **cd** */path* will be executed at the shell prompt. To determine if a prompt is available, **pslogin** searches for a **#**, **$** or **\>** character. Should these not be found, **pslogin** blocks indefinately. To avoid this behaviour, specify **/***path* as exactly **/**. This will leave you in your login directory. Alternatively, change your shell prompt to include one of these characters. - **--copy-mode** This is set by default with **copydir**. Approximate the behaviour of [**cp**(1)](/cp/1.md) as regards source and destination files. This option implies --keep-files. It overwrites any existing file or directory with the same name as a source file or directory, but does not delete files or directories unnecessarily. Multiple source files or directories may be given. The destination path must be a directory. - **--no-erase-directories** With this option, if a target directory conflicts with a source file, and the directory is not empty, then an error message will be printed and the program will abort. This is the default behaviour for **copydir**. - **--erase-directories** With this option, if a target directory conflicts with a source file, then the target directory will be recursively deleted. This is the default behaviour for **mirrordir**. - **--allow-empty-ftp-dirs** Some ftp servers do not produce a . or .. directory. This makes it appear as though you did not have permission to read the directory. This option overrides this by assuming that such completely empty directories are merely empty and do not have errors. If you get **unable to open directory: \*: Permission denied** errors, then you can use this option. This option is enabled by default (see next). - **--no-allow-empty-ftp-dirs** Because we now cd to the directory to check for permission to access it, the default behaviour is to allow empty directories. - **--only-delete** Do not make any changes to *mirror* that will cause the total space occupied by *mirror* to enlarge. This is a useful option when backing up onto a drive that has limited space, where changes have been made to *control* that might cause *mirror* to grow larger during the duration of the transfer. Running **mirrordir** once with this option and then once normally will ensure that the available space is not overrun. - **-i**, **--ignore-next-exclude** This option dictates that the next --exclude- type option must completely overlook those files regardless of their existence or non-existence in the mirror directory. It has the effect of a return value of **IGNORE** for the **--exclude-script** option below. This can be used to cause certain files to never be modified, for example if you would like */etc/named.boot* to never be modified, use **mirrordir /mnt/1 /mnt/2 -i --exclude** **/mnt/1/etc/named.boot**. Note that all paths must be specified in the control directory and not the mirror directory, hence **--exclude /mnt/2/etc/named.boot** won't work. This has the idiotic behaviour that if you want to avoid erasing a file, you have to have that file present in the control directory, even if it has zero length. - **--exclude** *path* Exclude file or directory *path*. A large number of excluded paths on the command-line will slow performance. An excluded path will be removed from the mirror tree if it already exists, in the same way as any absent directory or file. Use the --ignore option to ignore keep rather than delete these files. If you have a long list of files to exclude, use the **--exclude-from** option. - **--exclude-glob** *glob* Exclude file or directory names matching glob style expression *glob*. Matches the file without its full path. For shells, the expression should be enclosed in appropriate quotes to prevent substitutions. - **--exclude-regexp** *regex* Exclude full pathnames of files or directories matching *regex*. For shells, the expression should be enclosed in appropriate quotes to prevent substitutions. - **--exclude-script** \[*expr*\|*file*\] Cause execution of the script *expr* for each file before doing anything with that file. The script is a C style statement block, terminating with a \`**return** *expression*;´. The value of *expression* can be one of **INCLUDE**, **EXCLUDE**, **UNKNOWN** or **IGNORE** (see **-i** above) to explain what is to be done with that file. If *expr* does not contain a semi-colon (;) then it is assumed to be the name of a file - which is then loaded. In either case, the text is byte compiled into reverse-polish notation for fast execution. This option can be specified multiple times and the scripts will be executed in order until a script returns something other than **UNKNOWN**. A return value of **UNKNOWN** allows the remaining **--exclude-** options on the command-line to take effect. If you find that the interpretor incorrectly reports errors or segfaults, please report the breaking script to me. The scripting language itself is a subset of the full C programming language. For example, the following is a valid script: ``` /* PATH is the full name of the file including its path, DIR is the directory, excluding the trailing slash (/), CWD is the current directory, and depth() returns the number of forward slashes (/) less one. */ if (depth (DIR) - depth (CWD) > 3) { printf ("%s: excluded\n", PATH); return EXCLUDE; } else return INCLUDE; ``` The scripting language does not support the assignment operator, and hence does not support user defined variables. The following predefined macros are available. Note that the expansion of the macros applies equally well to directories as to files. - **FILE** current file without its path - **NAME** file name without its path or extension or trailing dot (.) - **EXTENSION** file extension without its leading dot (.) - **DIR** directory without file-name or trailing slash (/) - **PATH** full file name with path - **CWD** current working directory - **TIME** current time in seconds All of the logical, arithmetic and bitwise C operators are supported. These are **(** **)** **\>=** **\<=** **\>** **\<** **!=** **==** **&&** **\|\|** **!** **-** **+** **\*** **/** **%** **&** **^** and have the same meanings and precedences as in C. The following further predefined macros are available. Each returns an integer (type **long int** in C). These are based on a C **lstat** (or **stat** if **--follow-symlinks** is used) on the file. See [**stat**(2)](/stat/2.md) for a detailed explanation. **stat.st\_dev** - device\ **stat.st\_ino** - inode\ **stat.st\_mode** - permissions\ **stat.st\_nlink** - number of hard links\ **stat.st\_uid** - user id of owner\ **stat.st\_gid** - group id of owner\ **stat.st\_rdev** - device type\ **stat.st\_size** - file size in bytes\ **stat.st\_blksize** - block-size for file-system I/O\ **stat.st\_blocks** - number of blocks allocate\ **stat.st\_atime** - time of last accessed in seconds\ **stat.st\_mtime** - time of last modification in seconds\ **stat.st\_ctime** - time of creation\ The following functions return boolean values: - **strncmp**(*string1*, *string2*, *integer*); returns an integer less than, equal to, or greater than zero if string1 is found, respectively, to be less than, to match, or be greater than string2. - **glob**(*glob*, *string*); returns zero if *string* matches glob expression *glob*. Try to use only one glob expression in your code for efficiency of the underlying implementation. - **regexp**(*regexp*, *string*); returns zero if *string* matches regular expression *regexp*. Try to use only one regular expression in your code for efficiency of the underlying implementation. - **strstr**(*string1*, *string2*); returns the first occurance of *string2* in *string1* up to the length of *string1*, or zero if it did not occur. The following functions also return a boolean value and are analogous to the corresponding macros explained in [**stat**(2)](/stat/2.md). They return non-zero if the specified condition is true. **S\_ISLNK**(*integer*); - file is a sym-link\ **S\_ISREG**(*integer*); - file is a regular file\ **S\_ISDIR**(*integer*); - file is a directory\ **S\_ISCHR**(*integer*); - file is a character device\ **S\_ISBLK**(*integer*); - file is a block device\ **S\_ISFIFO**(*integer*); - file is an fifo\ **S\_ISSOCK**(*integer*); - file is a socket The following functions manipulate strings: - **strcat**(*string1*, *string2*); returns the concatenation of *string1* with *string2*. Note that the **+** operator also concatenates strings. - **depth**(*string*); returns one less than the number of forward slashes (/) in *string*. - **printf**(*format*, ...); behaves like [**printf**(3)](/printf/3.md) with an important exception: only **long int** format specifiers should be used. The behaviour of anything that results in conversion of other than a **long** **int** is undefined. For example, use **"%ld"** instead of **"%d"**. This function prints to stdout. The following functions do system calls: - **system**(*command*); executes **/bin/sh -c** *command*, but unlike the C version, it returns the exit code of the command. I.e. it executes a single line of shell script, *command*. - **exec**(*argv0*, *argv1*, ...); executes process *argv0* with arguments *argv1...*. *argv0* must be a full path name. This is faster than **system** because it need not envoke **sh**. - **popen**(\[*string*, \] *shell\_command*); like system, but returns the output of *shell\_command* as a string. If *string* is given, this writes string into the standard input of *shell\_command* and return zero on success. The following further integer constants are available and are analogous to the macros defined in stat.h and explained in [**stat**(2)](/stat/2.md). **S\_IFMT S\_IFSOCK S\_IFLNK S\_IFREG S\_IFBLK S\_IFDIR S\_IFCHR S\_IFIFO** **S\_ISUID S\_ISGID S\_ISVTX S\_IRWXU S\_IRUSR S\_IWUSR S\_IXUSR S\_IRWXG** **S\_IRGRP S\_IWGRP S\_IXGRP S\_IRWXO S\_IROTH S\_IWOTH S\_IXOTH** One of the the following constants should be returned using the **return** keyword, and imply to the caller as follows. If nothing is returned, the return value is assumed to be **UNKNOWN**. - **UNKNOWN** didn't know what to do, continue with other **--exclude-** options - **INCLUDE** include the file - **IGNORE** do nothing with the file regardless of its existence or non-existence in the mirror directory - **EXCLUDE** consider the file to be non-existent in the control directory and hence must be removed from the mirror directory (this does not override the option **--keep-files**) The following perform flow control analogous to C: The **if** clause causes *statement1*, *statement2*, etc. to be executed if *integer* is true (i.e. non-zero), or otherwise causes *statement1*, *statement2*, etc. to be executed. The **else** {...} part is optional. ``` if (integer) { statement1; statement2; . . . } else { statementA; statementB; . . . } ``` The **return** clause gives a value back to **mirrordir** and causes the script to exit: ``` return expression; ``` The **exit** function cause **mirrordir** to exit with the specified exit code. ``` exit(integer); ``` C scripts would typically be used to exclude types of files. Note that this is an excessive implementation of a scripting language, and all the features are not meant for general use. A typical script will, for example, do nothing more than cause device files to be excluded: ``` if (S_ISSOCK(stat.st_mode) || S_ISFIFO(stat.st_mode) || S_ISBLK(stat.st_mode) || S_ISCHR(stat.st_mode)) { return EXCLUDE; } else return INCLUDE; ``` C scripts can also be used to search for files with the **--recurs-mode** option (same as the **recursdir** command): ``` /* removes all core files */ /* This example has been moved to the EXAMPLES section. */ ``` - **--exclude-from** *file* Exclude from a list of paths listed in the file *file*. Empty lines and comment lines (with a # as the first character of the line) are ignored. This list of files is sorted and binary searched, so if you have lots of filenames to exclude, it is best to include them here for performance. This option can be specified multiple times with different files. **BUG**: the last path in *file* must end with a newline. - **--backup-extension** *level* Create backups of files before deleting or replacing them. *extension* is a C style format string e.g. .ORIG.%d (be careful of shell substitutions with the %). *level* is the highest number of revision to keep. *extension* is appended to the filename, the oldest file having the highest number. - **--backup-outdate** *sec* **Delete backup files older than** *sec* **seconds.** **This option is buggy therefore disabled in the Debian package.** - **--nice** *num* **Be nice to other processes by sleeping occasionally.** *num* **is a** **small integer. --nice causes the process to sleep for as long as it** **is active, times a factor of** *num***. Hence a value of 1 will (very** **roughly) double the time it takes to do a copy, and a value of 3 will** **quadruple the time. This can be used where one would like timed** **backups to place less load on the CPU. --nice may not be available** **on your system.** - **--no-chmod** **Normally the permissions of files are set. If you have restricted access** **and cannot change permissions, than this can be used to disable setting** **of permissions.** - **--no-chown** **Normally the ownerships of files are set. If you have restricted access** **and cannot change ownerships, than this can be used to disable setting** **of ownerships.** - **--mtime-threshold** *sec* **This is the deviation in mtime that is allowed for a file before it is** **overwritten. If you have mirrored an ftp site, the mtimes are accurate** **to within a minute only, thereafter an nfs mirror will cause every file** **to be copied: you can then use --mtime-threshold 60 to fix this.** - **--time-offset \[\[***+***\]\|***-***\]\[***H***\]***H***\[***:MM***\]** **This sets the time offset of any vfs (i.e. non-local) directory. For** **instance, I am 8 hours east of New York, so when I mirror from New York,** **I use --time-offset -08:00.** - **--test-only, --dry-run** **Do not make any changes. If used with --verbose, this will show what** **changes would be made. This is an effective way of comparing directory** **trees.** *This is untested - i.e. I don't know whether this option* **actually** *makes any changes or not!* - **--skip-symlinks** **Symlinks are treated as though they were not read - hence if they are** **found in the mirror directory then they are deleted.** - **--keep-files** **Don't remove files from** *mirror***, even if they don't exist in** *control***. This makes mirrordir somewhat like [cp(1)](/cp/1.md).** - **--no-hard-links** **mirrordir mirrors hard-links properly unless this option is set, in** **which case hard-links are copied as regular files.** - **--follow-symlinks** **mirrordir mirrors symlinks properly unless this option is set, in** **which case symlinks are copied as regular files. Useful for mirroring** **the Debian tree. Note that this follows symlinks in both the control and** **mirror directories, hence if symlinks already exist in the mirror** **directory, these will be left as symlinks. Note that** **--follow-symlinks implies --no-hard-links also.** - **--strict-locking** **Create shared read locks on files as they are read. This prevents** **clashes especially when copying files from the mail directory - mail** **programs will be trying to write to those files at the same time as** **mirrordir is trying to to read from them. This option has** **no effect with virtual file systems.** - **--max-bytes** \[\[***num***\[**k**\|**M**\|**G**\]\]\|***num***\] **Maximum number of bytes to write before giving the message, \`filled up** **all blocks - first file/dir not mirrored:** *path***', which is written** **to stdout. The remaining files are deleted from** *mirror***, but in** **their listed order - hence it is possible that the archive will grow** **greater than** *num* **while mirrordir is still running. You** **should make allowance for this eventuality by making** *num* **smaller** **than the maximum available space. Also note that some filesystems will** **give an error message \`No space left on device' before the device is** **completely full. You can restart mirrordir with the option** **--starting-file** *path* **and hence continue with this file on** **another volume. This enables mirrordir to back up across different** **devices.** *num* **can be appended by k, M or** **G (case insensitive) to specify kilobytes, megabytes or gigabytes** **respectively. If any single file is greater than this number, then an** **error message will be given. See also --block-size.** - **--password** *password* **Specify the password for FTP and mc:// connections. The anonymous password** **defaults to your login name @ your local machine name. You will be** **prompted for other login passwords. As usual, you are warned that** **including passwords in scripts is a security risk. It is much better** **to put the password in your** **~/.netrc** **file and then not use the --no-netrc option; see** **man ftp** **for details.** - **--password-exact** *password***\]** **Don't prepend a - to the anonymous password. With ftp** **anonymous passwords, a - is usually prepended to the** **password string. I don't know why the Midnight Commander vfs did** **this, but one user had problems with it, hence this option is** **given to send the password exactly as specified by** *password***.** - **--test-login** **When using --login-mode or pslogin, you may want to** **test access non-interactively (eg. in a shell script). To do** **this you can run pslogin with this option and then check** **its exit status. secure-mcserv uses this to verify if the** **user can login on the password server.** - **--no-warn-first-login** **The first time you try a secure connection to a machine, no** **public key exists on the local machine. There is hence no** **security against a man-in-the-middle attack. A warning to this** **effect is printed and the user is prompted if they want to** **continue. This option disables this warning and goes ahead** **regardless.** - **--read-password-from-stdin** **Instead of specifying the password on the command line, you can** **write it into the command via stdin. This is not the same as** **typing the invisible password as one normally does, because it** **can be used even if there is no terminal. This is useful** **for usage within other programs using, say [popen(3)](/popen/3.md).** **secure-mcserv uses this to verify if the user can login** **on the password server.** - **--netrc** **Scan the ~/.netrc file. By default, this option is on.** - **--no-netrc** **Turn off reading of ~/.netrc file.** - **--proxy-host** *host* **Set the proxy for ftp downloads. Don't know how or if this works. Consult the** **[mc(1)](/mc/1.md) man page for info in proxy support.** - **--secure** **(***This feature is BETA***) I have implemented a secure socket layer** **for mirrordir. It is enabled with this option and applies to** **connections to secure-mcserv (i.e. using mc:// type URLs).** **The secure socket library consists of libdiffie.a and a** **header file diffie-socket.h. If you include diffie-socket.h** **after you include sys/socket.h you can recompile the resulting** **program with all normal sockets turned into secure sockets. (This** **applies to** *any* **program that uses Unix socket calls, but is** **untested). The first service supporting this is secure-mcserv** **which compiles and installs by default. Hence you can use the** **mc:// file system with the --secure option, provided the** **remote host is running secure-mcserv (try secure-mcserv -h** **to for help). --secure uses a stream cipher (much more secure and** **faster than block ciphers like DES) with discrete logarithm key** **exchanges with public key server authentication (Diffie-Hellman and** **p-NEW schemes). For the full details, see the file diffie-socket.h** **in the source distribution. The default key size is 512 bits. Note that** **if you are using** *gcc* **you should compile mirrordir with the** **options -O3 -fomit-frame-pointer -s -Wall to speed key generation.** - **--key-size** *bits* **The default key size is 512 bits. Sizes require primes to be** **generated and compiled into the file field.c, hence only those** **primes listed in field.c are supported, which are at the moment** **512, 768, 1024 and 1536. 768 is the recommended value for medium** **security if you have slow computer. Otherwise 1536 is not an** **unreasonably paranoid size for long term security. 512 can be used** **if you are not worried about large corporations, well connected** **hackers, or governments snooping your connections. Note that the** **the stream cipher has a length of bits/2, which means that** **you have more chance of being hit by a meteor while cashing in at** **the state lottery, than cracking it. Be aware that a person can sniff** **your connection and then keep the output for as long as it takes to** **break it. In twenty years time a 1536 bit key will be considered** **small. Note also that the discrete logarithm problem (used here)** **is considered to be more difficult to solve than the factorisation** **problem, hence the key is effectively a little larger than for RSA.** **This is all my (rather uneducated) opinion.** - **--download-scripts** **Mirrordir comes in two versions, an** *International* **version and** **a** *US* **version (see --version). The** *US* **version contains** **no encryption code whatsoever. Instead it downloads the needed** **algorithms from encrypt.obsidian.co.za (in South Africa).** **These are written in a fast, native, C-style interpreted language. There** **are four scripts: one for the server Diffie-Hellman key exchange, one** **for the client Diffie-Hellman key exchange, one for initialising the** **stream cipher and one for actually performing the encryption using the** **stream cipher. Mirrordir automatically downloads these scripts** **when you try to use the security features. The option** **--download-scripts however can be used to force a download at any** **time. The** *International* **version contains compiled-in versions of** **the stream cipher, hence only two scripts are used, which need never be** **downloaded since they come with the distribution. No speed penalty is** **incurred by having the Diffie-Hellman exchanges in scripts, however, you** *will* **notice a significant speed different when using the** **encryption scripts compared to compiled-in encryption.** - **--version** **Prints out the version number as well as whether this is an** *International* **or** *US* **version of mirrordir. See** **--download-scripts.** - **-z, --gzip** **Enable compression for mc:// connections. This actually envokes a** **compressed socket layer at a lower level than the encryption.** **Compression uses the libz library of [gzip(1)](/gzip/1.md). The degree of** **compression is dynamically set to minimise transfer time. It will drop** **to no compression for fast ethernet connections, and will go to maximum** **compression for slow modem connections. The algorithm adjusts the** **compression level so that the time for a TCP write call is between 2%** **and 5% of the time for the same amount of data to be** *deflated* **(i.e. compressed).** - **--gzip-backups** **Backups are usually just a copy of the file. With this option they are** **compressed and a .gz is added to the default extension. Note that** **if you specify your own extension using --backup-extension, then** **it must have a .gz at the end for compression to work.** - **--case-insensitive, --for-Robert-Seese** **Ignores case in comparison of filenames and linknames. This option is** **useful when interacting with certain brain-dead operating systems. I am** **not sure if this option behaves correctly under all circumstances.** - **--to-lower** - **--to-upper** **Convert all new filenames to upper or lower case. When used with** **--case-insensitive, it will apply only when creating new files.** **When used without --case-insensitive, all files, existing or not,** **will be converted to uppercase in the inefficient way of deleting the** **old file and then copying the new file. This option is useful when** **interacting with certain brain-dead operating systems. I am not sure** **if this options behave correctly under all circumstances.** - **--no-use-passive-connections** **If you get an error message could not setup passive mode it is likely** **because you need to enable this option. I don't really understand what \`passive'** **means, so don't ask me.** - **--tar-file** *filename* **This is to be used only with recursdir. It creates a tar archive in** **the same format as GNU [tar(1)](/tar/1.md) and stores it in** *filename***.** **Leading special prefixes and leading slashes are removed, i.e.** *ftp://machine/dir/file* **becomes** *dir/file***. If the first** **character of the filename is a \| character, the remaining text is** **considered as a command through which the output is to be piped. Hence** **a gzipped archive can be created for example with**\ **recursdir ftp://machine/dir --tar-file '\| gzip \> foo.tar.gz'** - **--tar-block-size** *N* **Sets the block size for tar output to 512 \*** *N***. These are the units in** **which data is written to the archive. The default is 20. This is** **significant only when writing to block devices. This must not be** **confused with --block-size.** - **--block-size** *bytes* **The default block size is 1024 bytes. File sizes are rounded up to the** **nearest block when calculating the total number of blocks consumed. If** **the actual block size is greater than assumed then it is possible for** **fewer blocks to be counted than are actually consumed during writing.** **Hence it is essential to specify a block size greater than or equal to** **the actual block size when using the option --max-bytes.** - **--strict-mtimes** **When copying regular files, mirrordir normally only overwrites the** **mirror file if it is** *older* **than the control file. This option forces a** **copy if there is** *any* **discrepancy at all in the modified times of** **the files.** - **--no-mtimes** **Copy files only if their sizes differ. Ignore the file modification** **time.** - **--ignore-size** **Copy files depending on mtime, but ignore size differences.** - **--starting-file** *path* *path* **can be a file or directory. Until** *path* **is read, files** **or directories will be processed as though they were excluded files** **(i.e. if they exist in** *mirror* **they will be deleted). Directories** **that contain** *path* **will be created if they don't exist. After** *path* **is read, files are mirrored as usual.** *path* **itself will** **also be mirrored. If** *path* **or any of its subdirectories are not** **present, then mirrordir exits immediately. This is the only time** **mirrordir exits prematurely. This is to prevent the entire** **filesystem from being erased on account of** *path* **never being found.** ## FTP SUPPORT **Ftp transfers are supported using the Midnight Commander's Virtual** **File System (VFS), see [mc(1)](/mc/1.md). In short, this means that full** **URLs as well as local directories are supported. The following example** **demonstrates:** ``` mirrordir --verbose \ ftp://lava.obsidian.co.za/pub/mirrordir \ /home/mirrordir ``` **or alternatively,** ``` mirrordir --verbose /home/mirrordir \ ftp://psheer@lava.obsidian.co.za/home/ftp/pub/mirrordir ``` **also works, but will prompt me for a password first. If you are** *uploading* **to an ftp server, you should not have the** **--strict-mtimes options on, because modification times cannot** **be set via ftp, and hence everything will be copied.** **You will notice (with --verbose) that mirrordir repeatedly tries** **to set modification and access times over ftp. I am going to leave these** **messages in to remind users of the limitations of the VFS type in use.** **These attempts do not appreciably detract from performance. You can use** **mc:// instead although performance is poor when uploading with** **this protocol. Downloading is always preferable to uploading.** **In general you should NOT have an ftp upload in your cron jobs** **or use ftp upload to keep directories in sync. Always download from the** **other end when trying to keep directories in sync. Uploading is good** **for once off uploads only.** ## EXAMPLES **Here are some nifty things you can do with mirrordir.** - **Pedantic minimalist copy** **I have two source trees. I like to keep an old copy before I apply** **a patch. I just do mkdir tree.OLD and then** ``` mirrordir -v tree tree.OLD ``` **If I run mirrordir again, then only a minimal change is** **made, i.e. only the updated files are copied. ([cp(1)](/cp/1.md) can in** **fact do this).** - **System backups** **Some systems used timed backups onto tape archives. Others still use RAID** **devices which constantly maintain an identical copy of a partition.** **mirrordir provides a further alternative. You can install two** **drives in a system - one for use and one for backups - and place** **mirrordir in your [cron(8)](/cron/8.md) tables. The various backup options** **can be set to make backups of files that have changed. The backup** **directory might also be readable by users so that they can see their** **backed up files. Previous versions of files would also be available for** **users that might like to retrieve any older instances of the file.** **Because mirrordir executes only the absolutely minimal set of** **changes, it is extremely fast. It can be run several times a day or even** **continuously with the --nice option.** **Backups can also be made by FTP onto a remote machine for additional** **security against the destruction of the machine.** - **Backing up hourly between two machines:** **On one machine called dar2 I have a cron job that runs every six hours:** ``` #!/bin/sh # (this is just in case of any bugs I don't know about, # but I don't think it is necessary) killall -9 tee killall mirrordir >& /dev/null sleep 2 killall -9 mirrordir >& /dev/null ( \ date ; \ echo "mirrordir says (if it said nothing it is bad):" ; \ mirrordir mc://dar1:12346/ -p abcdefg /mnt/dar1/ \ -i --exclude-regexp '^mc://dar1:12346/var/lock/subsys/atd' \ --exclude-regexp '^mc://dar1:12346/proc/' \ --exclude-regexp '^mc://dar1:12346/mnt/[^/]*/.*$' \ -i --exclude-regexp '^mc://dar1:12346/boot/' \ -i --exclude-regexp '^mc://dar1:12346/etc/lilo.conf' \ -C \ ´ if (S_ISDIR (stat.st_mode)) { if (!regexp ("^mc://dar1:12346/[^/]*$", PATH)) printf ("Backing up: %s0, PATH); } ´ ; \ date ; \ echo "Done" ; \ ) 2>&1 \ | tee --ignore-interrupts --append /var/log/mirrordir.log \ | mail -s 'dar1 backup results' psheer@obsidian.co.za ``` - **Secure transfers and logins** **On the machine turing.co.uk execute**\ ``` secure-mcserv -p 12345 -d ``` **On some foreign machine**\ ``` copydir --secure -K 512 -z \ mc://alan@turing.co.uk:12345/usr/src/linux/.config . ``` **to securely copy a file using a security key of 512 bits and using** **compression, or**\ ``` pslogin mc://alan@turing.co.uk:12345/ ``` **to login to the machine securely.**\ - **Mirroring FTP sites** **Ftp sites often disable their -R option on ls so that a traditional** **[mirror(1)](/mirror/1.md) will fail (?). mirrordir does not suffer from this** **limitation.** ``` mirrordir -v ftp://metalab.unc.edu/pub /home/ftp/pub ``` - **FTP transfers** **Typical FTP transfers can be made easily with a single command using the** **--copy-mode option. You can copy multiple files in both directions and** **even between two ftp sites (albeit indirectly), just like with** **[cp(1)](/cp/1.md). I use**\ ``` copydir -v mirrordir-0.9.15.tar.gz \ mirrordir.lsm ftp://metalab.unc.edu/incoming/Linux ``` **to upload mirrordir to the sunsite.** **Passwords for non-anonymous ftp transfers are best placed in the** **~/.netrc** **file according to standard ftp convention, and the option --netrc used.** **Alternatively use ftp://myname@machine/ instead.** - **Finding files** **recursdir / -C** \ **'if (!glob ("\*.c", FILE)) printf ("%s\\\\n", PATH);'**\ **will print out all the C files on your system.**\ **recursdir / -C** \ **'if (S\_ISCHR(stat.st\_mode)) printf ("%s\\\\n", PATH);'**\ **will print out all character devices on your system.** - **Backup FTP sites onto tape** **Use**\ ``` recursdir ftp://user@remote.machine/ \ --exclude-regexp '//[^/]*/proc/' --tar-file /dev/mt ``` **to back-up a remote site onto tape.**\ - **Removing core files** **This removes all core files from your system:** ``` recursdir / -C ´ long l; if (strncmp (PATH, "/proc", 5)) { if (S_ISREG (stat.st_mode) && !strcmp ("core", FILE)) { if (strstr (popen ("file " + PATH), "ELF 32-bit LSB core")) { l = l + stat.st_size; printf ("removing: %s, cumu. total = %ldkB\n", PATH, l >> 10); exec ("rm", "-f", PATH); /* could also use system() */ } } } ´ ``` ## ENVIRONMENT VARIABLES - **TMPDIR** **The directory where you would like temporary files to be stored. The ftp** **filesystem downloads files first into the this temporary directory, and** **then copies the file into its correct place. See BUGS below.** **If TMPDIR is not specified, then it defaults to the directory of the** **current file in progress.** ## RETURN VALUE **mirrordir returns the following:** - **0** **Success.** - **1** **Some kind of error occurred like a write error, a permissions error or** **the like. In this case, the precise error would have been written to** **stderr.** - **2** **A file was in use and therefore could not be copied, but otherwise a** **success. In this case the error \`unable to open control file for writing'** **would have been written to stderr. If you grep for these in stderr, you** **can then run mirrordir with just these files. See [grep(1)](/grep/1.md).** ## BUGS **Some versions of the atd daemon create a lock (or pid?) file with** **a lock on the file that cause secure-mcserv to block indefinately.** **You will have to kill atd or exclude (?) this file before hand to** **prevent this.** **Mirroring over ftp where there is some daylight savings correction, seems** **to produce a one hour time offset. Use --time-offset as a temporary** **work-around. I don't know if this is mirrordir's fault though.** **A bug that caused mirrordir to eat CPU and appear to halt has been** **fixed.** **A large number of command-line --exclude expressions will slow** **performance. This is noted for the purposes of those wanting to** **exclude a large list of files. This list should be placed in a** **text file and excluded with the --exclude-from option.** **Modification and access times of symlinks (not the files they point to)** **are not duplicated.** **Note that the previous limitation that hardlinked files are treated as** **regular files has now been alleviated. The option --no-hard-links is** **provided to emulate the behaviour of versions prior to 0.9.8.** **No check is made to see if a hard-link would be created across devices,** **in which case an appropriate error will be reported.** **The ftp file-system downloads files first into the temporary directory,** **as a file with the name \*ftpfs\*. This is a waste of space but is** **the default behavior of the vfs library. If there is not enough space in** **this directory, then mirrordir will probably hang. See** **ENVIRONMENT VARIABLES above.** **The C scripting language used to exclude files is an excessive** **implementation.** ## FILES - **~/.netrc** **List of machines and corresponding passwords. See the option --netrc.** - **/etc/ssocket/accept.cs** **This script performs the key exchange and signature generation on the** **server side of the connection.** - **/etc/ssocket/connect.cs** **This script performs the key exchange and signature authentication on the** **client side of the connection.** - **/etc/ssocket/arcinit.cs** **This initialises the stream cipher encryption. (Not present in International versions.)** - **/etc/ssocket/arcencrypt.cs** **This performs the stream cipher encryption. (Not present in International versions.)** - **/etc/ssocket/private/** **This directory contains private keys for the host. Each key will be held** **in a file: filenames are 512, 1024 etc. This is a preferable alternative to** **databases of keys as it obviates the need for a key management utility. Soon** **the Reiser (spelling) filesystem will be standard and database files will** **become obsolete anyway.** - **/etc/ssocket/public/** **Analogous to /etc/ssocket/private/ for public keys.** ## STANDARDS **mirrordir is an invention of its author and does not belong to any** **operating system standard (although it should!).** ## AVAILABILITY **The latest version of the program can be found at either** **ftp://metalab.unc.edu/pub/Linux/system/backup, or** **ftp://lava.obsidian.co.za/pub/linux/mirrordir.** ## AUTHOR **Paul Sheer \ \** ## SEE ALSO **[mirror(1)](/mirror/1.md), [pavuk(1)](/pavuk/1.md), [cp(1)](/cp/1.md), [scp(1)](/scp/1.md),** **[find(1)](/find/1.md), [mc(1)](/mc/1.md), [ftp(1)](/ftp/1.md), [ssh(1)](/ssh/1.md),** **[tar(1)](/tar/1.md), [rlogin(1)](/rlogin/1.md), [rlogind(8)](/rlogind/8.md), forward-socket**