2UTF(1)

NAME

2UTF, fromUTF - translates char-sets to/from unicode(7), decodes MIME messages

SYNOPSIS

[2UTF|fromUTF] [options] [charmap_file_or_alias] <input >output

DESCRIPTION

2UTF is a filter which converts char-sets to unicode(7) (UCS - Universal Character Set) and reverse if possible. It can also display char-maps, console font and a range of unicode(7) glyphs in utf-8(7) encoding. It gets char-map for single-byte char-sets from locales or wg15-locale package database, from tables found at ftp://ftp.unicode.org/ or other similar files with user defined format. It can invoke external filters specified in configuration file.

charmap_file_or_alias is pathname, alias or filename for the file with char-map definition. Alias or filename is converted to uppercase. Aliases specified in locales char-maps are cached in special file if directory/file permissions allow this. If exact match for alias or filename isn't found, *alias* or *filename* glob pattern is used (except for aliases found in configuration file).

- (hyphen-minus) and _ (low line, spacing underscore) characters in aliases are ignored.

When invoked without char-map specified a mail message is assumed on standard input. I do not provide any warranty and recommend you to backup your mail if you use 2UTF as automatic mail filter. See procmailrc file in examples subdirectory. It should handle common text and multi-part types, MIME style encrypted and not 8-bit headers. Everything is converted to utf-8(7) encoding. Messages with MIME style pgp(1) signatures should be passed untouched.

OPTIONS

--: stops option checking for the rest of the command line.

-2 --UCS-2 --ucs-2: output (input if --reverse specified) 2 byte wide characters.

-4 --UCS-4 --ucs-4: output (input if --reverse specified) 4 byte wide characters.

-8 --UTF-8 --utf-8: (default) output (input if --reverse specified) multi-byte utf-8(7) characters.

-cFILENAME --charmap-file=FILENAME: the alternative way to specify filename or pathname for char-map file.

-C --create-aliases: rescans all available char-map files and (re)creates aliases file. You should have write permission for this file.

-d[N] --debug[=N]: Outputs debugging info to stderr. Implies --verbose. N is debug level from 1 to 9. Default is 1.

-f[FORMAT] --format[=FORMAT]: sscanf(3) format string for reading char-map file. If not specified the default format as output by 2UTF -h (used by char-maps from WG15 locales) is assumed. Aliases specified in locale char-map files are recognized. Lines beginning with % or # and lines not matching format are ignored. In case of duplicated lines the last line takes precedence. " 0x%x 0x%X " sscanf(3) format string is always assumed for char-map files ending in .TXT, .X or .x. This corresponds to char-maps you can get from ftp://ftp.unicode.org.

-o --forward: (default) converts *to* unicode(7) if invoked as 2UTF, and tries convert *from* unicode(7) if invoked as fromUTF.

-H --HTML --html: This aplies to approximations when converting from unicode(7). Special HTML characters appearing after approximations are changed to < > " and &.

-h -? --? --help -help: Prints the program's version number, default parameters, and a short usage message to the program's standard error output and exits.

-l --list-charmaps: Lists char-maps and aliases currently in aliases database, then exits. This includes only char-maps usable by 2UTF.

-p --pathnames: Prints pathnames for configuration file, default compiled-in directories for char-map files, actually used directories for char-map files, pathname for aliases cache.

-r --reverse: tries convert back *from* unicode(7) if invoked as 2UTF, and converts *to* unicode(7) if invoked as fromUTF.

-W --show-charmap: outputs table of char-map characters in utf-8(7) encoding. . (period) is substituted for 0x0000-0x001F and 0x007F. ? (question mark) is substituted for undefined characters.

-S --spit-glyphs: outputs table of characters in utf-8(7) encoding at F000-F1FF unicode(7) private use area. This corresponds to current console font in Linux.

-S[min][-][max] --spit-glyphs=[min][-][max]: outputs table of characters in utf-8(7) encoding at given range. min and max is unicode(7) hex numbers from 0 to 7FFFFFFF. min defaults to 0. max defaults to min + 511 if - is present.

-s --switch-to-UTF-8: tries to switch to utf-8(7) mode by writing <ESC>%G to the program's standard error output. Use echo -ne '\ 33%@' to switch back if required. This is Linux console escape sequences.

-u[X] --unknown-char[=X]: substitute X for unknown single byte characters and errors. If X isn't specified the default character as output by 2UTF -h is assumed. X can be a single character, hex (0x80), octal (0200) or decimal (128) number. This can be usefull when translating to single-byte encoding.

-v --verbose: verbose mode.

-V --version: shows program's version and some copyright information.

Rightmost option or alias takes precedence. Long options may be abbreviated. Short options may be grouped.

Default unicode(7) character for errors and unknown characters is 0xFFFD. Approximations can be performed if conversion is from Unicode to single-byte char-sets. US-ASCII strings up to 4 bytes length is substituted for characters undefined in the output charset. These strings are defined at the compile time.

EXAMPLES

To view ISO_8859-3:1988 document use:

2UTF --verbose --switch-to-UTF-8 8859-3 <document | less -r

To translate from CP1257 to BALTIC (ISO-IR-179) use:

2UTF -2 1257 <cp1257_file | fromUTF -2 baltic >baltic_charset_file

To call a BBS using 869 "code page" use:

minicom -l -t linux |2UTF --switch-to-UTF-8 IBM869

To convert everything from UTF-8 to US-ASCII:

fromUTF us-ascii <UTF-8_file

See also examples subdirectory.

FILES

There can be self-explanatory configuration file 2UTF.config. It is searched in /usr/local/etc/, /usr/etc/, /etc/ or other directories defined at compile time. It can specify directory names for char-map files and external filters for conversion to and from multi-byte char-sets and encodings.

BUG REPORTS

Bug reports, comments and suggestions please send to:

Ricardas Cepas <[email protected]> or <[email protected]>

BUGS

Due to the popen(3) bug in Linux libc inexistent commands in configuration file are not detected. So please check configuration file by hand.

Transformation from UTF-8 can be slow.

If char-map files you are using are incomplete characters can be lost.

Reverse transformation is not perfect.

See also TODO file.

Please use at your own risk.

COPYING

This program (including this man page) is distributed under BSD style license (see BSD_style_license file in the documentation directory) or GNU General Public License V2 except hdr.h, plan9.h and utf.c files (if used) from tcs(1), public domain code from mimedecode.c file. Copyright statements should be keeped unchanged.

For the copyright information see file copyright.