NAME
NoSQL - Relational Database Management System
INTRODUCTION
NoSQL is a fast, portable, Relational DataBase Management System without arbitary limits, (other than memory and processor speed).
NoSQL uses the 'operator/stream' DBMS paradigm described in "Unix Review", March, 1991, page 24, entitled "A 4GL Language". The operators are UNIX filters, i.e. they read STDIN and write STDOUT so they can be connected by pipes the ('|' char).
The operators work with relational data in ascii files. The fields in each row are separated by a 'TAB' char and the row is terminated with a 'NEWLINE' char. The first section of rows (the header) contains the names and data definitions for each column. The header also contains optional embedded documentation relating to the entire datafile and/or each data column. Subsequent rows (the body) contain the data values. A file in this form is said to be an 'rdbtable'. NoSQL is compliant with the 'Relational Model'.
These rdbtables are easily ported to and from spreadsheet programs on Macintoshes, MSDOS or MS-Windows computers.
All operators read an rdbtable via STDIN and most write a new rdbtable via STDOUT. Exceptions are operators that produce output listings instead of rdbtables (like 'nsq-pr' and 'nsq-report').
All operators take a '-help' option to print usage detail and other information online.
DESCRIPTION
Operators can be grouped into data movers, report generators, and utilities.
The data movers are operators that extract or rearrange the data in some way. They each read an rdbtable via STDIN and write an rdbtable via STDOUT and so are frequently connected using the UNIX pipe function to form a larger task. Each operator in such a "pipeline" style of operation gets its input from the output of the previous operator in the "pipeline". The data movers include:
- nsq-col
-
Selects columns by name, outputs columns in listed order.
- nsq-compute
-
Computes an arbitrary expression using column names.
- nsq-headchg
-
Generates and replaces (or removes) the header of an rdbtable.
- nsq-join
-
Natural or "Master/Detail" join of two rdbtables.
- nsq-l2t
-
Converts a file in list format to an rdbtable..
- nsq-merge
-
Merges two like rdbtables.
- nsq-row
-
Selects rows based on arbitrary Perl expressions.
- nsq-search
-
Selects rows based on a multi-column key of a sorted or indexed rdbtable.
- nsq-sort
-
Sorts a datafile by one or more columns.
- nsq-subtot
-
Lists subtotals of specified columns.
- nsq-t2l
-
Converts an rdbtable to list format.
- nsq-tee
-
Safely [over]writes an rdbtable.
- nsq-uniq
-
Makes an rdbtable unique on specified columns.
- nsq-updseq
-
Updates/Creates a unique record ID on each row of an rdbtable. The field must be the first column in the table.
The report generators each read an rdbtable via STDIN and produce a report on STDOUT, so when they are in a "pipeline" of operators they will be the operator at the end. The report generators are:
- nsq-pr
-
Quick and easy printing of output formatted from information in the header.
- nsq-report
-
Best form of output, with definable format.
- nsq-see
-
Prints the contents of a text file, visualizing TABS and Line-Ends.
- nsq-summ
-
Summary/Statistical information about data values in an rdbtable.
- nsq-valid
-
Verifies the structure of an rdbtable.
- nsq-words
-
Generates a list of unique words from a set of text files.
The utilities are used for manipulating the structure and content of rdbtables and are generally used as separate tasks, i.e. they do not read STDIN. The utilities are:
- nsq
-
NoSQL interactive terminal interface.
- nsq-ed
-
Uses an editor to allow modifications to an rdbtable.
- nsq-index
-
Generate rdbtable index files to be used by nsq-search.
- nsq-lock
-
Trivial table locking program, for serializing access to rdbtables.
- nsq-mktbl
-
Makes a file of data in columns into an rdbtable.
- nsq-repair
-
Attempts to repair candidate NoSQL datafiles.
- nsq-rollback
-
Prints an rdbtable the way it was at a point in time, using RCS.
All operators take a '-h[elp]' option to show details of operation online.
Data Types:
The type of data may be string, numeric, or month. The types are specified by an S, N, or M respectively in the header. The default is type string.
EXAMPLES
# Embedded documentation for this rdbtable.
# Any number of lines may exist.
# Also available for each column (not shown here).
NAME CNT TYP AMT
6 4N 3 5N
Hobbs 65 DD 1111
Bush 4 A 133
Perry 177 B 244
Jones 47 XY 77
Hansen 44 A 23
The above rdbtable after being processed by the command:
nsq-col < sample NAME TYP AMT CNT | nsq-sort TYP AMT | nsq-pr
NAME TYP AMT CNT
------ --- ----- ----
Hansen A 23 44
Bush A 133 4
Perry B 244 177
Hobbs DD 1111 65
Jones XY 77 47
ENVIRONMENT
Environment variables used by NoSQL:
- EDITOR
-
Used by the 'nsq-ed' utility (Default: 'vi').
- PAGER
-
Used by the 'nsq' utility (Default: 'less').
- TMPDIR
-
Where to store work files (Default: '/tmp').
- NSQLOCKER
-
Table locking program (Default: 'nsq-lock').
HISTORY
As of this writing, NoSQL is mostly an RDB clone. The original RDB author is Walt Hobbs, of RAND Organization (E-mail: <[email protected]>). NoSQL uses exactly the same table format as RDB, and therefore tables are called 'rdbtables' also in the NoSQL context.
AUTHOR
Carlo Strozzi <[email protected]>
BUGS
Speed is not the distinguishing feature of the original Perl operators. I'm trying to rewrite some of them in C + AWK.
The documentation is still incomplete.