---
name: "nq"
section: "1"
platform: "linux"
package: "nq"
version: "1.0-0.2"
release: "Debian 13 (Trixie)"
platforms: ["linux"]
url: "https://new.carta.tech/nq"
---

# nq(1)

job queue utility

## NAME

**nq** – job queue utility

## SYNOPSIS

**nq** \[**-c**\] \[**-q**\] *command line* *...*

**nq** **-t** *job id* *...*

**nq** **-w** *job id* *...*

## DESCRIPTION

The **nq** utility provides a very lightweight queuing system without requiring setup, maintenance, supervision or any long-running processes.

Job order is enforced by a timestamp **nq** gets immediately when started. Synchronization happens on file-system level. Timer resolution is milliseconds. No sub-second file system time stamps are required. Polling is not used. Exclusive execution is maintained strictly.

You enqueue(!) new jobs into the queue by running

```
nq command line ...
```

The job id (a file name relative to NQDIR, which defaults to the current directory) is output (unless suppressed using **-q**) and **nq** detaches from the terminal immediately, running the job in the background. Standard output and standard error are redirected into the job id file. [nqtail(1)](/nqtail/1.md) can be used to conveniently watch the log files.

The options are as follows:

- **-c**

  Clean up job id file when process exited with status 0.

- **-q**

  Suppress output of the job id after spawning new job.

- **-t**

  Enter *test* *mode*: exit with status 0 when *all* of the listed job ids are already done, else with status 1.

- **-w**

  Enter *waiting* *mode*: wait in the foreground until *all* listed job ids are done.

## ENVIRONMENT

- NQDIR

  Directory where lock files/job output resides. Each NQDIR can be considered a separate queue. The current working directory is used when NQDIR is unset. NQDIR is created if needed.

- NQDONEDIR

  When set, specifies a directory( must be on the same file system as NQDIR) where lock files/job output is moved to after successful execution of the job.

  Ignored when **-c** is used.

- NQFAILDIR

  When set, specifies a directory( must be on the same file system as NQDIR) where lock files/job output is moved to after unsuccessful execution of the job.

- NQJOBID

  The job id of the currently running job, exposed to the job itself.

## FILES

**nq** expects to control all files in NQDIR (respectively *.*) which start with “,” or “.,”. These files are created according to the following scheme:

```
,hexadecimal-time-stamp.pid
```

## EXIT STATUS

The **nq** utility exits 0 on success, and \>0 if an error occurs; unless *test* *mode* is used, in which case exit status 1 means there is a job running.

On fatal errors, exit codes 111 and 222 are used.

## EXAMPLES

Build [make(1)](/make/1.md) targets **clean**, **depends**, **all**, without occupying the terminal:

```
% nq make clean
% nq make depends
% nq make all
% nqtail
... look at output, can interrupt with C-c any time
without stopping the build ...
```

Simple download queue, accessible from multiple terminals:

```
% alias qget='NQDIR=/tmp/downloads nq wget'
% alias qwait='NQDIR=/tmp/downloads nqtail -q'
window1% qget http://mymirror/big1.iso
window2% qget http://mymirror/big2.iso
window3% qget http://mymirror/big3.iso
% qwait
... wait for all downloads to finish ...
```

As [nohup(1)](/nohup/1.md) replacement (The benchmark will run in background, every run gets a different output file, and the command line you ran is logged too.):

```
% ssh remote
remote% nq ./run-benchmark
,14f6f3034f8.17035
remote% ^D
% ssh remote
remote% nqtail
... see output, nqtail exits when job finished ...
```

## TRICKS

The "file extension" of the log file is actually the PID of the job. **nq** runs all jobs in a separate process group, so you can kill an entire job process tree at once using [kill(1)](/kill/1.md) with a negative PID. Before the job is started, it is the PID of **nq**, so you can cancel a queued job by killing it as well.

Thanks to the initial exec line in the log files, you can resubmit a job by executing it as a shell command file, i.e. running

```
sh job id
```

Creating **nq** wrappers setting NQDIR to provide different queues for different purposes is encouraged.

## INTERNALS

Enforcing job order works like this:

- every job has an output file locked using [flock(2)](/flock/2.md) and named according to FILES.

- every job starts only after all earlier flocked files are unlocked.

- the lock is released by the kernel after the job terminates.

## ASSUMPTIONS

**nq** will only work correctly when:

- NQDIR (respectively *.*) is writable.

- [flock(2)](/flock/2.md) works correctly in NQDIR (respectively *.*).

- [gettimeofday(2)](/gettimeofday/2.md) behaves monotonic (using CLOCK\_MONOTONIC would create confusing file names after reboot).

- No other programs put files matching ,\* into NQDIR (respectively *.*).

## SEE ALSO

[nqtail(1)](/nqtail/1.md), [nqterm(1)](/nqterm/1.md).

Alternatives to the **nq** system include [batch(1)](/batch/1.md), [qsub(1)](/qsub/1.md), [schedule(1)](/schedule/1.md), [srun(1)](/srun/1.md), and [ts(1)](/ts/1.md).

## AUTHORS

Leah Neukirchen ⟨*leah@vuxu.org*⟩

## CAVEATS

All reliable queue status information is in main memory only, which makes restarting a job queue after a reboot difficult.

## LICENSE

**nq** is in the public domain.

To the extent possible under law, the creator of this work has waived all copyright and related or neighboring rights to this work.

**http://creativecommons.org/publicdomain/zero/1.0/**