---
name: "public-inbox-extindex"
section: "1"
platform: "linux"
package: "public-inbox"
version: "1.9.0-1"
release: "Debian 13 (Trixie)"
platforms: ["linux"]
url: "https://new.carta.tech/public-inbox-extindex"
---

# public-inbox-extindex(1)

create and update external search indices

## NAME

public-inbox-extindex - create and update external search indices

## SYNOPSIS

public-inbox-extindex \[OPTIONS\] EXTINDEX\_DIR INBOX\_DIR...

public-inbox-extindex \[OPTIONS\] \[EXTINDEX\_DIR\] --all

## DESCRIPTION

public-inbox-extindex creates and updates an external search and overview database used by the read-only public-inbox PSGI (HTTP), NNTP, and IMAP interfaces.  This requires either the Search::Xapian XS bindings OR the Xapian SWIG bindings, along with DBD::SQLite and DBI Perl modules.

## OPTIONS

- \-j JOBS

  - \--jobs=JOBS

  - \--no-fsync

  - \--dangerous

  - \--rethread

  - \--max-size SIZE

  - \--batch-size SIZE

  These switches behave as they do for [**public-inbox-index**(1)](/public-inbox-index/1.md)

- \--all

  Index all `"publicinbox"` entries in `"PI_CONFIG"`.

  `"publicinbox"` entries indexed by `"public-inbox-extindex"` can have full Xapian searching abilities with the per-`"publicinbox"` `"indexlevel"` set to `"basic"` and their respective Xapian (`"xap15"` or `"xapian15"`) directories removed.  For multiple public-inboxes where cross-posting is common, this allows significant space savings on Xapian indices.

- \--gc

  Perform garbage collection instead of indexing.  Use this if inboxes are removed from the extindex, or if messages are purged or removed from some inboxes.

- \--reindex

  Forces a re-index of all messages in the extindex.  This can be used for in-place upgrades and bugfixes while read-only server processes are utilizing the index.  Keep in mind this roughly doubles the size of the already-large Xapian database.

  The extindex locks will be released roughly every 10s to allow [**public-inbox-mda**(1)](/public-inbox-mda/1.md) and [**public-inbox-watch**(1)](/public-inbox-watch/1.md) processes to write to the extindex.

- \--fast

  Used with `"--reindex"`, it will only look for new and stale entries and not touch already-indexed messages.

## FILES

[**public-inbox-extindex-format**(5)](/public-inbox-extindex-format/5.md)

## CONFIGURATION

public-inbox-extindex does not currently write to the [**public-inbox-config**(5)](/public-inbox-config/5.md) file, configuration may be entered manually.  The extindex name of `"all"` is a special case which corresponds to indexing `"--all"` inboxes.  An example for `"--all"` is as follows:

```
        [extindex "all"]
                topdir = /path/to/extindex_dir
                url = all
                coderepo = foo
                coderepo = bar
```

See [**public-inbox-config**(5)](/public-inbox-config/5.md) for more details.

## ENVIRONMENT

- PI\_CONFIG

  Used to override the default "~/.public-inbox/config" value.

- XAPIAN\_FLUSH\_THRESHOLD

  The number of documents to update before committing changes to disk.  This environment is handled directly by Xapian, refer to Xapian API documentation for more details.

  Setting `"XAPIAN_FLUSH_THRESHOLD"` or `"publicinbox.indexBatchSize"` for a large `"--reindex"` may cause [**public-inbox-mda**(1)](/public-inbox-mda/1.md), [**public-inbox-learn**(1)](/public-inbox-learn/1.md) and [**public-inbox-watch**(1)](/public-inbox-watch/1.md) tasks to wait long and unpredictable periods of time during `"--reindex"`.

  Default: none, uses `"publicinbox.indexBatchSize"`

## UPGRADING

Occasionally, public-inbox will update it's schema version and require a full index by running this command.

## CONTACT

Feedback welcome via plain-text mail to \<mailto:meta@public-inbox.org\>

The mail archives are hosted at \<https://public-inbox.org/meta/\> and \<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/\>

## COPYRIGHT

Copyright all contributors \<mailto:meta@public-inbox.org\>

License: AGPL-3.0+ \<https://www.gnu.org/licenses/agpl-3.0.txt\>

## SEE ALSO

Search::Xapian, DBD::SQLite