---
name: "condor_adstash"
section: "1"
platform: "linux"
package: "condor"
version: "23.9.6+dfsg-2.1+b10"
release: "Debian 13 (Trixie)"
platforms: ["linux"]
url: "https://new.carta.tech/condor_adstash"
---

# condor\_adstash(1)

HTCondor Manual Gather schedd and/or startd job history ClassAds and push them via a search engine or file interface.

## NAME

condor\_adstash - HTCondor Manual

Gather schedd and/or startd job history ClassAds and push them via a search engine or file interface.

## SYNOPSIS

**condor\_adstash** \[**--help** \]

**condor\_adstash** \[**--process\_name** *NAME*\] \[**--standalone** \] \[**--sample\_interval** *SECONDS*\] \[**--checkpoint\_file** *PATH*\] \[**--log\_file** *PATH*\] \[**--log\_level** *LEVEL*\] \[**--threads** *THREADS*\] \[**--interface** *{null,elasticsearch,jsonfile}*\] \[**--collectors** *COLLECTORS*\] \[**--schedds** *SCHEDDS*\] \[**--startds** *STARTDS*\] \[**--schedd\_history** \] \[**--startd\_history** \] \[**--schedd\_job\_epoch\_history** \] \[**--ad\_file** *PATH*\] \[**--schedd\_history\_max\_ads** *NUM\_ADS*\] \[**--startd\_history\_max\_ads** *NUM\_ADS*\] \[**--schedd\_history\_timeout** *SECONDS*\] \[**--startd\_history\_timeout** *SECONDS*\] \[**--se\_host** *HOST\[:PORT\]*\] \[**--se\_url\_prefix** *PREFIX*\] \[**--se\_username** *USERNAME*\] \[**--se\_use\_https** \] \[**--se\_timeout** SECONDS\] \[**--se\_bunch\_size** *NUM\_DOCS*\] \[**--es\_index\_name** *INDEX\_NAME*\] \[**--se\_no\_log\_mappings**\] \[**--se\_ca\_certs** *PATH*\] \[**--json\_dir** *PATH*\]

## DESCRIPTION

**condor\_adstash** is a tool that assists in monitoring usage by gathering job ClassAds (typically from *condor\_schedd* and/or *condor\_startd* history queries) and pushing the ClassAds as documents to some target (typically Elasticsearch).

Unless run in **--standalone** mode, *condor\_adstash* expects to be invoked as a daemon by a *condor\_master*, i.e. *condor\_adstash* should be invoked in standalone mode when run on the command-line. Whether invoked by *condor\_master* or run standalone, *condor\_adstash* gets its configuration, in increasing priority, from the HTCondor configuration macros beginning with **ADSTASH\_** (when **--process\_name** is not provided), then environment variables, and finally command-line options.

*condor\_adstash* must be able to write its **--checkpoint\_file** to a persistent location so that duplicate job ClassAds are not fetched from the daemons' histories in consecutive polls.

A named Elasticsearch index will be created if it doesn't exist, and may be modified if new fields (corresponding to ClassAd attribute names) need to be added. It is up to the administrator of the Elasticsearch instance to install rollover policies (e.g. ILM) on the named index and/or to set up the index as an alias.

## OPTIONS

- **-h**, **--help**

  Display the help message and exit.

- **--process\_name** *PREFIX*

  Give *condor\_adstash* a different name for looking up HTCondor configuration and environment variable values (see examples).

- **--standalone**

  Run condor\_adstash in standalone mode (runs once, does not attempt to contact *condor\_master*)

- **--sample\_interval** *SECONDS*

  Number of seconds between polling the list(s) of daemons (ignored in standalone mode)

- **--checkpoint\_file** *PATH*

  Location of checkpoint file (will be created if missing)

- **--log\_file** *PATH*

  Location of log file

- **--log\_level** *LEVEL*

  Log level (uses Python logging library levels: CRITICAL/ERROR/WARNING/INFO/DEBUG)

- **--threads** *THREADS*

  Number of parallel threads to use when polling for job ClassAds and when pushing documents to Elasticsearch

- **--interface** *{null,elasticsearch,opensearch,jsonfile}*

  Push ads via the chosen interface

## CLASSAD SOURCE OPTIONS

- **--schedd\_history**

  Poll and push *condor\_schedd* job histories

- **--startd\_history**

  Poll and push *condor\_startd* job histories

- **--schedd\_job\_epoch\_history**

  Poll and push *condor\_schedd* job epoch histories

- **--ad\_file** *PATH*

  Load Job ClassAds from a file instead of querying daemons (Ignores *--schedd\_history* and *--startd\_history*.)

## OPTIONS FOR HTCONDOR DAEMON (SCHEDD, STARTD, ETC.) HISTORY SOURCES

- **--collectors** *COLLECTORS*

  Comma-separated list of *condor\_collector* addresses to contact to locate *condor\_schedd* and *condor\_startd* daemons

- **--schedds** *SCHEDDS*

  Comma-separated list of *condor\_schedd* names to poll job histories from

- **--startds** *STARTDS*

  Comma-separated list of *condor\_startd* machines to poll job histories from

- **--schedd\_history\_max\_ads** *NUM\_ADS*

  Abort after reading NUM\_ADS from a *condor\_schedd*

- **--startd\_history\_max\_ads** *NUM\_ADS*

  Abort after reading NUM\_ADS from a *condor\_startd*

- **--schedd\_history\_timeout** *SECONDS*

  Abort if reading from a *condor\_schedd* takes more than this many seconds

- **--startd\_history\_timeout** *SECONDS*

  Abort if reading from a *condor\_startd* takes more than this many seconds

## SEARCH ENGINE (ELASTICSEARCH, OPENSEARCH, ETC.) INTERFACE OPTIONS

- **--se\_host** *HOST\[:PORT\]*

  Search engine host:port

- **--se\_url\_prefix** *PREFIX*

  Search engine URL prefix

- **--se\_username** *USERNAME*

  Search engine username

- **--se\_use\_https**

  Use HTTPS when connecting to search engine

- **--se\_timeout** *SECONDS*

  Max time to wait for search engine queries

- **--se\_bunch\_size** *NUM\_DOCS*

  Group ads in bunches of this size to send to search engine

- **--se\_index\_name** *INDEX\_NAME*

  Push ads to this search engine index or alias

- **--se\_no\_log\_mappings**

  Don't write a JSON file with mappings to the log directory

- **--se\_ca\_certs** *PATH*

  Path to root certificate authority file (will use certifi's CA if not set)

## JSON FILE INTERFACE OPTIONS

- **--json\_dir** *PATH*

  Directory to store JSON files, which are named by timestamp

## EXAMPLES

Running *condor\_adstash* in standalone mode on the command-line will result in *condor\_adstash* reading its configuration from the current HTCondor configuration:

```
$ condor_adstash --standalone
```

By default, *condor\_adstash* looks for HTCondor configuration variables with names are prefixed with **ADSTASH\_**, e.g. **ADSTASH\_READ\_SCHEDDS = \***. These values can be overridden on the command-line:

```
$ condor_adstash --standalone --schedds=myschedd.localdomain
```

*condor\_adstash* configuration variables can be also be named using custom prefixes, with the prefix passed in using **-\\-process\_name=PREFIX**. For example, if the HTCondor configuration contained **FOO\_SCHEDD\_HISTORY = False** and **FOO\_STARTD\_HISTORY = True**, *condor\_adstash* can be invoked to read these instead of **ADSTASH\_SCHEDD\_HISTORY** and **ADSTASH\_STARTD\_HISTORY**:

```
$ condor_adstash --standalone --process_name=FOO
```

Providing a **PREFIX** to **--process\_name** that does not match any HTCondor configuration variables will cause *condor\_adstash* to fallback to a default set of configuration values, which may be useful in debugging.

The configuration values that *condor\_adstash* reads from the current HTCondor configuration can be previewed by printing the help message. The values will be listed as the default values for each command-line option:

```
$ condor_adstash --help
$ condor_adstash --process_name=FOO --help
```

## AUTHOR

HTCondor Team

## COPYRIGHT

1990-2024, Center for High Throughput Computing, Computer Sciences Department, University of Wisconsin-Madison, Madison, WI, US. Licensed under the Apache License, Version 2.0.