---
name: "datalad-crawl"
section: "1"
platform: "linux"
package: "datalad"
version: "0.4.1-1"
release: "Debian 9 (Stretch)"
platforms: ["linux"]
url: "https://new.carta.tech/datalad-crawl"
---

# datalad-crawl(1)

## SYNOPSIS

 **datalad-crawl** \[--version\] \[-h\] \[-l LEVEL\] \[-p {condor}\] \[--is-pipeline\] \[-t\]\
               \[-r\] \[-C CHDIR\]\
               \[file\]

## DESCRIPTION

Crawl online resource to create or update a dataset.\

Examples:\

  $ datalad crawl  # within a dataset having .datalad/crawl/crawl.cfg

## OPTIONS

  file                  configuration (or pipeline if --is-pipeline) file\
                         defining crawling, or a directory of a dataset on\
                         which to perform crawling using its standard crawling\
                         specification. Constraints: value must be a string\
                         \[Default: None\]

  **--version**       show the program's version and license information\
   **-h**, **--help**, **--help-np**\
                         show this help message. --help-np forcefully disables\
                         the use of a pager for displaying the help message\
   **-l** LEVEL, **--log-level** LEVEL\
                         set logging verbosity level. Choose among critical,\
                         error, warning, info, debug. Also you can specify an\
                         integer \<10 to provide even more debugging information\
   **-p** {condor}, **--pbs-runner** {condor}\
                         execute command by scheduling it via available PBS.\
                         For settings, config file will be consulted\
   **--is-pipeline**   flag if provided file is a Python script which defines\
                         pipeline(). \[Default: False\]\
   **-t**, **--is-template**\
                         flag if provided value is the name of the template to\
                         use. \[Default: False\]\
   **-r**, **--recursive**\
                         flag to crawl subdatasets as well (for now serially).\
                         \[Default: False\]\
   **-C** *CHDIR*, **--chdir** *CHDIR*\
                         directory to chdir to for crawling. Constraints: value\
                         must be a string \[Default: None\]

## AUTHORS

 datalad is developed by The DataLad Team and Contributors \<team@datalad.org\>.