---
name: "chirp_stream_files"
section: "1"
platform: "linux"
package: "coop-computing-tools"
version: "1:7.14.5-1+b2"
release: "Debian 13 (Trixie)"
platforms: ["linux"]
url: "https://new.carta.tech/chirp_stream_files"
---

# chirp\_stream\_files(1)

move data to/from chirp servers in parallel

## NAME

**chirp\_stream\_files** - move data to/from chirp servers in parallel

## SYNOPSIS

FCchirp\_stream\_files \[options\] *\<copy\|split\|join\>* *\<localfile\>* { *\<hostname\[:port\]\>* *\<remotefile\>*FT

## DESCRIPTION

**chirp\_stream\_files** is a tool for moving data from one machine to and from many machines, with the option to split or join the file along the way.  It is useful for constructing scatter-gather types of applications on top of Chirp.

FCchirp\_stream\_files copyFT duplicates a single file to multiple hosts.  The *\<localfile\>* argument names a file on the local filesystem.  The command will then open a connection to the following list of hosts, and stream the file to all simultaneously.

FCchirp\_stream\_files splitFT divides an ASCII file up among multiple hosts.  The first line of *\<localfile\>* is sent to the first host, the second line to the second, and so on, round-robin.

FCchirp\_stream\_files joinFT collects multiple remote files into one.  The argument *\<localfile\>* is opened for writing, and the remote files for reading.  The remote files are read line-by-line and assembled round-robin into the local file.

In all cases, files are accessed in a streaming manner, making this particularly efficient for processing large files.  A local file name of FC-FT indicates standard input or standard output, so that the command can be used in a pipeline.

## OPTIONS

- **-a,--auth=*\<flag\>***

   Enable authentication mode: unix, hostname, address, ticket, kerberos, or globus.

- **-b,--block-size=*\<size\>***

  Set transfer buffer size. (default is 1048576 bytes)

- **-d,--debug=*\<flag\>***

  Enable debugging for this subsystem.

- **-i,--tickes=*\<files\>***

  Comma-delimited list of tickets to use for authentication.

- **-t,--timeout=*\<time\>***

  Timeout for failure. (default is 3600s)

- **-v,--version**

  Show program version.

- **-h,--help**

  Show help text.

## ENVIRONMENT VARIABLES

List any environment variables used or set in this section.

## EXIT STATUS

On success, returns zero.  On failure, returns non-zero.

## EXAMPLES

To copy the file FCmydataFT to three locations:

```

```

-

  % chirp\_stream\_files copy mydata server1.somewhere.edu /mydata\
                                    server2.somewhere.edu /mydata\
                                    server2.somewhere.edu /mydata

To split the file FCmydataFT into subsets at three locations:

```

```

-

  % chirp\_stream\_files split mydata server1.somewhere.edu /part1\
                                     server2.somewhere.edu /part2\
                                     server2.somewhere.edu /part3

To join three remote files back into one called FCnewdataFT:

```

```

-

  % chirp\_stream\_files join newdata server1.somewhere.edu /part1\
                                     server2.somewhere.edu /part2\
                                     server2.somewhere.edu /part3

## COPYRIGHT

The Cooperative Computing Tools are Copyright (C) 2022 The University of Notre Dame.  This software is distributed under the GNU General Public License.  See the file COPYING for details.

## SEE ALSO

- •

  **Cooperative Computing Tools Documentation**

- •

  **Chirp User Manual**

- •

  **[chirp(1)](/chirp/1.md)**  **[chirp\_status(1)](/chirp_status/1.md)**  **[chirp\_fuse(1)](/chirp_fuse/1.md)**  **[chirp\_get(1)](/chirp_get/1.md)**  **[chirp\_put(1)](/chirp_put/1.md)**  **[chirp\_stream\_files(1)](/chirp_stream_files/1.md)**  **[chirp\_distribute(1)](/chirp_distribute/1.md)**  **[chirp\_benchmark(1)](/chirp_benchmark/1.md)**  **[chirp\_server(1)](/chirp_server/1.md)**