--- name: "em_autobuild" section: "1" platform: "linux" package: "emdebian-tools" version: "1.4.3" release: "Debian 5.0 (Lenny)" platforms: ["linux"] url: "https://new.carta.tech/em_autobuild" --- # em\_autobuild(1) emdebian target package autobuilder ## NAME em\_autobuild - emdebian target package autobuilder ## SYNOPSIS **em\_autobuild** \[\[-a\] \| \[--arch\] *ARCHITECTURE*\] \[\[-e\] \| \[--errexit\]\] \[\[--script\] *FILENAME*\] \[\[-p\] \| \[--pbuilder\]\] \[\[--package\] *PACKAGE*\] \[\[-l\] \| \[--log\]\] \[\[-u\] \| \[--handler\]\] \[\[--machine-path\]*PATH*\] \[\[--machine\] *NAME* \[\[--variant\] *NAME*\]\] **em\_autobuild** \[\[-?\] \| \[-h\] \| \[--help\] \| \[--version\]\] ## DESCRIPTION Parses the */usr/lib/emdebian-tools/emdebian.crossd* suite script (by default) or a user-specified suite script, collates a list of base and required binary packages, converts each to the respective source package using the **apt-cross** cache and attempts to build each source package with **emsource** **--build-dep** **-b** $package. The same machine:variant support is available as in **emsource** itself except when using **--log** because **emsource** is then run inside the chroot instead of in the development environment. The build sequence halts when the first package fails to build cleanly - this includes lintian errors generated by the Emdebian lintian check set. (Errors from the standard Debian checks are ignored.) The logfile support is intended for unattended autobuilds via **at** or **cron**. Whilst **em\_autobuild** can be run as a normal user, various factors can complicate things when using a logfile because all output is redirected to the logfile. This can cause the autobuild to just halt without the user being able to provide the input necessary to give the sudo passphrase. Similarly, users who have an SVN username at www.emdebian.org will find that the SSH passphrase cache can time out, again without the ability to supply the passphrase. The easiest option is not to use a log file. One complication of this is that *machine:variant* support is not implemented if **--log** is used. To use **--log** on a development system, run **em\_autobuild** as the root user and in the **empdebuild** cross-building chroot. To use your existing **empdebuild** chroot: 1\. Ensure your **empdebuild** chroot is up to date: ``` $ sudo empdebuild update ``` 2\. Use the **--pbuilder** so that **em\_autobuild** can run **emsource** within the cross-building chroot. (The patches do need to be up to date to use this method.) 3\. Echo the command to **at** for the root user: ``` $ sudo echo "em_autobuild -l -o -a arm -p" | at now ``` This way, **sudo** is only needed once, the chroot is unpacked as root and **emsource** and **emdebuild** run within the chroot (your SVN username is not copied into the chroot). ## COMMANDS **-o**\|**--omit-current** Exclude packages that appear to be up to date. When running **em\_autobuild** via **at** or **cron**, remember that running scripts via the crontab for root means that Emdebian::Tools looks for data in */root/* instead of *$HOME*. The list of packages to omit is obtained via **emtargetcmp**. Another change that can catch out cron usage is incomplete locale configuration (causing lots of unnecessary perl warnings from the perl components of Emdebian::Tools). An example script to run an autobuilder: ``` #!/bin/sh export LANG=C export LC_ALL=C export LANGUAGE=C export ANSI_COLORS_DISABLED=true export HOME=/home/neil/ emtargetcmp em_autobuild --pbuilder --omit-current --log ``` - **--package** **PACKAGE** Auto-build just a single package - usually used with **--log**. **-h**\|**--help** print the usage message and exit. - **--version** print the usage message and exit. ## OPTIONS **-a**\|**--arch** *ARCHITECTURE* Override the **dpkg-cross** default architecture for this operation on the chroot. **-e**\|**--errexit** Allow the autobuild to halt at the first build that fails. Default behaviour is to carry on and build whatever packages can be built, recording those that fail using **embug**. **-s**\|**--script** *FILENAME* Override the default package selection and installation script with a customised debootstrap suite script (written in shell and compatible with whichever shell interpreter is to be installed on the target). Some customised scripts are provided with emdebian-tools. The default uses the standard Emdebian ´busybox´ package with ´dpkg´ and ´apt´. Replacement scripts need to be full debootstrap suite shell scripts that specify how to complete the first and second stage installations. Customised scripts packages with *emdebian-tools* include scripts for a root filesystem including libgtk2.0-0 and a complete GPE root filesystem. **-u**\|**--handler** Provide an executable script to do the upload of the build log and/or packages. To upload build logs after unattended autobuilds, specify a script which will be called once for each package with the full path to the build log file as the only argument. The script can then use whatever methods are supported in your environment to upload the build log. **--handler** is ignored if **--log** is not used. If the handler script also uploads the packages (e.g. using **emrecent** in noninteractive mode), the handler script needs to check that the build was successful then upload the packages before appending the .upload file to the end of the build log for that package. This allows the buildd.php report script to list the package as "Installed" instead of "pending". **-p**\|**--pbuilder** Run the autobuild in an **empbuilder** chroot (which must already exist and needs to be updated separately). **-l**\|**--log** **emdebuild** logs only the actual build process. **em\_autobuild** **--log** wraps that build log with details of the dependency installation (native and cross) at the start and the .changes contents at the end (if successful). When using --log, the emsource build log is written out to the package directory beneath *$WORK/buildd*, e.g. */opt/emdebian/buildd/a/apt/trunk/apt-arm-1213105765.log* . Log file support is intended for autobuilders where the detailed build log output is not useful when emailed to the admin by at or cron, instead the logs are intended for a website or similar output. Therefore, in **--log** mode, **em\_autobuild** simply outputs the list of packages being processed. The *$WORK/buildd* directory could then be made available over HTTP etc. When using **em\_autobuild**, note that enabling logging can cause you to be unable to enter passwords for SSH keys etc. (This is expected as --log is intended for use on unattended buildd machines that do not have SVN commit access and where the autobuild is likely to happen inside a chroot so that sudo is not an issue). When testing, consider using **-p** as well as **-l** or extending the length of time that your SSH and/or sudo passwords are cached. To monitor a logged autobuild (which is normally silent), use **ps**. e.g. if the terminal running the build is **tty1**: ``` $ ps fopid,f,cmd -t tty1 ``` - **--machine-path** *PATH* Override the default path to machine and variant configuration. By default, emsandbox uses *${WORK}/machine* where **$WORK** is the working directory specified to *emdebian-tools* in the debconf configuration. The specified path must already exist and contain the relevant *packages.conf* configuration as well as the *setup.sh* and *config.sh* shell scripts (which may be empty). **-m**\|**--machine** *MACHINE* Load machine specific configuration data from your Emdebian working directory. If no variant is specified, config is read from *$WORK/machine/$MACHINE/default/* where $WORK is the work directory specified in debconf for emdebian-tools. **-v**\|**--variant** *VARIANT* Load variant specific configuration data from your Emdebian working directory. Requires **--machine**. Configuration data is read from *$WORK/machine/$MACHINE/$VARIANT/* where $WORK is the work directory specified in debconf for emdebian-tools. ## MACHINE VARIANTS **emsandbox** supports a set of customisation routines for each combination of machine and variant, allowing the rootfs to be customised to specific variants of a specific machine. Configuration data is stored in the *machine* subdirectory of your Emdebian work directory. Using the **-m** option to **emsandbox** loads *packages.conf* from the *$WORK/machine/$MACHINE/default* subdirectory prior to starting debootstrap. Once the first stage install is complete, **emsandbox** calls *setup.sh* from the same directory, passing the location and architecture of the tarball, so that other fine tuning can take place prior to creating the tarball. At this stage, any operations inside the rootfs must not try to execute any binaries within the rootfs. Immediately before creating the tarball, *config.sh* is copied into the */machine/$MACHINE/default/* directory of the rootfs, ready to be called when **emsecondstage** has completed the second stage of the debootstrap process. Skeleton versions of *packages.conf*, *setup.sh* and *config.sh* are available in */usr/share/emdebian-tools/machine/*. *packages.conf* is intended to be the principal place for adjusting the emsandbox tarball to suit the needs of specific machine variants. *setup.sh* and *config.sh* can fine tune the results but in order to avoid reinventing the wheel, if more than a few machines need similar adjustments to the same files, future versions of *packages.conf* will collate those into a single configuration parameter available to all. *packages.conf* supports: **INCLUDE** Add a comma separated list of package names to the list of packages added to the tarball and installed in the second stage. Currently, debootstrap has problems with multiple repositories so either upload this package to the same repository as your other packages or create an apt-proxy that can serve as a local repository, set it in **PROXY** and specify a usable mirror for the device in **MIRROR**. DEFAULT: empty **SCRIPT** Overrides the default emsandbox suite-script that debootstrap uses to determine the base and required packages and the all important sequence in which the packages can be installed. SCRIPT can be overridden on the emsandbox command line. DEFAULT: */usr/lib/emdebian-tools/emdebian.crossd* **MIRROR** Overrides the default emsandbox mirror. This repository will be set in */etc/apt/sources.list* and will also be used by debootstrap to obtain all packages for the tarball unless **PROXY** is also set. DEFAULT: http://buildd.emdebian.org/emdebian/ **PROXY** Specifies a separate repository to pass to debootstrap that may be local or otherwise not intended for use once the tarball is installed. Use **MIRROR** to set the same value in debootstrap and */etc/apt/sources.list*. If **PROXY** is specified without **MIRROR**, the default emsandbox MIRROR (http://buildd.emdebian.org/emdebian/) will be written into */etc/apt/sources.list*. DEFAULT: empty **TARBALL\_NAME** Overrides the default name (emdebian-$ARCH) of the tarball. Do not specify a path here, just a filename with the .tgz suffix. DEFAULT: emdebian-$ARCH.tgz where $ARCH is specified to emsandbox or as the dpkg-cross default architecture. **SUITE** Not recommended to be changed. DEFAULT: unstable Due to limitations in the current debootstrap support, the only way of adding packages to the first stage is by providing a customised suite script. Even if emsandbox migrates to using a tool from Stag to overcome shortcomings in debootstrap, support for packages.conf, setup.sh and config.sh will remain. ## TIPS AND HINTS • Autobuilds can take a long time and downloading the native and cross dependencies can hog your network bandwidth, making it hard to do other work on the packages. Consider using **trickle**. From trickle (1): ``` $ trickle -u 10 -d 20 em_autobuild -a arm -p -l ... ``` Launch an autobuilder, limiting the upload capacity to 10 KB/s, and download capacity at 20 KB/s. ## AUTHOR **emsandbox** was written by Neil Williams \. This manual page was written by Neil Williams \ ## SEE ALSO See also *apt-cross* (1), *em\_make* (1), *dpkg-cross* (1), *emdebian-tools* (1). ## AUTHOR **Emdebian** **Neil Williams** - Author. ## COPYRIGHT