--- name: "bsd.regress.mk" section: "5" platform: "openbsd" package: "man" version: "8.0-current-20261101" release: "OpenBSD 8.0-current (2026-11-01)" platforms: ["openbsd"] url: "https://new.carta.tech/bsd.regress.mk" --- # bsd.regress.mk(5) regression test master Makefile fragment ## NAME **bsd.regress.mk** – regression test master Makefile fragment ## SYNOPSIS **.include \** ## DESCRIPTION **bsd.regress.mk** holds the standard routines used by the source regression tests. Some variables and targets are for its internal use only. The rest are documented here. Since this file also includes **bsd.prog.mk**, all of the usual OpenBSD Makefile variables may be used to build the regression test programs. ## TARGETS - **depend** Build any dependencies required to carry out the current set of regression tests. - **regress** Executes the regression targets defined in the Makefile. If a test fails, the line "FAILED" is printed to the standard output. By default, execution stops after the first test failure and **make** **regress** exits with a non-zero status. - **run-regress-\*** Runs an individual regression test. If the exit status of the program indicates an error or timeout, then a failure is logged, otherwise the test is marked as a success. ## VARIABLES - REGRESS\_CLEANUP If this variable is set, it contains a target that is executed after all regress targets. - REGRESS\_EXPECTED\_FAILURES Targets which are expected to fail. Specifically: sometimes tests are added for features that should be supported eventually, but are somewhat complicated to fix right away. - REGRESS\_FAIL\_EARLY If this variable is set to anything but “no”, the **regress** target will abort as soon as a test fails. Defaults to “yes” unless REGRESS\_LOG is set. - REGRESS\_LOG Points to the fully-qualified path of a file to which regression results are appended. Defaults to */dev/null*. If set to any other path, REGRESS\_FAIL\_EARLY defaults to “no”. - REGRESS\_ROOT\_TARGETS Targets for which root access is required to run the test. The SUDO variable should be used in the test parts that require root. See also SUDO. - REGRESS\_SETUP If this variable is set, it contains a target that is executed before each regress target. Regress targets depend on this one to enforce setup. - REGRESS\_SETUP\_ONCE If this variable is set, it contains a target that is executed once before all regress targets. Regress targets depend on a stamp file generated during setup. - REGRESS\_SKIP\_SLOW If this variable is not empty, skip over all the regression tests which have been marked as being 'slow' using the REGRESS\_SLOW\_TARGETS variable. - REGRESS\_SKIP\_TARGETS Targets which are unconditionally skipped. - REGRESS\_SLOW\_TARGETS Targets which are defined as 'slow'. All of these tests can be skipped by setting the REGRESS\_SKIP\_SLOW variable. - REGRESS\_TARGETS Targets which are invoked to run the set of regression tests for this Makefile. Defaults to **run-regress-${PROG}**. - SUDO Location of a command used to switch to root for certain test targets which require it. See [doas(1)](/doas/1.md). Some variables are intended to be set at runtime in the environment or in [mk.conf(5)](/mk.conf/5.md), but not in the regress Makefile itself. ## GUIDELINES If an individual test passes, **make** *testname* should return with an exit status of 0. If it fails, it should return with a non-zero exit status. If a test cannot be executed because a package is not installed or some environment variable is not set, **make** *testname* should print "SKIPPED" to stdout and exit with status 0. To skip everything, implement the **regress** target with a command that prints "SKIPPED". Some tests may require a special setup on the test machine that has to be done manually before testing. This requirement has to be documented in the Makefile or in a *README* file. The test should find out whether the setup exists before running and print "SKIPPED" and exit if it is missing. Tests should not fail because an intended feature has not been implemented yet. To avoid such false failures, a test should show the reason, print "DISABLED" to stdout and exit with status 0. Tests must be able to run with an *obj* directory. In case the test is not linked to the build or the tester forgot to run **make** **obj** before, this directory or symlink may not exist. Then the test should run anyway. Tests are executed with **make** **regress**, but running **make** **all** or **make** should have the same effect. Tests must be runnable by root, and may also succeed when run as a regular user. Tests must not assume they have a controlling tty, to allow them to be run by [cron(8)](/cron/8.md). An individual regress test may create a pseudo tty if it needs one. Tests should use the binaries installed and the kernel running on the local system. They may use environment variables to test alternative binaries or remote kernels running on other machines. In some cases a test may need binaries or libraries or object files to be present in */usr/obj/* that exist only after **make** **build** was run in */usr/src/*. The test must not assume that they have already been built, but should run **make** in the appropriate source directory as a dependency. For missing generated source or header files a target called **make** **generated** is common. The */usr/src/* tree can be found with a relative path or with the BSDSRCDIR variable. Tests should generally not set or modify MALLOC\_OPTIONS unless it is testing specific behaviour that depends on a particular flag or combination of flags. When running tests, the appropriate sysctl (vm.malloc\_conf) could be used instead. ## SEE ALSO [bsd.port.mk(5)](/bsd.port.mk/5.md) ## HISTORY The regression system originally came from NetBSD, with many tests added by OpenBSD since. The current Makefile framework was written by Artur Grabowski and Marc Espie for OpenBSD 3.1.