--- name: "PAR::Tutorial" section: "3pm" platform: "linux" package: "perl-par" version: "1.021-2" release: "Arch Linux 2026-11-01" platforms: ["linux"] url: "https://new.carta.tech/PAR::Tutorial" --- # PAR::Tutorial(3pm) Cross-Platform Packaging and Deployment with PAR ## NAME PAR::Tutorial - Cross-Platform Packaging and Deployment with PAR ## SYNOPSIS This is a tutorial on PAR, first appeared at the 7th Perl Conference. The HTML version of this tutorial is available online as \ ## DESCRIPTION ### On Deploying Perl Applications ``` % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101" Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1. BEGIN failed--compilation aborted at sshnuke.pl line 1. ``` - • Q: "Help! I can't run your program!" - • A1: Install Perl & `"perl -MCPAN -e'install(...)'"` - • How do we know which modules are needed? - • New versions of CPAN modules may break `"sshnuke.pl"` - • A2: Install Perl & `"tar zxf my_perllib.tgz"` - • Possibly overwriting existing modules; not cross-platform at all - • A3: Use the executable generated by `"perlcc sshnuke.pl"` - • Impossible to debug; `"perlcc"` usually does not work anyway ### PAR, the Perl Archive Toolkit - • Do what JAR (Java Archive) does for Perl - • Aggregates modules, scripts and other files into a Zip file - • Easy to generate, update and extract - • Version consistency: solves forward-compatibility problems - • Developed by community: `"par@perl.org"` - • PAR files can be packed into self-contained scripts - • Automatically scans perl script for dependencies - • Bundles all necessary 3rd-party modules with it - • Requires only core Perl to run on the target machine - • PAR also comes with `"pp"`, the Perl Packager: ``` % pp -o sshnuke.exe sshnuke.pl # stand-alone executable! ``` ### Simple Packaging - • PAR files are just Zip files with modules in it - • Any Zip tools can generate them: ``` % zip foo.par Hello.pm World.pm # pack two modules % zip -r bar.par lib/ # grab all modules in lib/ ``` - • To load modules from PAR files: ``` use PAR; use lib "foo.par"; # the .par part is optional use Hello; ``` - • This also works: ``` use PAR "/home/mylibs/*.par"; # put all of them into @INC use Hello; ``` ### PAR Loaders - • Use `"par.pl"` to run files inside a PAR archive: ``` % par.pl foo.par # looks for 'main.pl' by default % par.pl foo.par test.pl # runs script/test.pl in foo.par ``` - • Same thing, with the stand-alone `"parl"` or `"parl.exe"`: ``` % parl foo.par # no perl or PAR.pm needed! % parl foo.par test.pl # ditto ``` - • The PAR loader can prepend itself to a PAR file: - • `"-b"` bundles non-core modules needed by `"PAR.pm"`: ``` % par.pl -b -O./foo.pl foo.par # self-contained script ``` - • `"-B"` bundles core modules in addition to `"-b"`: ``` % parl -B -O./foo.exe foo.par # self-contained binary ``` ### Dependency Scanning - • Recursively scan dependencies with `"scandeps.pl"`: ``` % scandeps.pl sshnuke.pl # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN 'Crypt::SSLeay' => '0', # X # 'Net::HTTP' => '0', # # 'Crypt::SSLeay::X509' => '0', # S # Crypt::SSLeay 'Net::HTTP::Methods' => '0', # S # Net::HTTP 'Compress::Zlib' => '0', # X # Net::HTTP::Methods ``` - • Scan an one-liner, list all involved files: ``` % scandeps.pl -V -e "use Dynaloader;" ... # auto/DynaLoader/dl_findfile.al [autoload] # auto/DynaLoader/extralibs.ld [autoload] # auto/File/Glob/Glob.bs [data] # auto/File/Glob/Glob.so [shared] ... ``` ### Perl Packager: "pp" - • Combines scanning, zipping and loader-embedding: ``` % pp -o out.exe src.pl # self-contained .exe % out.exe # runs anywhere on the same OS ``` - • Bundle additional modules: ``` % pp -o out.exe -M CGI src.pl # pack CGI + its dependencies, too ``` - • Pack one-liners: ``` % pp -o out.exe -e 'print "Hi!"' # turns one-liner into executable ``` - • Generate PAR files instead of executables: ``` % pp -p src.pl # makes 'source.par' % pp -B -p src.pl # include core modules ``` ### How it works - • Command-line options are almost identical to `"perlcc"`'s - • Also supports `"gcc"`-style long options: ``` % pp --gui --verbose --output=out.exe src.pl ``` - • Small initial overhead; no runtime overhead - • Dependencies are POD-stripped before packing - • Loads modules directly into memory on demand - • Shared libraries (DLLs) are extracted with File::Temp - • Works on Perl 5.6.0 or above - • Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX, Cygwin, AIX, Solaris, HP-UX, Tru64... ### Aggregating multiple programs - • A common question: ``` > I have used pp to make several standalone applications which work > great, the only problem is that for each executable that I make, I am > assuming the parl.exe is somehow bundled into the resulting exe. ``` - • The obvious workaround: ``` You can ship parl.exe by itself, along with .par files built by "pp -p", and run those PAR files by associating them to parl.exe. ``` - • On platforms that have `"ln"`, there is a better solution: ``` % pp --output=a.out a.pl b.pl # two scripts in one! % ln a.out b.out # symlink also works % ./a.out # runs a.pl % ./b.out # runs b.pl ``` ### Cross-platform Packages - • Of course, there is no cross-platform binary format - • Pure-perl PAR packages are cross-platform by default - • However, XS modules are specific to Perl version and platform - • Multiple versions of a XS module can co-exist in a PAR file - • Suppose we need `"out.par"` on both Win32 and Finix: ``` C:\> pp --multiarch --output=out.par src.pl ...copy src.pl and out.par to a Finix machine... % pp --multiarch --output=out.par src.pl ``` - • Now it works on both platforms: ``` % parl out.par # runs src.pl % perl -MPAR=out.par -e '...' # uses modules inside out.par ``` ### The Anatomy of a PAR file - • Modules can reside in several directories: ``` / # casual packaging only /lib/ # standard location /arch/ # for creating from blib/ /i386-freebsd/ # i.e. $Config{archname} /5.8.0/ # i.e. Perl version number /5.8.0/i386-freebsd/ # combination of the two above ``` - • Scripts are stored in one of the two locations: ``` / # casual packaging only /script/ # standard location ``` - • Shared libraries may be architecture- or perl-version-specific: ``` /shlib/(5.8.0/)?(i386-freebsd/)? ``` - • PAR files may recursively contain other PAR files: ``` /par/(5.8.0/)?(i386-freebsd/)? ``` ### Special files - • MANIFEST - • Index of all files inside PAR - • Can be parsed with `"ExtUtils::Manifest"` - • META.yml - • Dependency, license, runtime options - • Can be parsed with `"YAML"` - • SIGNATURE - • OpenPGP-signed digital signature - • Can be parsed and verified with `"Module::Signature"` ### Advantages over perlcc, PerlApp and Perl2exe - • This is not meant to be a flame - • All three maintainers have contributed to PAR directly; I'm grateful - • perlcc - • "The code generated in this way is not guaranteed to work... Use for production purposes is strongly discouraged." (from perldoc perlcc) - • *Guaranteed to not work* is more like it - • PerlApp / Perl2exe - • Expensive: Need to pay for each upgrade - • Non-portable: Only available for limited platforms - • Proprietary: Cannot extend its features or fix bugs - • Obfuscated: Vendor and black-hats can see your code, but you can't - • Inflexible: Does not work with existing Perl installations ### MANIFEST: Best viewed with Mozilla - • The URL of `"MANIFEST"` inside `"/home/autrijus/foo.par"`: ``` jar:file:///home/autrijus/foo.par!/MANIFEST ``` - • Open it in a Gecko browser (e.g. Netscape 6+) with Javascript enabled: - • No needed to unzip anything; just click on files to view them ### META.yml: Metadata galore - • Static, machine-readable distribution metadata - • Supported by `"Module::Build"`, `"ExtUtils::MakeMaker"`, `"Module::Install"` - • A typical `"pp"`-generated `"META.yml"` looks like this: ``` build_requires: {} conflicts: {} dist_name: out.par distribution_type: par dynamic_config: 0 generated_by: 'Perl Packager version 0.03' license: unknown par: clean: 0 signature: '' verbatim: 0 version: 0.68 ``` - • The `"par:"` settings controls its runtime behavior ### SIGNATURE: Signing and verifying packages - • OpenPGP clear-signed manifest with SHA1 digests - • Supported by `"Module::Signature"`, `"CPANPLUS"` and `"Module::Build"` - • A typical `"SIGNATURE"` looks like this: ``` -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS ... -----BEGIN PGP SIGNATURE----- ... -----END PGP SIGNATURE----- ``` - • Use `"pp"` and `"cpansign"` to work with signatures: ``` % pp -s -o foo.par bar.pl # make and sign foo.par from bar.pl % cpansign -s foo.par # sign this PAR file % cpansign -v foo.par # verify this PAR file ``` ### Perl Servlets with Apache::PAR - • Framework for self-contained Web applications - • Similar to Java's "Web Application Archive" (WAR) files - • Works with mod\_perl 1.x or 2.x - • A complete web application inside a `".par"` file - • Apache configuration, static files, Perl modules... - • Supports Static, Registry and PerlRun handlers - • Can also load all PARs under a directory - • One additional special file: `"web.conf"` ``` Alias /myapp/cgi-perl/ ##PARFILE##/ Options +ExecCGI SetHandler perl-script PerlHandler Apache::PAR::Registry ``` ### Hon Dah, A-par-che! - • First, make a `"hondah.par"` from an one-liner: ``` # use the "web.conf" from the previous slide % pp -p -o hondah.par -e 'print "Hon Dah!\n"' \ --add web.conf % chmod a+x hondah.par ``` - • Add this to `"httpd.conf"`, then restart apache: ``` PerlModule Apache2 PerlAddVar PARInclude /home/autrijus/hondah.par PerlModule Apache::PAR ``` - • Test it out: ``` % GET http://localhost/myapp/cgi-perl/main.pl Hon Dah! ``` - • Instant one-liner web application that works! ### On-demand library fetching - • With LWP installed, your can use remote PAR files: ``` use PAR; use lib 'http://aut.dyndns.org/par/DBI-latest.par'; use DBI; # always up to date! ``` - • Modules are now cached under `$ENV{PAR_GLOBAL_TEMP}` - • Auto-updates with `"LWP::Simple::mirror"` - • Download only if modified - • Safe for offline use after the first time - • May use `"SIGNATURE"` to prevent DNS-spoofing - • Makes large-scale deployment a breeze - • Upgrades from a central location - • No installers needed ### Code Obfuscation - • Also known as *source-hiding* techniques - • It is *not* encryption - • Offered by PerlApp, Perl2Exe, Stunnix... - • Usually easy to defeat - • Take optree dump from memory, feed to `"B::Deparse"` - • If you just want to stop a casual `"grep"`, "deflate" already works - • PAR now supports pluggable *input filters* with `"pp -f"` - • Bundled examples: Bleach, PodStrip and PatchContent - • True encryption using `"Crypt::*"` - • Or even \_product activation\_ over the internet - • Alternatively, just keep core logic in your server and use RPC ### Accessing packed files - • To get the host archive from a packed program: ``` my $zip = PAR::par_handle($0); # an Archive::Zip object my $content = $zip->contents('MANIFEST'); ``` - • Same thing, but with `read_file()`: ``` my $content = PAR::read_file('MANIFEST'); ``` - • Loaded PAR files are stored in `%PAR::LibCache`: ``` use PAR '/home/mylibs/*.par'; while (my ($filename, $zip) = each %PAR::LibCache) { print "[$filename - MANIFEST]\n"; print $zip->contents('MANIFEST'); } ``` ### Packing GUI applications - • GUI toolkits often need to link with shared libraries: ``` # search for libncurses under library paths and pack it % pp -l ncurses curses_app.pl # same for Tk, Wx, Gtk, Qt... ``` - • Use `"pp --gui"` on Win32 to eliminate the console window: ``` # pack 'src.pl' into a console-less 'out.exe' (Win32 only) % pp --gui -o out.exe src.pl ``` - • "Can't locate Foo/Widget/Bar.pm in `@INC`"? - • Some toolkits (notably Tk) autoloads modules without `"use"` or `"require"` - • Hence `"pp"` and `"Module::ScanDeps"` may fail to detect them - • Tk problems mostly fixed by now, but other toolkits may still break - • You can work around it with `"pp -M"` or an explicit `"require"` - • Or better, send a short test-case to `"par@perl.org"` so we can fix it ### Precompiled CPAN distributions - • Installing XS extensions from CPAN was difficult - • Some platforms do not come with a compiler (Win32, MacOSX...) - • Some headers or libraries may be missing - • PAR.pm itself used to suffer from both problems - • ...but not anymore -- `"Module::Install"` to the rescue! ``` # same old Makefile.PL, with a few changes use inc::Module::Install; # was "use ExtUtils::MakeMaker;" WriteMakefile( ... ); # same as the original check_nmake(); # make sure the user have nmake par_base('AUTRIJUS'); # your CPAN ID or a URL fetch_par() unless can_cc(); # use precompiled PAR only if necessary ``` - • Users will not notice anything, except now it works - • Of course, you still need to type `"make par"` and upload the precompiled package - • PAR users can also install it directly with `"parl -i"` ### Thank you! - • Additional resources - • Mailing list: `"par@perl.org"` - • Subscribe: Send a blank email to `"par-subscribe@perl.org"` - • List archive: \ - • PAR::Intro: \ - • Apache::PAR: \ - • Module::Install: \ - • Any questions? ### Overview of PAR.pm's Implementation - • Here begins the scary part - • Grues, Dragons and Jabberwocks abound... - • You are going to learn weird things about Perl internals - • PAR invokes four areas of Perl arcana: - • `@INC` code references - • On-the-fly source filtering - • Overriding `DynaLoader::bootstrap()` to handle XS modules - • Making self-bootstrapping binary executables - • The first two only works on 5.6 or later - • DynaLoader and `%INC` are there since Perl 5 was born - • PAR currently needs 5.6, but a 5.005 port is possible ### Code References in @INC - • On 1999-07-19, Ken Fox submitted a patch to P5P - • To \_enable using remote modules\_ by putting hooks in `@INC` - • It's accepted to come in Perl 5.6, but undocumented until 5.8 - • Type `"perldoc -f require"` to read the nitty-gritty details - • Coderefs in `@INC` may return a fh, or undef to 'pass': ``` push @INC, sub { my ($coderef, $filename) = @_; # $coderef is \&my_sub open my $fh, "-|", "wget ftp://example.com/$filename"; return $fh; # using remote modules, indeed! }; ``` - • Perl 5.8 let you open a file handle to a string, so we just use that: ``` open my $fh, "<", \($zip->memberNamed($filename)->contents); return $fh; ``` - • But Perl 5.6 does not have that, and I don't want to use temp files... ### Source Filtering without Filter::\* Modules - • ... Undocumented features to the rescue! - • It turns out that `@INC` hooks can return **two** values - • The first is still the file handle - • The second is a code reference for line-by-line source filtering! - • This is how `"Acme::use::strict::with::pride"` works: ``` # Force all modules used to use strict and warnings open my $fh, "<", $filename or return; my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n"); return ($fh, sub { return 0 unless @lines; push @lines, $_; $_ = shift @lines; return length $_; }); ``` ### Source Filtering without Filter::\* Modules (cont.) - • But we don't really have a filehandle for anything - • Another undocumented feature saves the day! - • We can actually omit the first return value altogether: ``` # Return all contents line-by-line from the file inside PAR my @lines = split( /(?<=\n)/, $zip->memberNamed($filename)->contents ); return (sub { $_ = shift(@lines); return length $_; }); ``` ### Overriding DynaLoader::bootstrap - • XS modules have dynamically loaded libraries - • They cannot be loaded as part of a zip file, so we extract them out - • Must intercept DynaLoader's library-finding process - • Module names are passed to `"bootstrap"` for XS loading - • During the process, it calls `"dl_findfile"` to locate the file - • So we install pre-hooks around both functions - • Our `"_bootstrap"` just checks if the library is in PARs - • If yes, extract it to a `"File::Temp"` temp file - • The file will be automatically cleaned up when the program ends - • It then pass the arguments to the original `"bootstrap"` - • Finally, our `"dl_findfile"` intercepts known filenames and return it ### Anatomy of a Self-Contained PAR executable - • The par script ($0) itself - • May be in plain-text or native executable format - • Any number of embedded files - • Typically used to bootstrap PAR's various dependencies - • Each section begins with the magic string "FILE" - • Length of filename in pack('N') format and the filename (auto/.../) - • File length in pack('N') and the file's content (not compressed) - • One PAR file - • Just a regular zip file with the magic string `"PK\003\004"` - • Ending section - • A pack('N') number of the total length of FILE and PAR sections - • Finally, there must be a 8-bytes magic string: `"\012PAR.pm\012"` ### Self-Bootstrapping Tricks - • All we can expect is a working perl interpreter - • The self-contained script \*must not\* use any modules at all - • But to process PAR files, we need XS modules like Compress::Zlib - • Answer: bundle all modules + libraries used by PAR.pm - • That's what the `"FILE"` section in the previous slide is for - • Load modules to memory, and write object files to disk - • Then use a local `@INC` hook to load them on demand - • Minimizing the amount of temporary files - • First, try to load PerlIO::scalar and File::Temp - • Set up an END hook to unlink all temp files up to this point - • Load other bundled files, and look in the compressed PAR section - • This can be much easier with a pure-perl `inflate()`; patches welcome! ### Thank you (again)! - • Any questions, *please*? ## SEE ALSO PAR, pp, par.pl, parl ex::lib::zip, Acme::use::strict::with::pride App::Packer, Apache::PAR, CPANPLUS, Module::Install ## AUTHORS Audrey Tang \ You can write to the mailing list at \, or send an empty mail to \ to participate in the discussion. Archives of the mailing list are available at \ or \. Please submit bug reports to \. ## COPYRIGHT Copyright 2003, 2004, 2005, 2006 by Audrey Tang \. This document is free documentation; you can redistribute it and/or modify it under the same terms as Perl itself. See *LICENSE*.