From 20f2a97112fd7cbee8e5fa2b7c627ed229ca7460 Mon Sep 17 00:00:00 2001 From: Eric Timmons Date: Tue, 16 Mar 2021 09:47:06 -0400 Subject: [PATCH] Add README.static-executable --- README.static-executable | 250 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 250 insertions(+) create mode 100644 README.static-executable diff --git a/README.static-executable b/README.static-executable new file mode 100644 index 0000000000..ffd12f35fe --- /dev/null +++ b/README.static-executable @@ -0,0 +1,250 @@ +This branch of SBCL is maintained by Eric Timmons (@daewok) and contains a set +of patches necessary to build a completely static executable with SBCL. Such an +executable has all necessary foreign libraries statically linked into the +runtime and has no support for dynamic loading and unloading of +libraries. While the lack of dynamic loading support is certainly constraining, +the benefit of building an executable this way is it requires no libraries to +be installed by the user of the executable. This makes it ideal for archival +purposes, distributing executables to a non-technical audience, distributing an +executable where you must know the exact versions of foreign libraries used at +runtime, or distributing executables that Just Work^TM (like many executables +written in golang). + +While other solutions exist to statically link foreign libraries into the SBCL +runtime, to the best of my knowledge there has been no publicly advertised +method of building SBCL with libc statically linked. The lack of static linking +for libc means that the user of the executable must have a compatible libc +installed. Unfortunately, the most commonly used libc in the Linux world +(glibc) is frequently not backward compatible with itself. For evidence of +this, see the fact that SBCLs built on Debian Buster (like the official +releases since 1.5.6) do not run on Debian Stretch. + +Unfortunately, glibc doesn't even really support static linking at +all. Therefore, I recommend that static SBCL executables be built with musl +libc. Musl is designed with static linking in mind and is broadly compatible +with most libraries that don't do tricksy things with libc. And if you find a +library not compatible with musl libc, it seems most maintainers are welcoming +to patches that add support. + +Alpine Linux is a great OS for building statically linked executables as it +uses musl libc by default. I further recommend using Docker for building static +executables so that you don't need to maintain a separate Alpine install. Plus, +you can use the clfoundation/sbcl:alpine3.13 image as a starting point. + +THEORY + +The biggest issue with creating a static executable is ensuring that foreign +symbols are accessible from the Lisp core. In normal, dynamic use, SBCL uses +dlsym to look up the address of symbols and stores them in a vector in foreign +memory called the "linkage table". The lisp core then maintains a hash table +mapping foreign symbol names to their index in the linkage table. This is +called the linkage info. + +In a static executable, we cannot count on having a working dlsym, even if +libdl is linked into the runtime. When performing static linkage, musl libc +replaces all libdl functions with stubs that simply return errors. Therefore, +we have to use the system linker to resolve the references for us. But in order +to have the linker do that for us, we need to know at link time which foreign +symbols our lisp code will want to use! + +EXTRACTING LINKAGE INFO + +There are two approaches described below to generate a static executable. Both +of them require a file describing the desired linkage info. While you could +generate this by hand, it is easiest to extract it from a core. + +In order to extract the linkage info from a running core, use +tools-for-build/dump-linkage-info.lisp. After loading that into the core, +evaluate (sb-dump-linkage-info:dump-to-file #p"/path/to/output.sexp"). It also +takes an keyword argument :make-undefined, a list of symbol names to make +undefined in the output. This is useful for approach two below. + +The sexp written to the output file is a single list of lists. Each sublist has +three elements. The first is a string naming the symbol. The second is T if the +symbol is entered into the linkage info as data (it is a foreign variable) and +NIL otherwise (it is a foreign function). The third is T if the symbol is +undefined and NIL otherwise. It is critical that undefined symbols be +maintained for approach one below. + +The following two sections describe two approaches on how to generate a static +executable, step-by-step. The demo static executable contains the sb-gmp +contrib and runs its test quite when executed. It requires that the static +libraries for libgmp and libz are installed on your system. There is some +weirdness with how the tests are loaded. This is because the tests do not seem +to work after being dumped: I have not yet figured out why this is. + +BUILDING A STATIC EXECUTABLE - APPROACH ONE + +This approach to building a static executable is preferred if you're you want +to minimize the amount of time compiling C and Lisp code. It takes advantage of +the fact that musl inserts stub functionality for libdl such that it can still +be linked against. + +The general process for this approach is: + +0. Build SBCL with the :sb-prelink-linkage-table feature (:sb-linkable-runtime +is also strongly recommended). + +1. Build a core containing the lisp code you want to package in the static +executable. + +2. Dump the linkage info to a file. + +3. Dump the core to a file (with save-lisp-and-die). + +4. Generate a C file that contains the info needed to build the linkage table. + +5. Relink the runtime. This time statically *and* with the object file +generated from the C file in step 4. + +6. Load the saved core into the new static runtime, dumping again with +:executable t if desired. + +Some notes about this approach: + ++ The build IDs of the dynamic runtime (used to generate the core in step 1) +and the static runtime *must* match. The easiest way to achieve this is to +install SBCL with the feature :sb-linkable-runtime. This installs sbcl.o (the +SBCL runtime in a ingle object file) along with everything else. + ++ No modifications must be made to the linkage info file generated in step 2 +and no symbols can be filtered out of it. + +Here is a step-by-step procedure to build the demo static executable using this +approach. + +Step 0: + + sh make.sh --fancy --with-sb-linkable-runtime --with-sb-prelink-linkage-table + sh install.sh + +Steps 1-3: + + sbcl --non-interactive \ + --no-sysinit --no-userinit \ + --eval '(require :uiop)' \ + --eval '(require :sb-gmp)' \ + --eval '(require :sb-rt)' \ + --eval '(defvar *sb-gmp-tests* (uiop:read-file-string "contrib/sb-gmp/tests.lisp"))' \ + --load tools-for-build/dump-linkage-info.lisp \ + --eval '(sb-dump-linkage-info:dump-to-file "/tmp/linkage-info.sexp")' \ + --eval '(sb-ext:save-lisp-and-die "/tmp/sb-gmp-tester.core")' + +Step 4: + + sbcl --no-sysinit --no-userinit \ + --script tools-for-build/create-linkage-table-prelink-info-override.lisp \ + /tmp/linkage-info.sexp \ + /tmp/linkage-table-prelink-info-override.c + +Step 5: + + # Get all the variables SBCL used to build defined in the current environment. + while read l; do + eval "${l%%=*}=\"${l#*=}\""; + done < /usr/local/lib/sbcl/sbcl.mk + + $CC $CFLAGS -Wno-builtin-declaration-mismatch -o /tmp/linkage-table-prelink-info-override.o -c /tmp/linkage-table-prelink-info-override.c + $CC -no-pie -static $LINKFLAGS -o /tmp/static-sbcl /usr/local/lib/sbcl/$LIBSBCL /tmp/linkage-table-prelink-info-override.o -lgmp $LIBS + +Step 6: + + /tmp/static-sbcl --core /tmp/sb-gmp-tester.core \ + --non-interactive \ + --no-sysinit --no-userinit \ + --eval '(sb-ext:save-lisp-and-die "/tmp/sb-gmp-tester" :executable t :toplevel (lambda () (uiop:load-from-string *sb-gmp-tests*) (sb-rt:do-tests) (exit)) :compression t)' + + +Look at the dumped executable. You should see that it is a static executable. + + ldd /tmp/sb-gmp-tester + +Test that it works! + + /tmp/sb-gmp-tester + +BUILDING A STATIC EXECUTABLE - APPROACH TWO + +This approach results in an executable that is not linked with libdl at +all. This makes it a little bit more "pure" than than the previous approach, +but that comes at the cost of needing to fully recompile both the runtime and +core after the necessary foreign symbols are determined. + +The general process for this approach is: + +1. Build a core containing the lisp code you want to package in the static +executable. + +2. Dump the linkage info to a file. + +3. Recompile SBCL, passing in the linkage info during build. + +4. Rebuild your core with the new runtime and corresponding core. + +5. Dump with :executable t. + +Some notes about this approach: + ++ The libdl symbols must be stripped out of the linkage info file generated in +step 2. The easiest way to do this is pass sb-dump-linkage-info:*libdl-symbols* +as the :make-undefined argument to dump-to-file. + ++ Further modifications can be made to the linkage info file generated in step +2. You can reorder the symbols at will. You can add new symbols. You probably +don't want to remove any (besides libdl functions). + +Steps 1-2: + + sh run-sbcl.sh --non-interactive \ + --no-sysinit --no-userinit \ + --eval '(require :uiop)' \ + --eval '(require :sb-gmp)' \ + --eval '(require :sb-rt)' \ + --eval '(defvar *sb-gmp-tests* (uiop:read-file-string "contrib/sb-gmp/tests.lisp"))' \ + --load tools-for-build/dump-linkage-info.lisp \ + --eval '(sb-dump-linkage-info:dump-to-file "/tmp/linkage-info.sexp" :remove-symbols sb-dump-linkage-info:*libdl-symbols*)' + +Step 3: + + LDLIBS="-lgmp" LINKFLAGS="-no-pie -static" IGNORE_CONTRIB_FAILURES="yes" sh make.sh --extra-linkage-table-entries=/tmp/linkage-info.sexp --without-os-provides-dlopen --without-os-provides-dladdr --fancy + +Steps 4-5: + + sh run-sbcl.sh --non-interactive \ + --no-sysinit --no-userinit \ + --eval '(require :uiop)' \ + --eval '(require :sb-gmp)' \ + --eval '(require :sb-rt)' \ + --eval '(defvar *sb-gmp-tests* (uiop:read-file-string "contrib/sb-gmp/tests.lisp"))' \ + --eval '(sb-ext:save-lisp-and-die "/tmp/sb-gmp-tester" :executable t :toplevel (lambda () (uiop:load-from-string *sb-gmp-tests*) (sb-rt:do-tests) (exit)) :compression t)' + +Look at the dumped executable. You should see that it is a static executable. + + ldd /tmp/sb-gmp-tester + +Test that it works! + + /tmp/sb-gmp-tester + +BUILDING A STATIC EXECUTABLE WITH DOCKER + +See the Dockerfile at tools-for-build/Dockerfile.static-executable-example for +an example of how to build the demo executable using Docker and approach +one. The benefit of Docker is that it is a cheap way to build with musl libc +even if you use glibc locally. + +The following commands will build the demo executable inside docker and extract +it from the image, placing it at /tmp/sb-gmp-tester on your local file +system. The following commands also try to avoid polluting your Docker +namespace by not tagging the image or naming the container used to extract the +executable. + + IMAGE_ID_FILE="$(mktemp)" + CONTAINER_ID_FILE="$(mktemp)" + rm "$CONTAINER_ID_FILE" + docker build --iidfile "$IMAGE_ID_FILE" -f tools-for-build/Dockerfile.static-executable-example . + docker create --cidfile "$CONTAINER_ID_FILE" "$(cat "$IMAGE_ID_FILE")" + docker cp "$(cat "$CONTAINER_ID_FILE"):/tmp/sb-gmp-tester" /tmp/sb-gmp-tester + docker rm "$(cat "$CONTAINER_ID_FILE")" + rm "$IMAGE_ID_FILE" + rm "$CONTAINER_ID_FILE"