dart2native requires libc?

dart2native seems to require the presence of several libraries, including libc, as referenced in this issue: #39253. Dart2native thus doesn’t seem to provide the promise of a truly self contained binary as is stated in its’ documentation.

While dart2native offers big advantages (THANK YOU Dart Team!), this prevents dart2native executables from being run in standard alpine linux containers (afaik, a key use case for dart2native).

It would be helpful if this were clearer in the documentation.

Author: Fantashit

7 thoughts on “dart2native requires libc?

  1. The issue I am raising is not the size of the Dart executable, but rather the documentation describing the executable as stand alone.

    When you containerize the application, the tiny Alpine Linux container that will happily run a go binary fails with the dart executable because it lacks the Dart executable’s dependencies. If you use one of Google’s Dart docker containers, the container + executable clock in at ~800 MB. So you have to roll your own image or use a third party image that has manually installed libc on Alpine to get the container + executable size back down to a reasonable size. This is not a big deal and exposing dart2native is a big step forward. I am just suggesting that being clearer about what dart2native outputs will save developer time and help move the ecosystem forward, faster.

  2. /cc @mit-mit

    We have been talking about providing a base docker image containing the minimal requirements for dart2native-created executables (which would include the needed shared libraries)

    Something like this:

    # Make a self-contained executable out of the application.
    FROM google/dart AS dart-runtime
    WORKDIR /app
    ADD pubspec.* /app/
    RUN pub get
    ADD bin /app/bin/
    ADD lib /app/lib/
    RUN dart2native /app/bin/server.dart
    
    # Build the bare minimum image.
    FROM scratch
    COPY --from=dart-runtime /app/bin/server.exe /app/bin/server.exe
    COPY --from=dart-runtime /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libpthread.so.0
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so.2
    ENTRYPOINT ["/app/bin/server.exe"]
  3. @a14n

    The Dart VM uses normal POSIX apis for network functions. That includes DNS lookups.

    The functions are implemented by the standard shared libraries. Those need a few configuration files (e.g. /etc/resolv.conf, /etc/nsswitch.conf) which can be added to the minimal image the same way as the so files. It’s probably not very hard to identify which ones exactly.

    Once we make a base docker image, these will of course be included.

  4. The more precise runtime dependencies of libc for DNS lookups are:

    FROM scratch
    
    # For name-service order configuration, predefined hostnames like "localhost", dns server IPs
    COPY --from=dart-runtime /etc/nsswitch.conf /etc/nsswitch.conf
    COPY --from=dart-runtime /etc/hosts /etc/hosts
    COPY --from=dart-runtime /etc/resolv.conf /etc/resolv.conf
    
    # For performing the actual DNS queries to DNS servers
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libnss_dns.so.2 /lib/x86_64-linux-gnu/libnss_dns.so.2
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libresolv.so.2 /lib/x86_64-linux-gnu/libresolv.so.2
    
    ...

    (in addition to Dockerfile mentioned in #39296 (comment))

  5. Dart 2.8 additionally requires:

    COPY --from=dart-runtime /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/librt.so.1

    This works for me:

    FROM google/dart:2.8 AS dart-runtime
    
    WORKDIR /app
    
    ADD pubspec.* ./
    RUN pub get
    ADD bin bin
    ADD lib lib
    RUN pub get --offline
    RUN dart2native /app/bin/main.dart -o /app/bin/main
    
    FROM scratch
    COPY --from=dart-runtime /app/bin/main /app/bin/main
    COPY --from=dart-runtime /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so.2
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libpthread.so.0
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/librt.so.1
    
    # For name-service order configuration, predefined hostnames like "localhost", dns server IPs
    COPY --from=dart-runtime /etc/nsswitch.conf /etc/nsswitch.conf
    COPY --from=dart-runtime /etc/hosts /etc/hosts
    COPY --from=dart-runtime /etc/resolv.conf /etc/resolv.conf
    
    # For performing the actual DNS queries to DNS servers
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libnss_dns.so.2 /lib/x86_64-linux-gnu/libnss_dns.so.2
    COPY --from=dart-runtime /lib/x86_64-linux-gnu/libresolv.so.2 /lib/x86_64-linux-gnu/libresolv.so.2
    
    ENTRYPOINT ["/app/bin/main"]
    
    EXPOSE 8080
  6. I’ve consolidated everything needed to create the leanest possible images based on scratch into this repo, and added documentation, tests, and examples for both JIT- and AOT-compiled servers.

    The image is published on Docker Hub at subfuzion/dart:slim. There’s also a Medium blog post that gives a bit more detail and why (lean) size matters when it comes to microservice/serverless images.

    The subfuzion/dart:slim image weighs in at a super slim 4.09 MB (2.11 MB compressed on Docker Hub).

    Image test

    $ cd test
    $ ./dartslim_test.sh
    [+] Building 9.7s (15/15) FINISHED
    
    00:00 +0: test/image_test.dart: Is server reachable? ping-test
    00:00 +1: test/image_test.dart: Server DNS client tests remote-ping-test
    00:00 +2: All tests passed!

    Timing tests

    $ time docker pull subfuzion/dart:slim
    slim: Pulling from subfuzion/dart
    Digest: sha256:5bf1b8083f40b83c437301da991dbca8901ae48c525f56ccda05669040b93e1a
    Status: Downloaded newer image for subfuzion/dart:slim
    docker.io/subfuzion/dart:slim
    
    real    0m1.656s
    user    0m0.156s
    sys     0m0.082s
    
    $ docker image ls subfuzion/dart:slim
    REPOSITORY       TAG       IMAGE ID       CREATED        SIZE
    subfuzion/dart   slim      d92db830c85b   12 hours ago   4.09MB
    $ cd ./examples/dart-aot
    $ docker build -t server-aot .
    [+] Building 11.5s (15/15) FINISHED
    
    $ time docker run -it -p 8080:8080 --name server-aot-test server-aot --quit
    Starting server
    Serving at http://0.0.0.0:8080
    time elapsed: 1 ms
    
    real    0m0.643s
    user    0m0.152s
    sys     0m0.071s
    $ cd ./examples/dart-vm
    $ docker build -t server-vm .
    [+] Building 4.0s (16/16) FINISHED
    
    $ time docker run -it -p 8080:8080 --name server-vm-test server-vm --quit
    Starting server
    Serving at http://0.0.0.0:8080
    time elapsed: 91 ms
    
    real    0m0.777s
    user    0m0.144s
    sys     0m0.070s
    Compiler Build image Container launch until server listening main() until server listening
    AOT 11.5s 0.643s 1ms
    JIT 4.0s 0.777s 91ms

    Test machine for timing tests

    • iMac Pro (2017), 3.2 GHz 8-core Intel Xeon W, 64 GB 2666 MHz DDR4
    • Docker engine 20.10.0-beta1
    • Dart SDK version: 2.10.4 (stable) on “linux_x64”

Comments are closed.