Posted in Uncategorized

dart2native requires libc?

   January 7, 2021  7 Comments on dart2native requires libc?

dart2native seems to require the presence of several libraries, including libc, as referenced in this issue: #39253. Dart2native thus doesn’t seem to provide the promise of a truly self contained binary as is stated in its’ documentation.

While dart2native offers big advantages (THANK YOU Dart Team!), this prevents dart2native executables from being run in standard alpine linux containers (afaik, a key use case for dart2native).

It would be helpful if this were clearer in the documentation.

Author: Fantashit

7 thoughts on “dart2native requires libc?”

  1. The issue I am raising is not the size of the Dart executable, but rather the documentation describing the executable as stand alone.

    When you containerize the application, the tiny Alpine Linux container that will happily run a go binary fails with the dart executable because it lacks the Dart executable’s dependencies. If you use one of Google’s Dart docker containers, the container + executable clock in at ~800 MB. So you have to roll your own image or use a third party image that has manually installed libc on Alpine to get the container + executable size back down to a reasonable size. This is not a big deal and exposing dart2native is a big step forward. I am just suggesting that being clearer about what dart2native outputs will save developer time and help move the ecosystem forward, faster.

  2. /cc @mit-mit

    We have been talking about providing a base docker image containing the minimal requirements for dart2native-created executables (which would include the needed shared libraries)

    Something like this:

    # Make a self-contained executable out of the application.
FROM google/dart AS dart-runtime
WORKDIR /app
ADD pubspec.* /app/
RUN pub get
ADD bin /app/bin/
ADD lib /app/lib/
RUN dart2native /app/bin/server.dart

# Build the bare minimum image.
FROM scratch
COPY --from=dart-runtime /app/bin/server.exe /app/bin/server.exe
COPY --from=dart-runtime /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libpthread.so.0
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so.2
ENTRYPOINT ["/app/bin/server.exe"]

  4. @a14n

    The Dart VM uses normal POSIX apis for network functions. That includes DNS lookups.

    The functions are implemented by the standard shared libraries. Those need a few configuration files (e.g. /etc/resolv.conf, /etc/nsswitch.conf) which can be added to the minimal image the same way as the so files. It’s probably not very hard to identify which ones exactly.

    Once we make a base docker image, these will of course be included.

  5. The more precise runtime dependencies of libc for DNS lookups are:

    FROM scratch

# For name-service order configuration, predefined hostnames like "localhost", dns server IPs
COPY --from=dart-runtime /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=dart-runtime /etc/hosts /etc/hosts
COPY --from=dart-runtime /etc/resolv.conf /etc/resolv.conf

# For performing the actual DNS queries to DNS servers
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libnss_dns.so.2 /lib/x86_64-linux-gnu/libnss_dns.so.2
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libresolv.so.2 /lib/x86_64-linux-gnu/libresolv.so.2

...

    (in addition to Dockerfile mentioned in #39296 (comment))

  6. Dart 2.8 additionally requires:

    COPY --from=dart-runtime /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/librt.so.1

    This works for me:

    FROM google/dart:2.8 AS dart-runtime

WORKDIR /app

ADD pubspec.* ./
RUN pub get
ADD bin bin
ADD lib lib
RUN pub get --offline
RUN dart2native /app/bin/main.dart -o /app/bin/main

FROM scratch
COPY --from=dart-runtime /app/bin/main /app/bin/main
COPY --from=dart-runtime /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so.2
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libpthread.so.0
COPY --from=dart-runtime /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/librt.so.1

# For name-service order configuration, predefined hostnames like "localhost", dns server IPs
COPY --from=dart-runtime /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=dart-runtime /etc/hosts /etc/hosts
COPY --from=dart-runtime /etc/resolv.conf /etc/resolv.conf

# For performing the actual DNS queries to DNS servers
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libnss_dns.so.2 /lib/x86_64-linux-gnu/libnss_dns.so.2
COPY --from=dart-runtime /lib/x86_64-linux-gnu/libresolv.so.2 /lib/x86_64-linux-gnu/libresolv.so.2

ENTRYPOINT ["/app/bin/main"]

EXPOSE 8080

  7. I’ve consolidated everything needed to create the leanest possible images based on scratch into this repo, and added documentation, tests, and examples for both JIT- and AOT-compiled servers.

    The image is published on Docker Hub at subfuzion/dart:slim. There’s also a Medium blog post that gives a bit more detail and why (lean) size matters when it comes to microservice/serverless images.

    The subfuzion/dart:slim image weighs in at a super slim 4.09 MB (2.11 MB compressed on Docker Hub).

    Image test

    $ cd test
$ ./dartslim_test.sh
[+] Building 9.7s (15/15) FINISHED

00:00 +0: test/image_test.dart: Is server reachable? ping-test
00:00 +1: test/image_test.dart: Server DNS client tests remote-ping-test
00:00 +2: All tests passed!

    Timing tests

    $ time docker pull subfuzion/dart:slim
slim: Pulling from subfuzion/dart
Digest: sha256:5bf1b8083f40b83c437301da991dbca8901ae48c525f56ccda05669040b93e1a
Status: Downloaded newer image for subfuzion/dart:slim
docker.io/subfuzion/dart:slim

real    0m1.656s
user    0m0.156s
sys     0m0.082s

$ docker image ls subfuzion/dart:slim
REPOSITORY       TAG       IMAGE ID       CREATED        SIZE
subfuzion/dart   slim      d92db830c85b   12 hours ago   4.09MB
    $ cd ./examples/dart-aot
$ docker build -t server-aot .
[+] Building 11.5s (15/15) FINISHED

$ time docker run -it -p 8080:8080 --name server-aot-test server-aot --quit
Starting server
Serving at http://0.0.0.0:8080
time elapsed: 1 ms

real    0m0.643s
user    0m0.152s
sys     0m0.071s
    $ cd ./examples/dart-vm
$ docker build -t server-vm .
[+] Building 4.0s (16/16) FINISHED

$ time docker run -it -p 8080:8080 --name server-vm-test server-vm --quit
Starting server
Serving at http://0.0.0.0:8080
time elapsed: 91 ms

real    0m0.777s
user    0m0.144s
sys     0m0.070s
    Compiler Build image Container launch until server listening main() until server listening
    AOT 11.5s 0.643s 1ms
    JIT 4.0s 0.777s 91ms

    Test machine for timing tests

    • iMac Pro (2017), 3.2 GHz 8-core Intel Xeon W, 64 GB 2666 MHz DDR4
    • Docker engine 20.10.0-beta1
    • Dart SDK version: 2.10.4 (stable) on “linux_x64”

Comments are closed.