Native crash reports missing symbol information (file name, line number, function name)

[READ] Step 1: Are you in the right place?

Issues filed here should be about bugs in the code in this repository.
If you have a general question, need help debugging, or fall into some
other category use one of these other channels:

  • For general technical questions, post a question on StackOverflow
    with the firebase tag.
  • For general Firebase discussion, use the firebase-talk
    google group.
  • For help troubleshooting your application that does not fall under one
    of the above categories, reach out to the personalized
    Firebase support channel.

[REQUIRED] Step 2: Describe your environment

  • Android Studio version: 4.0.1
  • Firebase Component: Crashlytics ndk
  • Component version: crashlytics-gradle 2.3.0, crashlytics-ndk 17.2.1

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

  • Build a release apk using crashlytics ndk and upload debugging symbols: ./gradlew clean assembleRelease uploadCrashlyticsSymbolFileRelease
  • Force a native crash in the app
  • Expected behavior: crash appears in Firebase Crashlytics console with file and line number information. For example:
    Crashed: Thread #1 SIGSEGV 0x0000000000000000
    0 libc.so (Missing)
    1 liblinphone.so linphonecore.c - Line 2020 linphone_core_set_adaptive_rate_algorithm + 2020
    2 base.odex (Missing)
    3 (Missing)
    
  • Actual behavior: crash appears in Firebase Crashlytics console with library *.so name and Missing. For example:
    Crashed: Thread #1 SIGSEGV 0x0000000000000018
    0 liblinphone.so (Missing)
    1 liblinphone.so (Missing)
    2 liblinphone.so (Missing)
    3 liblinphone.so (Missing)
    4 liblinphone.so (Missing)
    5 liblinphone.so (Missing)
    6 liblinphone.so (Missing)
    7 liblinphone.so (Missing)
    8 liblinphone.so (Missing)
    9 (Missing)
    ...
    63 libart.so (Missing)
    

Relevant Code:

Here’s a sample app: https://github.com/libon/CrashObfuscationTest

It doesn’t include native code, but it depends on a native library, linphone.
The sample project has a screen with a button, which when pressed, forces a crash in linphone by calling a method with a null value which isn’t supported by linphone:

private fun forceCrash() {
    Factory.instance().setDebugMode(true, "Linphone")
    val core = Factory.instance().createCore(null, null, this.getApplicationContext())
    core.start()
    core.adaptiveRateAlgorithm = null
}

To upload the debugging symbols to crashlytics, it extracts them from the debug artifact provided by linphone and copies them to the relevant folders specified by strippedNativeLibsDir and unstrippedNativeLibsDir.

Note, the crashes appear correctly with the file names and line numbers if Android Gradle Plugin 3.6.3 is used instead of 4.0.1.

The problem also exists with Android Gradle Plugin 4.2.0-alpha10 (the latest version as of today)

4 thoughts on “Native crash reports missing symbol information (file name, line number, function name)

  1. I can confirm that it works on my side too with:

    • com.google.firebase:firebase-crashlytics-ndk:17.3.0
    • com.google.firebase:firebase-crashlytics-gradle:2.4.1

    Using com.android.tools.build:gradle:4.1.0 and gradle wrapper 6.5

  2. Looks like this is fixed with:

    * `com.google.firebase:firebase-crashlytics-ndk:17.3.0`
    
    * `com.google.firebase:firebase-crashlytics-gradle:2.4.1`
    

    I checked this using com.android.tools.build:gradle:4.1.1 and gradle 6.7.1

    I’ve just tried using the same configuration and it still doesn’t work.

    Running generateCrashlyticsSymbolFileProdRelease and uploadCrashlyticsSymbolFileProdRelease scripts respectively, .cSYM files are generated and uploaded correctly to the firebasecrashlyticssymbols.googleapis.com backend but native stack traces for both old and new crashes are still missing.

    My project tree is structured like detailed here: #1700 (comment)

    —— Dec 2020 update ——-

    Thanks to the Firebase support I finally got it back to work!
    In my case the issue has been solved by adding: -Wl, --build-id to my LDFLAGS and rebuild my native libs.

    I leave a quick recap here hoping it will help those who are facing a similar issue.

    Build setup

    • Toolchain: Android NDK r17c (standalone toolchain)
    • Compiler: GCC v4.9
    • Build tools: Autotools (Autoconf, Automake, Libtool, ecc.)

    Checklist

    • Be sure you’re passing the -Wl, --build-id to the linker when building your native libs
    • Check that both unstripped and stripped libs have the same Build ID by running the command:
      path/to/your-arch-toolchain-folder/bin/<arch-prefix>-readelf -n path/to/your_lib.so
    • Check you have properly enabled native symbols upload in your app build.gradle:
    android.buildTypes.all { buildType ->
        firebaseCrashlytics {
            nativeSymbolUploadEnabled true
            strippedNativeLibsDir "path/to/stripped-libs-folder"
            unstrippedNativeLibsDir "path/to/unstripped-libs-folder"
        }
    }
    
    • Run gradlew clean assemble<BuildVariant> generateCrashlyticsSymbolFile<BuildVariant> and when it’s done, check that the generated *.cSYM files into the folder app/build/crashlytics/<BuildVariant>/nativeSymbols have the proper build id within their names (e.g. libname-aarch64-<build-id>.cSYM).
    • Run gradlew uploadCrashlyticsSymbolFile<BuildVariant> to upload the native symbols to the Crashlytics backend.

    Now you should be able to see the stacktraces on the Firebase console when a native crash occurs.

  3. FYI, if you are using firebase-crashlytics-ndk 17.3.0+ and firebase-crashlytics-gradle 2.4.1+, you no longer need to specify the stripped libs directory.