OverlappingFileLockException crashes with up to date libraries versions

[JeremyR34]

• Android Studio version: 3.6.3
• Firebase Components & versions:
  com.google.firebase:firebase-core 17.4.0
  com.google.firebase:firebase-analytics 17.4.0
  com.google.firebase:firebase-messaging 20.1.6
  com.crashlytics.sdk.android:crashlytics 2.10.1
  com.google.firebase:firebase-dynamic-links 19.1.0
  com.google.firebase:firebase-database 19.3.0
  com.google.firebase:firebase-config 19.1.4

Describe the problem

After releasing a new version ni production with up to date Firebase libraries we have many new crashes (Google Play developer console & Firebase Crashlytics).

Steps to reproduce:

Up-to-date libraries versions.
Notes

• it seems to happen only on Android 7 & 8 devices.
• it may be related to this: Fixing the OverlappingFileLockException. #1234

Relevant Code:

Fatal Exception: java.nio.channels.OverlappingFileLockException
       at sun.nio.ch.SharedFileLockTable.checkList(SharedFileLockTable.java:255)
       at sun.nio.ch.SharedFileLockTable.add(SharedFileLockTable.java:152)
       at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1080)
       at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
       at com.google.firebase.installations.CrossProcessLock.acquire(CrossProcessLock.java:52)
       at com.google.firebase.installations.FirebaseInstallations.getPrefsWithGeneratedIdMultiProcessSafe(FirebaseInstallations.java:376)
       at com.google.firebase.installations.FirebaseInstallations.doNetworkCall(FirebaseInstallations.java:327)
       at com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0(FirebaseInstallations.java:323)
       at com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run(FirebaseInstallations.java:4)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
       at java.lang.Thread.run(Thread.java:764)

Another kind of new crashes we have that seems also related (FirebaseInstallations):

Fatal Exception: java.lang.Error: java.io.IOException: Bad file descriptor
       at sun.nio.ch.FileKey.create(FileKey.java:46)
       at sun.nio.ch.SharedFileLockTable.<init>(SharedFileLockTable.java:120)
       at sun.nio.ch.FileLockTable.newSharedFileLockTable(FileLockTable.java:47)
       at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:993)
       at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1015)
       at java.nio.channels.FileChannel.lock(FileChannel.java:875)
       at com.google.firebase.installations.CrossProcessLock.acquire(CrossProcessLock.java:52)
       at com.google.firebase.installations.FirebaseInstallations.getPrefsWithGeneratedIdMultiProcessSafe(FirebaseInstallations.java:376)
       at com.google.firebase.installations.FirebaseInstallations.doNetworkCall(FirebaseInstallations.java:327)
       at com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0(FirebaseInstallations.java:323)
       at com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run(FirebaseInstallations.java)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
       at java.lang.Thread.run(Thread.java:760)

Caused by java.io.IOException: Bad file descriptor
       at sun.nio.ch.FileKey.init(FileKey.java)
       at sun.nio.ch.FileKey.create(FileKey.java:44)
       at sun.nio.ch.SharedFileLockTable.<init>(SharedFileLockTable.java:120)
       at sun.nio.ch.FileLockTable.newSharedFileLockTable(FileLockTable.java:47)
       at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:993)
       at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1015)
       at java.nio.channels.FileChannel.lock(FileChannel.java:875)
       at com.google.firebase.installations.CrossProcessLock.acquire(CrossProcessLock.java:52)
       at com.google.firebase.installations.FirebaseInstallations.getPrefsWithGeneratedIdMultiProcessSafe(FirebaseInstallations.java:376)
       at com.google.firebase.installations.FirebaseInstallations.doNetworkCall(FirebaseInstallations.java:327)
       at com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0(FirebaseInstallations.java:323)
       at com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run(FirebaseInstallations.java)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
       at java.lang.Thread.run(Thread.java:760)

[google-oss-bot]

I found a few problems with this issue:

• I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
• This issue does not seem to follow the issue template. Make sure you provide all the required information.

[andirayo]

@JeremyR34 : Thank you for the bug report! It's late now in the Bay Area, but we will look into this tomorrow!

[andirayo]

@JeremyR34 :
Thank you for reporting this. We are looking into this.
You correctly identified FirebaseInstallations (firebase-installations) as the cause of these issues. fyi: FirebaseInstallations is an infrastructure service used by many Firebase services (e.g. FirebaseMessaging).

Correct me if I'm wrong: You report 3 crashes.

• [A] The first in your list is an OverlappingFileLockException.
• [B] The other two are java.io.IOException: Bad file descriptor

Case [A] indeed looks identical to an issue we have fixed in the past (as you pointed out yourself: #1234). Are you sure that this exception was correctly attributed to the version numbers you list in the description? Is it possible that the exceptions happened to older versions of your app?
fyi: We are first looking into case [B], the IOExceptions...

[andirayo]

@JeremyR34 :
We identified the issue as a java.lang.Error wrapping a possible IOException for Java versions 7 (>= v40) and Java 8.
Compare: https://stackoverflow.com/questions/35727180/

We will fix this issue by catching java.lang.Error when trying to acquire a CrossProcessLock.
Thanks for bringing this to our attention.
The fix will most likely be released with v16.3.1 of FirebaseInstallations (and FirebaseMessaging v20.1.8) on May 21, 2020.

[andirayo]

This issue should be fixed with #1521.
This update is scheduled to be released on May 21, 2020.

[JeremyR34]

Case [A] indeed looks identical to an issue we have fixed in the past (as you pointed out yourself: #1234). Are you sure that this exception was correctly attributed to the version numbers you list in the description? Is it possible that the exceptions happened to older versions of your app?

I'm sure of the versions, the issue starts happening on a new release rollout. That's why I was surprised to find an old fix related to this.

I reverted my libraries versions on a new update and the issue is gone.
FYI I use:
com.google.firebase:firebase-core 17.2.1
com.google.firebase:firebase-analytics 17.2.1
com.google.firebase:firebase-messaging 20.1.0
com.crashlytics.sdk.android:crashlytics 2.10.1
com.google.firebase:firebase-dynamic-links 19.0.0
com.google.firebase:firebase-database 19.2.0
com.google.firebase:firebase-config 19.0.4

Side note: you give me a release update, but how to check if a particular fix is included in a release. I wanted to check when and if the #1234 has been released but did not find this info. On the https://firebase.google.com/support/release-notes/android page there is only a resume but not full details.