F-Droid vulnerability allows bypassing certificate pinning

@sky9000

It is easy to take shots at FDroid but they are the ones attempting heaving lifting with chain of trust.

This is complete nonsense. They're not doing any heavy lifting and are making the situation much worse. They've introduced major usability issues and confusion by reusing app ids with their own signing keys. This does not provide a chain of trust but rather breaks it since the apps are not signed by the developers and securely provided to you but rather built on insecure server infrastructure but quite clearly untrustworthy people who do not care about security in the slightest but rather only going through the motions of pretending they do.

The stark truth is that people learn how to install, GOS, the most secure mobile OS in the world and then hit a wall when starting to install apps.

The main way to get Android apps is the Play Store and it works fine. A bunch of other options are available and it's not a GrapheneOS issue that those are quite fragmented.

The entire time it has been Github Dev crowd poking at FDroid and FDroid supporters poking back.

F-Droid is incredibly poorly designed and maintained. It's a complete security disaster. It's a usability disaster. It has never been a good option for obtaining apps and was always preventing progress in the space due to people focusing on it and not making something much better.

F-Droid automatically downloads and builds code, so despite their false marketing it does not protect users from the developers in any real way. It often adds substantial delays for updates including security patches. Their builds have often rolled back dependency versions, signing scheme version and the SDK to much older versions with security flaws. They've consistently introduced security flaws to the apps. It adds additional trusted parties who have demonstrated a clear lack of trustworthiness including several of their core team members spreading fabricated stories and repeatedly engaging in cover ups. In what sense is this a safe way to get apps?

It makes no sense that this has gone on this long. Much appreciated that the GrapheneOS team has endorsed Accrescent. It is just frustrating that this kind of effort took this long and is moving so slowly.

F-Droid has massively hindered progress because it got the mind share of a lot of the open source community before people started realizing how awful it is and that it's not ever going to become a great platform. The people developing it are incapable of making good software, do not like the overall platform, do not understand it and don't truly want it to succeed. The main developer has repeatedly made statements against app sandboxing and other basic tenets of privacy and security.

As near as I can tell, Accrescent, Appverifier are nearly solo projects just like Divest which has ended. Are they sustainable?

What makes you assume F-Droid is sustainable? F-Droid has severe security flaws throughout their app, repository, infrastructure for building/signing apps and servers. These have been repeatedly pointed out and the remaining team has focused on covering up issues, misdirecting from it and making excuses for it. F-Droid also causes major usability issues with Android profiles due to them reusing app ids with different signing keys, which they refuse to even acknowledge is against the basic best practices for Android development. They also still publish an outdated F-Droid version on their site as the main download option, causing more usability issues and also not giving people the latest security fixes from the start.

Great that a credible researcher has found an issue. However, will chain of trust be solved any time soon or will there just be endless poking at each other?

If you continue to misrepresent, downplay and deny the many real issues about F-Droid, you'll no longer be participating in our community.

Could more people with skills actually help out?

F-Droid is the main barrier to getting well designed, secure distribution of open source and other apps fully implemented and widely adopted. It's a huge barrier to it. F-Droid is actively negligent and is putting users at risk. They've taken many anti-security positions and are against basic tenets of security. Their team has consistently attacked GrapheneOS. Helping them deters progress rather than enabling it. You portray this as if we're trying to accomplish the same thing and should be working with people involved in harassment towards our team.

I have a question that is on-topic. Can someone explain what certificate pinning it is that can be bypassed? What is the consequences of this vulnerability for the end-user?

I suppose F-Droid repository looks pretty much like on any Linux distribution, that is, the repository data your device retrieves is signed with a key controlled by F-Droid, and then each app mentioned there has a SHA256 of the APK file. My understanding was that F-Droid always retrieves source code releases for apps, and build apps themselves. Is it the source code archives that is signed in a way that can be bypassed? Or is there some other component here that works in a way not typical of how Linux app repositories are maintained?

I couldn't really understand from the vulnerability description or the linked mailing list posts, so would appreciate if someone would explain.

sky9000 Great that a credible researcher has found an issue. However, will chain of trust be solved any time soon or will there just be endless poking at each other?

Could more people with skills actually help out?

My impression is that there is absolutely zero agreement about how even an ideal app repository for GrapheneOS would look like. There just seem to be so many competing security goals, with no proposed solution solving them all.

I guess the best bet to replace the use-case of F-Droid would be for a trusted party to start downloading and compiling the source code for a very large number of apps, in a secure and modern environment, and then distribute it all like a traditional signed Linux app repository. And making local patches to the source code for an app whenever it has a too low target SDK, has trackers or ads, or other security or privacy issues. This would be close to how F-Droid is run, and would give the number of apps needed for critical mass without relying on each app developer themself submitting their app to a yet mostly unknown app repository, but could take security seriously too, in how we who use F-Droid today would want security to be.

Problem is, the amount of work to maintain an app repository, let alone one that has most modern open source apps, is a gigantic undertaking, not something a single person can really do. But if someone where to do it, I think most or all GrapheneOS users using F-Droid today as their primary or only app store would switch to it pretty much immediately.

ryrona It looks like there are multiple issues. One of the issues looks like not properly checking signatures for imported APKs where malicious APKs can be imported and end up included in the signed metadata. As far as we're concerned, the existing major design flaws and lack of trustworthiness of people with access to their critical infrastructure including the badly maintained build / signing infrastructure are a bigger issue. The status quo of what's known about F-Droid's processes, infrastructure and team is already very scary.

F-Droid doesn't provide most modern open source apps in their official repository. A huge portion of the apps it does have are completely abandoned or very poorly maintained. A huge portion fail to meet important privacy and security standards which even the very lenient Play Store implements such as a target API level requirement. It's a poor way of showing off the open source app ecosystem for Android. It provides most apps which come from particular parts of the open source community, but with delayed updates and often problematic changes including introducing security vulnerabilities via downgraded dependencies.

It's their own builds on their own infrastructure, almost entirely with their own signing keys except the small portion they use their problematic reproducible builds system. That system locks in using a problematic legacy build environment and will result in even more significantly delayed updates due to how they set it up. They make some minor, undocumented and often quite problematic changes to strip out functionality depending on Play services, etc. They certainly don't systemically update library dependencies or anything like that. They often actually downgrade dependencies. They've consistently introduced serious vulnerabilities to apps through using legacy tools, SDK, libraries, etc.

Simply building a similar kind of system following security best practices, not reusing app ids, not using a bunch of legacy software but still redoing the builds like a traditional Linux distribution be far better. There are massive problems with both their approach and implementation.

We don't consider F-Droid to be a trustworthy source of apps at all. They consistently disregard security and actively take anti-security positions. They don't care about or follow best practices for Android or security. They put on a show with the near useless shallow audits. They fundamentally don't understand or care about it.. They also consistently engage in cover ups, misdirection and misinformation as tactics to deal with security vulnerabilities and deep security design flaws throughout the software.

Several F-Droid core developers supported Copperhead's 2018 takeover attempt on GrapheneOS and then supported Copperhead's attacks over the following years. They moved on to engaging in spreading fabricated stories about our team themselves with the goal of directing harassment towards us. They supported extreme harassment from others. Nothing was done about their project members baselessly calling one of our team members insane, schizophrenic, delusional, etc. across a bunch of chat rooms and elsewhere. They made up fabricated stories about that too. It reflects on the lack of character of the overall F-Droid team. Why should GrapheneOS users trust people engaging in such underhanded tactics with the critical role of building and signing most of their apps? It's never something our community is going to broadly support even aside from the technical issues. Technical issues could theoretically be fixed (although there's no sign of much changing) but it's still not going to be run by trustworthy people.

ryrona do you agree that the "xz" situation can always happen if you blindly trust app devs without reproducible builds enforced and checked for each update?

Having a security model like Accrescent, where app devs do whatever they want and the chain of trust uncontrolled starts on their desk, is problematic.

So I think that both approaches have issues. A recent, patched and secure build environment with focus on recent Android versions, is a must of course.

DeletedUser87

The solution as others keep mentioning seems simple. Don't use F-Droid or any client for it.

Use Obtainium and GrapheneOS recommended App Store apps: Accrescent, Obtainium with AppVerifier to check the hashes/checksums of dowloaded APK files. Google Play store app to access stuff with a Google account. And the pre-installed App Store that comes with GOS.

Aurora Store is not recommended.

ryrona couldn't have phrased it better

missing-root The xz backdoor was almost entirely in the Git source repository and could have been completely done there. It had part of it included in the source tarball at the end without it being reproducible from the source repository. However, they could have put it into the source repository and it almost certainly wouldn't have been spotted based on that, especially considering most of it was already there. They took a bigger risk of discovery by modifying the source tarball the way they did. It appears it happened the way it did because previously they didn't have the ability to make the source tarballs themselves so they were doing it at a source code level. They switched to an easier but more easily discovered approach once they got control over releases. Look at how much was included in the source repository and how they did the final part in the source tarball, and you can see they clearly could have done the rest in the source repository. They already had nearly all of the backdoor as public source code without discovery, just not the final additions to the test cases.

F-Droid blindly fetches and builds code from app developers. How do reproducible builds for a tiny portion of the apps help anything when the source code isn't actually being checked? It has no practical benefits aside from marketing. GrapheneOS has reproducible builds and someone is now actually checking it for each release, but how does that actually benefit anyone in practice?

ryrona

Where and why would they import APKs? Aren't they building everything from source? If they are importing already pre-compiled APKs without verifying them through reproducing the builds, that is a pretty big issue in my threat model, even if there are no certificate pinning bypass.

They almost entirely build and sign the apps themselves. For a small portion of the apps, they use the developer signed APKs if they match what they build. This can go very wrong where updates get indefinitely delayed especially with them using a problematic and outdated build environment. There are prebuilt libraries either way. They do not review anything when fetching and building the code but rather do automated scanning including with antivirus. Fetching and building the code is automated. Signing is triggered manually for batches of code. It's almost entirely automated on a server other than signing, and they might automate signing too. The signing not being automated doesn't mean there's some kind of review process before it's signed.

Because, unfortunately, there are no alternatives.

There are a bunch of alternatives. F-Droid is hardly the only way to obtain builds of open source apps.

App developers seldom has the expertise and capabilities to build their apps in secure environments, they just build them on their regular computer. Most of them probably just store the signing key there too, not protecting it using dedicated hardware at all. Who knows how many more than the app developers can modify or sign the releases. Individual app developers may also easily comply to sign malicious builds if persuaded to do so by an authority. I think it is fair to say it is mostly only developers of privacy apps, and cryptocurrency apps, that gets it right, and even then they have been hit by supply chain attacks repeatedly, while traditional Linux repositories have been able to withstand that better, even if only because of being somewhat slower moving. The xz backdoor failed, because they targeted those slow moving Linux distributions, but many have lost all their cryptocurrencies because of the wallet app they were using upgraded to a malicious dependency, without the developer knowing the dependency was compromised, and it was all deployed to end-users within days.

The xz backdoor did not fail. It was shipped and then discovered by someone in production due to a completely unnecessary flaw in the backdoor. It shipped in the more bleeding edge variants of Debian and RHEL. It was only caught because it had a major performance issue they should have avoided. They also unnecessarily modified the release tarball in a way that wasn't why it was detected but could have been detected. They made nearly all their changes in the source repository but then unnecessarily bundled the final bits of the obfuscated payload in the tarball. It's a strange decision since it was well obfuscated as compressed data in tests. It looks a lot like they spent a lot of time doing things slowly and carefully but then rushed to finish it and blew their cover in doing that. It's entirely possible there were a bunch of servers compromised due to it based on crawling all the servers with sshd across the whole IPv4 space looking for it.

The ones who maintain an app repository is expected to be able to build apps in a secure environment, to maintain signing keys using dedicated hardware in a secure way, and to be able to act on reports of backdoors and security vulnerabilities even when the app developer isn't.

F-Droid is building on a server, not local infrastructure. That's much worse than a developer building on their workstation especially since it would result in the entire repository being compromised not only a single app. A developer's workstation being compromised means a backdoor can be put directly in the sources, and as the xz situation showed, the backdoor being present in the sources doesn't mean it will be found. The xz backdoor was largely present in the source repository and fully present in the source tarball. It was a source code level backdoor, not something included in binary releases from the project. You portray it as if that's not the case but it was, and it did ship in distributions including Debian. It didn't make it all the way to Debian stable because of how slowly it updates. If they had done a better job and not introduced a huge performance problem, it would not have been discovered when it was. It's unclear when it would have been discovered.

Yes, I have zero trust in Accrescent. They take the pre-compiled APKs from the developers, do not perform any verification like reproducible builds or anything, and have despite their very small number of apps already accepted one with a bundled affiliate link with a tracker, and seem to not care about that. (Organic Maps, by the way, F-Droid removes that tracker.)

That's not a tracker.

That would go from having minimal peer review to having no peer review at all. Absolutely not a suitable choice in my threat model. I need that peer review, my security very much relies on that.

F-Droid does not review the code. They automatically fetch and build it. They scan for non-open-source code and run it through antivirus. That's what they actually do, not what people believe they do.

Unless we have an alternative by then, it would probably be the point at which I have to stop using phones for security and privacy sensitive things.

You prefer using a system where a backdoor was shipped to users and only discovered because it had awful performance because you wrongly attribute the discovery of that backdoor to it. It doesn't make much sense. The reality is that the system you're promoting completely failed at stopping this and adds a huge number of trusted parties. Extremely delayed updates leaving people vulnerable to many unpatched security bugs is what protected people from getting it, not any kind of review or safety provided by thousands of packagers acting as additional trusted parties in Debian.

The xz situation in fact demonstrated that this packaging system even with reproducible builds will not stop developers shipping a backdoor to users via the source code built by these distributions / repositories. It was a backdoor in the source tarball and they shipped it to users. It was caught due to incredibly poor performance due to low quality of implementation for the backdoor, not these packaging systems.

missing-root Too bad it's untrue.

GrapheneOS

Not sure if this is whataboutism or some other wrong argument.

For sure it is an issue to use software made by other people.

In the end, if you have reproducible builds you can check the code and be safe. If you just take some release APKs, you can check the sources and still get malware.

ryrona
Having reproducible builds is nice. But I'm not using GOS or and other apps because it is reproducible. I'm using them because I trust the devs to write code with good will and build it correctly. I also trust Google Play to verify apps genuine and deliver apps to me in a secure way.
I think if your threat model requires reproducible builds, you should also check the entire source and trust yourself to be able to find any backdoors in it. It's not enough to just trust others to inspect source for you. it'd be better to have your own secure build environment and build it yourself