One year of postmarketOS

Posted Jun 12, 2018 17:29 UTC (Tue) by excors (subscriber, #95769)
In reply to: One year of postmarketOS by marcH
Parent article: One year of postmarketOS

> Stable APIs are ironically a major feature of... closed-source systems (e.g. for PCs: Wintel).

I think they're largely orthogonal issues. E.g. Android has a fairly stable API in its SDK and NDK (which are open source), so apps (usually closed source) will remain compatible with future Android versions. But its HAL interfaces (open source) often aren't stable (though I think Treble is fixing / has fixed that), so HAL implementations (closed source) might need updating or rewriting on each upgrade.

The HAL implementations also rely on syscalls to standard kernel APIs (open source), which are stable, and calls to firmware (closed source) via proprietary kernel drivers (open source though not in spirit), which probably don't have stable APIs. The proprietary kernel drivers themselves rely on unstable internal kernel APIs. It's all a mixture - there seems to be no clear correlation between openness and stability.

Conway's law is probably a bigger factor: software architecture mirrors organisational structure. API stability is very expensive to maintain, so it's only worthwhile for APIs between people who cannot make coordinated changes. Kernel developers can coordinate easily with each other, so Linux doesn't need a stable internal API; but they cannot coordinate with application developers, so they do need stable external APIs. On Android devices, there is typically one company (Samsung, Sony, Huawei, etc) who controls the entirety of the software (except for apps) and can easily coordinate between bootloader and kernel and Android platform and HAL and firmware etc - those are probably all managed in the same Gerrit/Perforce/etc server by the same developers with a single CI system - so it makes sense to avoid the expense of stable APIs even for their kernel drivers. (Even if some of that code is blobs from other vendors, that one company is responsible for integrating it all and can apply patches atomically across the entire system.)

But that makes it hard for Google to take back control of part of the device's software - that's changing the organisational structure so that it no longer matches the software architecture, and it takes a lot of effort to rearchitect it with the new boundaries. So Android devices have largely remained under control of a single company, for better or worse. (In terms of security, usually worse, since those companies care much less about security than Google does.)

One year of postmarketOS

Posted Jun 12, 2018 19:38 UTC (Tue) by marcH (subscriber, #57642) [Link] (2 responses)

> there seems to be no clear correlation between openness and stability.

I don't think anyone said that interfaces to open-source components are automatically unstable. They CAN be unstable/forked more easily and frequently than interfaces to closed-source components and lead to a fragmentation mess like the one you described. Random example of forked kernel interface(s): https://elinux.org/Android_Binder

On the other hand, when an organization (hi Conway!) delivers a closed-source component to another organization then stability expectations are obviously higher otherwise things will just stop working.

I'm everything but a HAL expert but I thought changes to HAL interfaces were carefully versioned:
https://developer.android.com/reference/android/hardware/...

One year of postmarketOS

Posted Jun 12, 2018 20:48 UTC (Tue) by excors (subscriber, #95769) [Link] (1 responses)

> when an organization (hi Conway!) delivers a closed-source component to another organization then stability expectations are obviously higher otherwise things will just stop working.

From what I've seen on Android, it's not unusual for a vendor to provide a package containing a firmware blob, some kernel patches, and some userspace demo code that shows how to use the API, where that API is unstable between releases and there's no attempt at backward compatibility. You have to apply all three parts simultaneously, plus update your own code that uses the API, else you'll get build failures (if you're lucky) or crashes (if you're less lucky but still quite lucky). And that's okay, since the receiving organisation controls their entire software stack and can push all the patches at once.

The difference with Windows isn't that Windows is closed source, it's that no single organisation controls all the software on a Windows machine - some comes from Microsoft, some drivers come directly from third-party vendors, others provide applications and utilities that depend deeply on Windows internals, etc. That drives the need for stable APIs, which then makes long-term support more feasible since one component can be updated without touching anything else.

> I thought changes to HAL interfaces were carefully versioned

Camera API2 (aka camera2) is designed for Camera HAL3, and that HAL is fairly carefully versioned and sensibly extensible, with good documentation of the HAL API usage rules. But as the numbers suggest, there was a HAL2 (which I think might have been shipped in about one device, before it was considered an overly-complex failed experiment and evolved into the broadly similar but incompatible HAL3), plus the original and vastly different HAL1 (which was not extensible and suffered from plenty of vendor-specific hacks, and (if I remember correctly) often changed incompatibly between Android releases). So the camera seems alright now, but it took a lot of effort to reach this point.

Camera HAL2/3 was needed because cameras were adding sophisticated features that HAL1 was completely unsuitable for, and that redesign gave an opportunity to fix versioning and extensibility based on lessons learned. Other non-camera HALs never went through a similar redesign, so they didn't have an opportunity to fix the mistakes left over from Android's early history. I assume they've probably been fixed now as part of Project Treble, but I haven't paid much attention to recent changes.

One year of postmarketOS

Posted Jun 12, 2018 21:11 UTC (Tue) by marcH (subscriber, #57642) [Link]

> The difference with Windows isn't that Windows is closed source, it's that no single organisation controls all the software on a Windows machine

OK, I think I understand the misunderstanding now. I tried to clarify but you seemed to have missed it:
> ... when an organization (hi Conway!) DELIVERS a closed-source component to another organization...

When I wrote "closed-source API" I meant: *released* (and stable) closed-source API.

If a single organization controls all the software and doesn't release any particular piece, just the whole product then who cares which APIs are used behind that doors? It's irrelevant.