Prebuilt Linux binaries require GLIBC_2.38, excluding Ubuntu 22.04 / Debian 12 — could the release job pin ubuntu-22.04?

[devsafecloudpro]

Thanks for sqlite3.dart — the prebuilt-binary hook makes SQLCipher genuinely painless to ship, which is why I'm raising this rather than working around it.

The problem: the prebuilt Linux assets reference GLIBC_2.38, so an application bundling them fails to start on any distro older than that — including Ubuntu 22.04 LTS (glibc 2.35, supported to 2027) and Debian 12 (2.36, supported to 2028).

Evidence, from libsqlcipher.x64.linux.so in the sqlite3-3.3.4 release (sha256 46205b32…a0b63, matching asset_hashes.dart):

$ objdump -T libsqlcipher.x64.linux.so | grep GLIBC_2.38
0000000000000000 DF UND 0000000000000000 (GLIBC_2.38) __isoc23_strtol
One symbol, and it isn't deliberate: __isoc23_strtol is what glibc ≥ 2.38 headers transparently redirect a plain strtol() to. It's a fingerprint of the build host, not a feature the code uses. libsqlcipher.arm64.linux.so (sha256 6826ecd9…a78db) has the identical symbol — expected, since build_sqlite cross-compiles every Linux ABI from one runner.

Why I think this is cheap to fix: in .github/workflows/compile_sqlite.yml, both build_openssl and build_sqlite use runs-on: ${{ matrix.os }} with ubuntu-latest in the matrix, and install gcc-aarch64-linux-gnu etc. to cross-compile the other ABIs. Since every Linux ABI already comes from that single host, pinning it to ubuntu-22.04 would lower the floor for all of them at once:

strategy:
matrix:
os: [ubuntu-22.04, macos-latest, windows-latest]
Ubuntu 22.04 is glibc 2.35, and 22.04 runners are still available on GitHub Actions.

Questions, since you know the constraints better than I do:

Is ubuntu-latest deliberate here — e.g. do the sanitizer or Android jobs need a newer toolchain? Those could stay on ubuntu-latest independently; only the release-artifact jobs need the older host.
Would you rather set an explicit floor via a sysroot or -D_ISOC23_SOURCE-style opt-out than pin the runner? Either works for us; pinning just looked like the smallest change.
I've only measured the sqlcipher assets. Presumably the sqlite3 and sqlite3mc Linux assets from the same job have the same floor — worth checking if you take this on.
Workarounds we considered, for context on why I'm asking upstream: the hook resolves releaseTag from a const bound to the package version and user_defines exposes no tag or hash override, so pinning an older asset isn't possible from the consumer side — and the previous release (sqlite3-3.3.3, the first with sqlcipher assets) ships byte-identical binaries anyway (same sha256 and same BuildID), so there's nothing older to fall back to. Building SQLCipher ourselves means reproducing the static OpenSSL 3.6.2 cross-build too, which we'd rather not take on.

Happy to open the PR if you'd like — it's a one-line change, though only you can confirm it doesn't break the jobs I can't see the constraints on.

[simolus3]

Hi, you raise a good point and the glibc version incompatibility is an oversight.

Why I think this is cheap to fix: in .github/workflows/compile_sqlite.yml, both build_openssl and build_sqlite use runs-on: ${{ matrix.os }} with ubuntu-latest in the matrix, and install gcc-aarch64-linux-gnu etc. to cross-compile the other ABIs. Since every Linux ABI already comes from that single host, pinning it to ubuntu-22.04 would lower the floor for all of them at once:

I feel like pinning an older actions runner might have other downsides as well, e.g. us having to raise the minimum version when that runner stops being supported.

We could look into using a Docker container for the build, or see if we can grab an older glibc somewhere and use that as a sysroot with a modern gcc. Basically, I want to run as little of the build on old software as possible, to make sure my local setup matches what we do in CI and that I can debug stuff easily.

I know this is a bit more complicated than that one-line change, but I'd still be happy to review a PR if you want to open one. Otherwise let me know and I can take a look myself.

[devsafecloudpro]

Thanks — and your instinct is right, my one-line suggestion doesn't actually work. I dug into the artifacts to check before agreeing, and the mechanism turns out to be more specific than either of us assumed. Sharing it in case it saves you time.

It's one symbol, and the residual floor is lower than I asked for. Parsing the shipped libsqlcipher.*.linux.so version tables, __isoc23_strtol is the only symbol bound above GLIBC_2.34:

libc.so.6 required: 2.17, 2.25, 2.28, 2.33, 2.34, 2.38
bound to GLIBC_2.38: __isoc23_strtol (nothing else)

So whatever the fix ends up being, it lands the artifacts at glibc 2.34, not 2.35 — which covers Ubuntu 22.04 and Debian 12 with room to spare, and picks up RHEL/Rocky 9 as a bonus.

The trigger is _GNU_SOURCE, not the C standard — this is the part worth knowing, because "just pass -std=gnu17" is the obvious first thing to try and it is a no-op here. glibc's features.h makes _GNU_SOURCE imply _ISOC2X_SOURCE, and __GLIBC_USE_ISOC2X is:

#if (defined _ISOC2X_SOURCE
|| (defined STDC_VERSION && STDC_VERSION > 201710L))

— an OR, not an AND. So once _GNU_SOURCE is defined, -std is irrelevant. And ubuntu-latest is still Ubuntu 24.04, whose gcc 13.3 and clang 18.1.3 both already default to gnu17, so there is nothing for a -std flag to change.

The decisive evidence is in the binary itself: it contains both

strtol@GLIBC_2.17 (plain, unredirected)
__isoc23_strtol@GLIBC_2.38 (redirected)

side by side. A global flag could never produce that mixture — it would redirect all call sites or none. The split tracks exactly which translation units have _GNU_SOURCE defined, and both affected sources define it themselves (SQLCipher's sqliteInt.h, and OpenSSL's rand_unix.c on the OpenSSL side), so it can't be switched off from the command line either.

Which means you were right and I was wrong: since the redirect can't be disabled at the call site, the only real fix is to compile against an older glibc — your sysroot or container idea. I'd gently vote for whichever you'd rather maintain; from the outside the container looks simplest, but you're the one who has to debug it, and your point about wanting your local setup to match CI is a better reason than any of mine.

One small note in favour of not pinning the runner, beyond what you said: it wouldn't be a stable fix anyway, since a future runner image bumping glibc would silently re-break it with no signal. A sysroot pins the actual thing that matters.

I'm going to pass on opening the PR — it's your CI across five ABIs and two build systems, and I'd be guessing at constraints you can see and I can't. But I'd be glad to verify a fix: I have a real Ubuntu 22.04 target and a symbol-floor checker wired up, so if you push a branch or a draft PR I can run the produced artifacts through it and report the resulting floor per architecture within a day. Happy to do that as many times as it takes.

Thanks for looking at this so quickly.

[simolus3]

Fixed in sqlite3: ^3.6.0.