Skip to content

Add an option to take all random values directly from the OS - #315

Merged
utelle merged 4 commits into
utelle:mainfrom
SchwarzDigits:feat/os-rng
Nov 19, 2026
Merged

utelle merged 4 commits into
utelle:mainfrom
SchwarzDigits:feat/os-rng

Conversation

@mtrossbach

@mtrossbach mtrossbach commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

sqlite3mc gets its IVs, nonces, salts and the keys for temporary files from chacha20_rng, a ChaCha20 generator seeded from the OS. This PR adds the compile-time option SQLITE3MC_USE_OS_RNG. With the option set to 1, chacha20_rng takes every value directly from the operating system: from entropy(), the source that seeds the generator today, and on Windows from BCryptGenRandom. If the OS fails, it aborts, as it does now. The default is 0, so nothing changes unless someone switches it on.

Two cases where this helps:

  • A virtual machine restored from a snapshot that includes memory. The process carries on with the same generator state and hands out the same values again until the next reseed. With chacha20, aegis or ascon128, the same page can then be encrypted twice with the same nonce. The OS can notice a restore (Linux reseeds when the hypervisor provides a VM generation ID); a generator inside the process can't.
  • Security reviews. Auditors know the random source of the platform but not ours, so the generator is one more thing to explain. SQLCipher avoids this by asking its crypto provider for every page.

It costs one OS call per request, which is small compared with encrypting a page. In the smallest write transactions (one row, no fsync) it added 1 to 3.5 % on macOS; on Windows the difference was within the noise. With fsync the share is much smaller, and reading needs no random values at all.

On Windows, entropy() uses rand_s by default, which delivers only 4 bytes per call: fine for the seed, but the 128 bytes the sqlcipher scheme asks for per page would take 32 calls. So with the option on, Windows uses BCryptGenRandom, which fills the whole buffer in one call and is also available to UWP apps.

Build options: -DSQLITE3MC_USE_OS_RNG=ON for CMake and --enable-os-rng for configure. On Windows both link bcrypt; with MSVC the #pragma comment(lib, "bcrypt.lib") in the code does that as well. I left premake and the makefiles in build/ alone; there the define has to be set by hand, and MinGW builds need -lbcrypt.

A new CI step runs the tests a second time with the option (only in the gcc jobs, to keep the CI time down).

entropy() rejects an all-zero result. That is fine because every caller asks for at least 8 bytes; a comment says so, so that nobody adds a one-byte request later.

With the option on, the name chacha20_rng no longer fits. I kept it so that none of the callers change, but feel free to rename it.

Testing, with the option on, all CI test programs (cryptotest, tempfiletest, oomtest, rekeytest, threadtest, crashtest, vfsstacktest) and a small check program, not part of this PR, that writes and reopens a database with every cipher:

  • Linux x86_64 and arm64, macOS arm64.
  • WASM with Emscripten and Node, both with the Web Crypto call and with SQLITE3MC_WASM_USE_GETENTROPY (all except the thread test).
  • Android API 24 and 34 (x86_64 emulator) and API 37 (arm64 emulator).
  • Windows MSVC and MinGW: the check program; with MinGW also cryptotest.
  • The build options: CMake on macOS, Windows MSVC and MinGW, configure on Linux and MinGW. Without the CMake option, a MinGW build with only the define fails to link because bcrypt is missing, as expected.

The check program sends about 110,000 requests to getrandom on Linux and to crypto.getRandomValues under WASM with the option, and 2 to 3 (the seed) without it.

You wrote in October that taking all random bytes from the OS would be fine with you. If you like, I can also switch the default to 1 in this PR.

@utelle
utelle merged commit da776c0 into utelle:main Nov 19, 2026
13 checks passed
@utelle

utelle commented Nov 19, 2026

Copy link
Copy Markdown
Owner

With the option on, the name chacha20_rng no longer fits. I kept it so that none of the callers change, but feel free to rename it.

Yes, in the new variant chacha20 plays no role at all in the implementation. So, renaming the function seems to be the right thing to do.

You wrote in October that taking all random bytes from the OS would be fine with you.

It is. The amount of random bytes needed ist relatively small, so the runtime penalty should usually be small.

If you like, I can also switch the default to 1 in this PR.

For now I'd leave it as is, but let's discuss this again, when the next release will be prepared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants