Skip to content

Decode text timestamps ~40x faster on macOS, ~15x on Windows and ~1.4x on Linux, without timegm() - #759

Merged
larskanis merged 3 commits into
ged:masterfrom
czj:perf/text-timestamp-without-timegm
Oct 27, 2026
Merged

larskanis merged 3 commits into
ged:masterfrom
czj:perf/text-timestamp-without-timegm

Conversation

@czj

@czj czj commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Following up on #758: while profiling our Rails app on macOS, decoding a timestamp turned out to cost ~3.6 µs per value, 50 times an integer. Nearly all of it is timegm(), which macOS implements as a binary search over gmtime() (~3.4 µs per call, against 33–55 ns with glibc).

This counts the seconds since the epoch arithmetically instead (Howard Hinnant's days_from_civil), for timestamps with an explicit offset and for timestamps without one read as UTC, which is what Rails does by default. Windows gets the fast path for these too: it has no timegm(), so it always went through Time.new. TIMESTAMP_DB_LOCAL still goes through mktime(), so TimestampLocal (and Rails with default_timezone = :local) stays as slow as before on macOS, and keeps using Time.new on Windows, where be07045 explains why mktime() must stay off. Values a 32-bit time_t can't hold still take the slow path, which now keeps nanoseconds too, so both paths return the same Time.

Each figure is the median of 7 passes, each averaging 10,000 values, with Ruby 4.0.9. On GitHub Actions, both versions were compiled from source with the same flags and run alternately, and the table shows the median of 3 rounds (run). macOS was measured locally:

TimestampUtc TimestampWithTimeZone TimestampLocal
macOS, Apple silicon 3,604 → 90 ns (~40x) 3,444 → 91 ns unchanged
Windows x64 (ucrt) 1,988 → 126 ns (~16x) 1,889 → 131 ns (~14x) 2,009 → 2,001 ns
Linux x86_64 218 → 152 ns (−30%) 221 → 152 ns (−31%) 2,583 → 2,591 ns
Linux arm64 210 → 150 ns (−29%) 208 → 150 ns (−28%) 1,139 → 1,136 ns

On Windows, the fast path also allocates 1 object per value instead of 10.

In our app, an e-commerce page decodes 750 to 3,100 timestamps (created_at/updated_at add up). That's 2.6 to 11 ms per page in development on a Mac, and 0.03 to 0.2 ms in production on Linux.

To check equivalence, I decoded 200,000 random timestamps (years 1 to 294276, BC, offsets in hours, minutes and seconds) with the 4 text timestamp decoders, with pg 1.7.0 and with this branch: under 3 time zones on macOS and Linux (x86_64 and arm64), and under the default one on Windows. Output is identical for every value with up to 6 fractional digits, i.e. everything PostgreSQL sends, and 100,000 values with 9 fractional digits match Time.utc at nanosecond precision on every platform. Beyond microseconds, pg 1.7.0 truncated whenever it fell back to the slow path (macOS before 1900 or BC, Windows, 32-bit time_t); the new spec covers that and fails on macOS with pg 1.7.0. I also ran the type specs with the fast path disabled, to cover 32-bit builds, and a new spec walks february, leap years, centuries and BC dates.

One trade-off: with a leap-second zone such as TZ=right/UTC, glibc's timegm() counted leap seconds, as Ruby's Time does there, while the arithmetic doesn't, like PG::BinaryDecoder::Timestamp already. On glibc under such a zone, UTC timestamps would now come out 27 seconds apart from Time.utc. These zones are rare, but I'd rather mention it. Is it acceptable for you, or should the decoder keep timegm() on glibc?

Full suite: 1,166 examples, 0 failures except tells about the libpq library path, which only comes from my build setup (compiled with extconf.rb + make rather than rake compile).


🤖 Generated with Claude Code, but prompted, read, and verified by @czj, a caring human :)

macOS implements timegm() as a binary search over gmtime(), taking ~3.4µs
per call. Counting days arithmetically decodes timestamps ~40x faster there,
keeps nanoseconds for years the libc can't convert, and gives Windows the
fast path.
Comment thread ext/pg_text_decoder.c Resolved
@larskanis

Copy link
Copy Markdown
Collaborator

Good catch! gmtime() is missing on Windows, so that the patch should give a good speedup there too.

- Windows' mktime() fails on certain dates (be07045), so local timestamps
  keep using Time.new there, as before dropping the timegm() check
- The slow path keeps nanoseconds, so 32 bit time_t builds decode the same
  values as the fast path
- The UTC and explicit offset branches share one computation, and struct tm
  is only filled for mktime()
- Specs cover february, leap years, centuries and BC dates
On Windows long is 32 bit, so Fixnums stop at ~1.07 billion and
sec * 1000000000 + nsec became a Bignum: TimestampLocal, which still takes
Time.new there, decoded 12% slower with 17.8 objects per value instead of 10.
Microseconds, all PostgreSQL sends, keep the numerator a Fixnum again.
@czj czj changed the title Decode text timestamps ~40x faster on macOS and ~1.4x on Linux, without timegm() Decode text timestamps ~40x faster on macOS, ~15x on Windows and ~1.4x on Linux, without timegm() Oct 26, 2026
@czj

czj commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

You're right, thanks! Windows has no timegm(), so these timestamps always went through Time.new there. On a GitHub Actions Windows runner, TimestampUtc goes from 1,988 to 126 ns and TimestampWithTimeZone from 1,889 to 131 ns, with 1 object per value instead of 10. TimestampLocal stays on Time.new because of mktime() (be07045). I've added these figures to the description.

@larskanis

larskanis commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

I think this is OK how it is. Thank you very much!

I'll check if the #ifndef _WIN32 is still necessary, because 8 years ago, when this was implemented, the old MSVCRT was used on Windows, which is completely replaced by UCRT in the meantime. And the 3 different ways to build sec_value isn't necessary then any longer.

Update: PR is here #764

@larskanis
larskanis merged commit 20d13a2 into ged:master Oct 27, 2026
35 checks passed
@czj
czj deleted the perf/text-timestamp-without-timegm branch October 27, 2026 09:16
@czj

czj commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @larskanis, and nice to see #764 simplify the Windows path!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants