conformance

UXID conformance corpus

Test vectors for an implementation of UXID in another language. Each case is one input and the answer this library gives for it, generated by calling the library. A port vendors this directory at a release tag and runs every case unchanged; a port that passes them all produces and reads the same strings as this library for those inputs.

The rules the cases instance are in the wire format guide (guides/wire-format.md). Where a case and the guide disagree, the guide states the format and the case is a defect.

This directory is not part of the Hex package.

Never edited by hand

Every case file and manifest.json are written by mix uxid.corpus from the authored source under source/. The quality gate's Corpus stage runs mix uxid.corpus --check, which builds the files in memory and fails on any byte difference from the checked-in ones. A hand edit to a case file, or to the hash, fails the gate; the fix is to change the source (or the library) and regenerate:

mix uxid.corpus          # write the case files and the manifest
mix uxid.corpus --check  # build in memory and byte-compare; write nothing

The generator refuses to write anything when a case fails to build: the library disagreeing with a golden vector (a case in source/ that carries its expected id), an encode case that does not decode back, an invalid case the library does not refuse with the error the case names, a monotonic case whose draws break the monotonic rule, or a duplicate id.

Layout

File	What it holds
manifest.json	The corpus version, the case files in order with their kind and case count, and the corpus hash
encode.json	Time-based ids: inputs to the id a producer emits
deterministic.json	Deterministic ids: inputs to the id a producer emits
decode.json	Strings to what a decoder reads from them
invalid.json	Strings a decoder refuses and options a producer refuses, to the error's name
monotonic.json	One monotonic generator's calls and draws to the ids it emits
schema/	One JSON Schema per case file (describing one line) and one for the manifest
source/	The authored source the generator reads; not needed to run the cases

The format

• A case file holds one JSON object per line, each line ended by a newline. Object keys are sorted and there is no whitespace between tokens.
• Every case has a stable id, its kind and a slug (encode/xl-standard-lower), its input, and either its expected answer or, in invalid.json, its expected_error. A case id never changes meaning; a case that has to change gets a new id.
• An error is a name from a closed set, never a message: the decode refusal reasons and the generation errors the wire format guide lists ("Refusals" and "Generation errors").
• Every integer is at most 2^53 - 1, so a JSON parser that reads numbers as doubles reads each one exactly.
• Bytes are lowercase hex. Times are integer milliseconds since the Unix epoch. Sizes are xs, s, m, l and xl. A missing prefix is null.
• The delimiter is _ in every case.
• manifest.json is one line: corpus_version (1), files (each with kind, file and case_count, in a fixed order) and corpus_hash, the string sha256: followed by the lowercase hex SHA-256 of the case files' bytes concatenated in the order of files.

Running the cases

1. Read manifest.json. Check corpus_version is one your runner knows. Concatenate the bytes of each file in files, in order, hash them with SHA-256 and compare to corpus_hash; a mismatch means the vendored copy is not the one the manifest describes.
2. For each file, split on newlines and parse each line as one case. Check the line count against case_count.
3. Run each case by its kind:
   • encode: build an id from input with the random field set to the bytes of rand (not drawn from a random source) and compare it to expected. prefix "" and null are both no prefix.
   • deterministic: build the deterministic id of input.from under input.prefix at input.size in input.case and compare it to expected.
   • decode: decode input and compare the prefix, size, compact flag, deterministic flag and time to expected. The random field is not read back on decode, so no case carries one.
   • invalid: for input.decode, decode the string and check it is refused with the reason expected_error (the body length is counted in bytes). For input.generate, make the request with those options (an absent option takes its default) and check it is refused with the generation error expected_error; how a port maps its own refusal to the name is the port's.
   • monotonic: run a monotonic generator, lower case, for input.prefix, input.size and input.compact, with its random source replaced by the case's draws, and call it once per entry of input.times with that time. The first call draws initial_seed; each later call draws the next of steps; the call whose step overflows the field also draws overflow_seed (a case overflows at most once). Compare the ids to expected in order. The draws are values, not a seed for a pseudorandom generator, so no generator algorithm is part of the corpus. No time after the first is later than the last id's time, so no other fresh value is drawn.
4. Report each failing case by its id.

Compact times and the decoder's clock

A compact body carries only the low 40 bits of its time; the decoder supplies the rest from its own clock (the guide's "Decoding the time"). The compact decode cases carry the full time this library recovers, and they all hold for a decoder clock from 1699913600000 ms (2023-11-13T22:13:20.000Z) inclusive to 2164577117183 ms (2038-08-04T23:25:17.183Z) exclusive:

• the cases at 1700000000000 ms are recovered by a clock no more than one day (86400000 ms) behind them, which sets the start;
• decode/s-compact-guard-last-valid is the compact guard's last valid millisecond (the largest compact time field, 1065151889407) from the window before, and is recovered by the one-window subtraction only while 2^40 + 1065151889407 is more than one day ahead of the clock, which sets the end.

A runner testing compact decode on a clock outside that range injects the clock rather than reading the system time.

The compact xs and compact m cases decode to the standard reading the guide names (their lengths are those of standard xs and m): the same size, compact false, and a time read from the first 10 body characters as a standard time.