Settings without a consumer and consumers without settings #1403

Many features need two things in a deployment before they do anything. One is a settings section. The other is the consumer that acts on it, being a middleware, a module, a request filter, a page request preparer or a part of the adapter the deployment runs on. When one is present without the other, Trusted Server refuses to start, logs a warning, or runs as though nothing were wrong, and which of the three happens depends on the feature. Each feature that checks at all has written its own check, the checks run at different times and say different things, and several features have no check.

Code references are at 3aaf720c0, the head of the draft stack, which is the cookies by path change (#1400). Startup means an adapter loading the settings and building its state. Push means ts config validate, ts config diff and ts config push, which run the same checks. In the Code column below, core/ stands for crates/trusted-server-core/src/.

One case, the 51Degrees client hints

The client hints change (#1386, row 44 of the stack) adds the middleware 51degrees.client_hints, which writes the Delegate-CH tag and the client hint headers on the pages a [[serve]] entry names it for. The middleware needs [shared.51degrees], which holds the cloud endpoint and the license key. Whenever that table is present, the crate registers two things, a request filter that asks the cloud service which hints the browser should send, and the middleware that writes the answer into the page (lib.rs lines 110 to 132).

The settings without the consumer. With [shared.51degrees] and no [[serve]] entry naming the middleware, push accepts the document and startup logs one warning, that the middleware "changes no page" (registry.rs lines 1724 to 1731). The request filter is not tied to the entry, and core runs every registered filter (registry.rs lines 2640 to 2674), so on Fastly the filter still asks the service on every page request (lib.rs lines 352 to 381). A deployment that has not placed the middleware therefore asks the service, through its cache, about every page request, and no page gets the tag.

The consumer without the settings. With the [[serve]] entry and no [shared.51degrees], the crate does not run, so nothing supplies the middleware. Startup refuses the entry as naming "51degrees.client_hints, which no module that runs supplies" (registry.rs lines 1669 to 1685). The message never says that [shared.51degrees] is missing, because the note naming absent shared tables (lines 1162 to 1178) is added to a refused section selection and not to a refused entry. Push accepts the same document, because push checks only the shape of each entry (settings.rs lines 4776 to 4791) and the crate's own validation stops early when the table is absent (lib.rs lines 142 to 150).

Both present, on an adapter without request filters. Only the Fastly adapter runs request filters (Fastly app.rs lines 836 to 867). The Axum, Cloudflare and Spin adapters call no request filter, so there the middleware runs on every page an entry names, finds no hints left for it (lib.rs lines 264 to 268 and 286 to 298) and writes nothing, with no log line.

The cases

Three more things stand out across the table.

1. The first check that fails stops the start, and most checks stop at the first gap they find, so a document with two gaps of different kinds is fixed one failed start at a time.
2. Push builds no registry, so every check that lives in the registry runs at startup only. A document that push accepts can still take a service down when it next starts.
3. Each placement check was written for one feature. InMobi and the ads.txt sellers count any entry for their media type, whereas robots.txt requires an exact path.

The common startup model

One model in core, in four points.

1. Every consumer declares the settings it needs. A builder already declares the shared table it reads, its secret settings, the caches it loads through and the middleware it supplies with its phase and media type, as the 51Degrees builder does (lib.rs lines 94 to 100). The builder would also declare, for each middleware, module, request filter and page request preparer it supplies, the settings that consumer needs and what places it, being a section selecting it or an entry naming it for a media type and path. A request filter or preparer that only feeds one middleware is declared as part of that middleware, so it runs only where an entry names the middleware. A consumer that runs on defaults declares its settings optional, so leaving them out is not a gap.
2. Every settings section has a consumer. Core's own sections declare their consumers in the same way, for example [robots-txt] and its middleware, so a section with no consumer that is linked, selected and placed is a gap.
3. Each adapter declares the parts it has, such as running request filters, dispatching asset routes, reading the telemetry sink, honoring certificate_check, a host geo lookup and a host device module. A section or a consumer that needs a part the adapter lacks is then a gap on that adapter, reported by name, rather than accepted and ignored.
4. Startup, ts config validate and ts config push give one report listing every gap. Before any builder builds, core goes through the declarations once and collects every gap into one report, in which each line names the section, the consumer, the missing half and what to add. Each line refuses or warns, as the design document sets out, and startup refuses when any line refuses. Push runs the same check against the builders the tool links and the parts of the adapter the document is pushed for, so a document with a gap that refuses is stopped before it is written to the config store.

The declaration types and the check live in core and name no vendor, standard or platform. Each vendor crate declares its own consumers in its own folder, and each adapter declares its own parts.

What it would replace

• Core's refusals of an entry and of a [middleware.<name>] table naming a middleware nothing supplies, and the warning for a middleware no entry names (registry.rs lines 1669 to 1731).
• The refusal of a shared table no builder reads and the note naming absent tables (registry.rs lines 2044 to 2069 and 1162 to 1178).
• The section selection check and its notes on modules no section selects (registry.rs lines 1774 to 1825 and 1118 to 1157).
• Core's robots.txt placement check (settings.rs lines 4414 to 4462) and the placement checks the InMobi prompt and the ads.txt sellers each wrote (inmobi lib.rs lines 141 to 158, iabtl lib.rs lines 152 to 167).
• The cache table check (mod.rs lines 1008 to 1025), which already runs at push and becomes one kind of line in the report.
• The one-off notice for a setting nothing acts on, being the DataDome warning for its reserved GraphQL flag (datadome lib.rs lines 513 to 515).
• The fixes the related issues below each ask for one setting at a time, such as a startup warning for asset routes (proxy.asset_routes is parsed and validated on every adapter but dispatched only on Fastly #1125) or for certificate_check (proxy.certificate_check is accepted and ignored by the Axum adapter #1132).

Related issues

Of the same shape:

• proxy.asset_routes is checked on every adapter and dispatched only on Fastly (proxy.asset_routes is parsed and validated on every adapter but dispatched only on Fastly #1125)
• proxy.certificate_check is accepted and ignored by the Axum adapter (proxy.certificate_check is accepted and ignored by the Axum adapter #1132)
• Unknown integration IDs and fields pass validation and are ignored (Unknown integration IDs and fields in [integrations.*] pass validation and are ignored #1201)
• Two Edge Cookie cluster settings are documented as controls and nothing reads them (Two Edge Cookie cluster settings are parsed and documented as controls but nothing reads them #1097)
• One health and startup failure contract across the adapters (Define a uniform cross-adapter health and startup-failure contract #1142)
• Tinybird telemetry settings are accepted on adapters that send nothing (Implement or reject Tinybird telemetry configuration coherently #1148)
• An adapter with no geo lookup fails closed on Edge Cookie creation and the auction (An adapter with no geo module fails closed on Edge Cookie creation and on the auction #1134)
• On Cloudflare no region is resolved, so every US privacy signal fails open (On Cloudflare no region is resolved, so every US privacy signal fails open #1102)
• Cloudflare startup does not read the store ts config push writes (Connect Cloudflare config store to ts config push output #1140)
• Axum does not read the local store ts config push writes (Make Axum consume the local config store written by ts config push #1141)
• Adapter stores that are declared and never built (Implement adapter stores or retire dead store manifests #1139)
• The test-only list of integration IDs that deploy validation checks against (Remove or harden the test-only deploy ID inventory #1146)
• The Spin adapter cannot build its state and answers every request but the health probe with 503 (The Spin adapter cannot build application state, so it answers every request except the health probe with 503 #1101)
• The plan to refuse unknown integration IDs in a new configuration layout (Define integrations crate split and ordered configuration #1194)
• RUST_LOG had no effect on the Axum adapter (RUST_LOG has no effect on the Axum adapter, which always logs at trace #1129, closed)
• Configuration errors silently disabled integrations (Configuration errors silently disable integrations #408, closed)

The issues whose changes added checks of this shape, each written for its own feature, are a vendor's shared table (#1284), a module's own loading cache (#1320), a middleware's own settings table (#1331) and the 51Degrees client hints (#1333). The stack as a whole is listed in #777.

A design document for this model follows in the pull request that holds every design document of the stack (#1342).