[bbirdsong2]

Adds a new labeled_quantity metric for relevance scores ad_client.relevance that can be included in an ad request. labeled_quantity is a new metric type added in a recent PR to the glean_parser go_server.go generator.

Scores are computed on the client with differential privacy and sent over OHTTP. No client identifiers are included in ad requests that contain scores.

Each stored value is the score (from 0 to 1) multiplied by 10,000 and stored as an integer. This is because there did not seem to be a good option for persisting a map of unbounded strings to decimals through glean. So we multiple the decimal score and retain at most 4 decimal places.

[gleonard-m]

Pending data review form

[gleonard-m]

Pulled PR and ran glean_parser version 22.0.1 to generated server_events.go. Clean file generation.

[bbirdsong2]

Merging this now so the column gets added tonight. The data collection has already been reviewed, we are finalizing the data review form to post shortly.

[bbirdsong2]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

Do relevance scores computed on the client, with differential privacy, make sponsored shortcuts on the New Tab page more relevant to users, as measured by engagement (impressions, clicks, and click-through rate)?

Which privacy parameters produce a useful relevance signal while keeping privacy loss as low as possible?

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

Sponsored shortcuts are part of how Firefox is funded. Today the sponsored shortcuts a user sees are chosen without any signal about what the user is interested in, so many users see sponsors that are not relevant to them. We want to show more relevant sponsors without building an interest profile on our servers and without linking anything to a user identifier. Users are benefited by more relevant sponsored shortcuts, using a design where browsing history never leaves the device.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

Keep selecting sponsors without a relevance signal. This is the baseline we compare against and it does not improve relevance.

Send browsing history, or a list of visited domains, to the server. Rejected because it sends raw browsing data off the device.

Choose among sponsors entirely on the client. This would mean sending a much larger set of candidate sponsors to every client, limiting backend sponsor allocation and pacing functionality. It may also offer less privacy benefit than it seems, since impressions and clicks would still be reported, and the sponsor a client chose could itself reveal what it inferred. Future work is planned to explore the balance between selection on the client and on the server.

The chosen design computes scores on the client from history that never leaves the device. It adds differential privacy noise and post-processing. Scores are sent only over OHTTP and without any client identifier. It answers the questions with much less data and lower identifiability than the alternatives.

4. Can current instrumentation answer these questions?

No. Existing sponsored shortcut telemetry records impressions and clicks, but there is no signal about interest, so it cannot tell us whether a relevance signal improves the sponsors shown.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description	Data Collection Category
relevance field in the sponsored shortcuts ad request from Firefox Desktop New Tab to the Mozilla Ad Routing Service (MARS) sent over OHTTP with no client identifiers.	Category 3: Stored content and communications
relevance in the mars-telemetry interaction and request-stats pings.	Category 3: Stored content and communications

Stored only on the device, never sent: daily noisy scores for up to 30 days (used for the trend fit), the released scores, and the time of the last release. These are cleared when the user clears history, deletes pages from history, or turns off sponsored shortcuts.

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

• The server-side metric is collected using Glean and is documented in its metrics.yaml in mars-telemetry and in the Glean Dictionary at https://dictionary.telemetry.mozilla.org/apps/ads_backend

7. How long will this data be collected? Choose one of the following:

• We want to permanently monitor this data.

8. What populations will you measure?

• US Only, All release channels. This feature will be rolled out through nimbus.

9. If this data collection is default on, what is the opt-out mechanism for users?

The feature is rolled out through Nimbus. This is default on, opt-out by turning off "sponsored shortcuts" and "sponsored stories" or flipping the "support firefox"
toggle to off in the "Home" section of preferences.

10. Please provide a general description of how you will analyze this data.

• Compare engagement with sponsored shortcuts to see if higher relevance scores relate to higher user engagement.
• Evaluate the privacy parameters still produce a usable signal and adjust values to tune them.

11. Where do you intend to share the results of your analysis?

This will not be shared.

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection? If so:

This will go through server side glean via MARS (Mozilla Ad Routing Service), the same way it does today. Notably, this
means that we don't include any client info from glean-core, like most glean pings would.