CLDSRV-917: allow writing objects directly to a cold location

[francoisferrand]

Users had no way to store an object in a cold location without first writing it hot and waiting for a lifecycle transition rule to kick in — impractical when the intent is already known at write time.

PutObject, CreateMultipartUpload and CopyObject now accept the name of a cold location in x-amz-storage-class. The data is still written to the hot location as usual, but the object is stamped with the cold storage class and flagged as transition-in-progress, so the lifecycle queue populator picks it up from the oplog and drives the transition from there. Cloudserver writes no Kafka message itself, and a requeue keeps the flag set.

Resulting metadata for such a write:

field	value
x-amz-storage-class	the cold location
dataStoreName	the hot location (unchanged)
archive	absent
x-amz-scal-transition-in-progress	true
x-amz-scal-transition-time	now
originOp	the real S3 event (Put / CompleteMultipartUpload / Copy)

Gated by a new off-by-default enableDirectToCold option (ENABLE_DIRECT_TO_COLD). Which identities may use a given storage class is left to the existing s3:x-amz-storage-class IAM condition key, so there is no new authorization code here.

Also drops the dead CLDSRV-639 lowercase/uppercase storage class juggling in CreateMultipartUpload — the value is validated beforehand, so it can just be stored as-is. That code could never match anyway, since validStorageClasses only holds uppercase values.

Issue: CLDSRV-917

[bert-e]

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request.
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Incorrect fix version

The Fix Version/s in issue CLDSRV-917 contains:

• 9.5.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 9.4.3

Please check the Fix Version/s of CLDSRV-917, or the target
branch of this pull request.

[codecov]

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
9932	5	9927	0

View the full list of 5 ❄️ flaky test(s)

"before each" hook for "should grant read access"::With default signature PUT Bucket ACL using predefined groups - authenticated request "before each" hook for "should grant read access"

Flake rate in main: 100.00% (Passed 0 times, Failed 37 times)

Stack Traces | 0.167s run time

We encountered an internal error. Please try again.

"before each" hook for "should grant read access"::With default signature PUT Bucket ACL using predefined groups - unauthenticated request "before each" hook for "should grant read access"

Flake rate in main: 100.00% (Passed 0 times, Failed 37 times)

Stack Traces | 0.051s run time

We encountered an internal error. Please try again.

"before each" hook for "should grant read access"::With default signature PUT Object ACL using predefined groups - authenticated request "before each" hook for "should grant read access"

Flake rate in main: 100.00% (Passed 0 times, Failed 37 times)

Stack Traces | 0.036s run time

We encountered an internal error. Please try again.

"before each" hook for "should grant read access"::With default signature PUT Object ACL using predefined groups - unauthenticated request "before each" hook for "should grant read access"

Flake rate in main: 100.00% (Passed 0 times, Failed 37 times)

Stack Traces | 0.069s run time

We encountered an internal error. Please try again.

"before each" hook for "should grant read access"::With v4 signature PUT Bucket ACL using predefined groups - authenticated request "before each" hook for "should grant read access"

Flake rate in main: 100.00% (Passed 0 times, Failed 37 times)

Stack Traces | 0.031s run time

We encountered an internal error. Please try again.

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}