[DRAFT] feat(google-api-core): add support for resumable uploads

[parthea]

Towards b/457416314, b/556259599

[gemini-code-assist]

Code Review

This pull request introduces a new resumable transfer library for Google APIs, implementing both synchronous (using requests) and asynchronous (using aiohttp) resumable upload sessions, supported by a sans-I/O protocol state machine and comprehensive tests. The feedback highlights several key areas for improvement: ensuring backward compatibility with Python 3.7/3.8 by replacing asyncio.to_thread with loop.run_in_executor, handling byte-type header keys in the state machine, rejecting unsupported str and dict stream types early, retrying timeouts globally in the synchronous session, and removing redundant deadline checks.

[gemini-code-assist]

In the synchronous implementation, requests.exceptions.Timeout is not retried if stall control is enabled, even for control requests (like initiate or _recover) which are not subject to data transfer stall control. This can cause premature failures on flaky networks. Since chunk timeouts are already caught and converted to TransferStalledError (which is not retryable) in _transmit_chunk, we can safely allow requests.exceptions.Timeout to be retried globally to match the asynchronous implementation's behavior.

                if isinstance(exc, requests.exceptions.Timeout):
                    return True

[gemini-code-assist]

If a str or dict is passed as the stream to upload_async, it will pass the isinstance(stream, Iterable) check but fail later with a cryptic TypeError inside the background task when trying to read the first chunk. We should explicitly reject str and dict early to fail fast with a clear TypeError.

Suggested change

	if isinstance(stream, Iterable):
	if isinstance(stream, (str, dict)):
	raise TypeError(f"Unsupported stream type: {type(stream)}")
	if isinstance(stream, Iterable):

References

1. Defensive programming: Always type-validate structure and inputs to avoid unexpected runtime TypeErrors. ^(link)

[gemini-code-assist]

While header values are decoded if they are bytes, header keys are not. If a header key is passed as bytes, it will remain bytes in req_headers, which can cause issues with downstream HTTP libraries. We should decode both keys and values if they are bytes.

Suggested change

	if headers:
	for k, v in headers:
	req_headers[k] = v.decode("utf-8") if isinstance(v, bytes) else str(v)
	if headers:
	for k, v in headers:
	key = k.decode("utf-8") if isinstance(k, bytes) else str(k)
	val = v.decode("utf-8") if isinstance(v, bytes) else str(v)
	req_headers[key] = val

References

1. Defensive programming: Always type-validate structure and inputs to avoid unexpected runtime TypeErrors. ^(link)

[gemini-code-assist]

The check remaining <= 0 after calling _get_deadline_remaining() is redundant and unreachable because _get_deadline_remaining() itself raises DeadlineExceeded when remaining <= 0. We can simplify this by just calling _get_deadline_remaining().

Suggested change

	remaining = self._get_deadline_remaining()
	if remaining is not None and remaining <= 0:
	raise exceptions.DeadlineExceeded(
	f"Resumable upload deadline {self._config.deadline} exceeded."
	)
	self._get_deadline_remaining()

[gemini-code-assist]

The check remaining <= 0 after calling _get_deadline_remaining() is redundant and unreachable because _get_deadline_remaining() itself raises DeadlineExceeded when remaining <= 0. We can simplify this by just calling _get_deadline_remaining().

Suggested change

	remaining = self._get_deadline_remaining()
	if remaining is not None and remaining <= 0:
	raise exceptions.DeadlineExceeded(
	f"Resumable upload deadline {self._config.deadline} exceeded."
	)
	self._get_deadline_remaining()

[gemini-code-assist]

The error message "Server offset {received} precedes active buffer. Stream cannot be rewound." is misleading if received actually exceeds the active buffer (i.e., received > chunk_end). We should make the error message more general or distinguish between preceding and exceeding.

Suggested change

	err = exceptions.UnseekableStreamError(
	f"Server offset {received} precedes active buffer. Stream cannot be rewound."
	)
	err = exceptions.UnseekableStreamError(
	f"Server offset {received} does not align with the active buffer. Stream cannot be repositioned."
	)