[SVLS-8582] feat(logs): change default of durable log buffer size from 0 to 5 - #1324
[SVLS-8582] feat(logs): change default of durable log buffer size from 0 to 5#1324lym953 wants to merge 1 commit into
Conversation
Durable functions no longer need this env var set to get logs enriched with durable execution context. Non-durable functions are unaffected once the extension learns the function is not durable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
lym953
changed the title
There was a problem hiding this comment.
Pull request overview
Updates Bottlecap’s Lambda configuration to restore the non-zero default for durable-function log buffering (DD_LAMBDA_DURABLE_FUNCTION_LOG_BUFFER_SIZE), so logs can be held briefly to await durable execution context and be enriched when tracer metadata arrives.
Changes:
- Change
LambdaConfigdefaultlambda_durable_function_log_buffer_sizefrom0to5. - Update inline documentation for the config field and the env-var-backed source field to reflect the new default and behavior.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| /// Maximum number of request IDs whose logs are held in `held_logs` waiting for durable | ||
| /// execution context. Set to 0 to disable log holding; logs will be flushed immediately | ||
| /// without durable execution context enrichment. Defaults to 0 until the tracer-side | ||
| /// durable execution support is released; set to 50 to re-enable enrichment. | ||
| /// without durable execution context enrichment. Only affects durable functions: once the | ||
| /// extension learns the function is not durable, logs bypass holding regardless of this value. |
There was a problem hiding this comment.
Nit, but I think this is worth clarifying.
litianningdatadog
left a comment
litianningdatadog
left a comment
There was a problem hiding this comment.
Approved, but you may want to address the comment made
What if the customer has upgraded their extension but not their tracer? I think we should really consider this possibility. Also to support customers who do not have apm and therefore no tracer at all. |
Background
DD_LAMBDA_DURABLE_FUNCTION_LOG_BUFFER_SIZEmeans how many logs (in terms of number of invocations) the extension buffers to wait for enrichment. If it's N (which is 5 right now), then the extension holds all logs for up to N invocations, waits for traces for these invocations to be sent from the tracer, then uses the metadata from the traces to enrich the logs.If the function is not a durable function, then the extension holds all logs (for up to N invocations) at cold start. Then, as soon as it receives the
platform.InitStartevent and learns that the function is not a durable function, it releases all held logs and no longer holds logs.Ideally, its default value should be a non-zero value, so the behavior is correct for both durable functions and non-durable functions, i.e.:
Actually, the default was 5 at the beginning. However, when the log holding logic was released, tracer-side changes that adds the necessary metadata to traces had not been released. As a result, the extension never gets the metadata it's waiting for, and some logs are somehow dropped. Therefore, #1239 changes the default back to 0 as a temporary mitigation of this issue.
Now, all the tracer-side changes (
datadog-lambda-js,datadog-lambda-python) have been released, so we can change the default back to 5. The dropped-logs problem won't happen as long as the user uses the latest version for the tracer.Overview
Changes the default of
lambda_durable_function_log_buffer_size(
DD_LAMBDA_DURABLE_FUNCTION_LOG_BUFFER_SIZE) from0to5, so instrumenting a durablefunction no longer requires setting this env var to get logs enriched with durable execution
context.
Testing
cargo test --lib— 541 passed, 0 failedcargo clippy --all-targets -- -D warnings— cleancargo fmt --check— cleanNo manual test on a live durable function yet.
Existing integration tests
Non-durable functions — covered. These suites ensures logs for non-durable functions are flushed correctly:
integration-tests/tests/on-demand.test.ts— node/python/java/dotnet, 2 invocations each;asserts the
Hello world!log is retrievable by request ID for both the cold and warminvocation.
integration-tests/tests/lmi.test.ts— Managed Instance mode; asserts logs exist and thatthe
Hello world!log is present.integration-tests/tests/auth.test.ts— assertslogs.length > 0under delegated auth.Durable functions — not covered. However, we have done lots of manual tests using
DD_LAMBDA_DURABLE_FUNCTION_LOG_BUFFER_SIZE == 5and ensured this works well for durable functions.Next steps:
Update onboarding doc saying if extension v100+ is used, then the user doesn't need to set
DD_LAMBDA_DURABLE_FUNCTION_LOG_BUFFER_SIZEto 5 since it's 5 by default.🤖 Partially generated with Claude Code