Skip to content

@daft.method / @daft.method.batch silently ignore max_retries and on_error #6710

@everettVT

Description

@everettVT

Summary

@daft.method and @daft.method.batch accept max_retries and on_error keyword arguments in their public signatures (`daft/udf/init.py:497-505`, `547-555`). `mark_cls_method` writes them to the method object as `MAX_RETRIES_ATTR` / `ON_ERROR_ATTR` (`daft/udf/udf_v2.py:409-410`).

But they're never read back. `Func._from_method` (`daft/udf/udf_v2.py:131-168`) reads `UNNEST_ATTR`, `BATCH_ATTR`, `BATCH_SIZE_ATTR`, and `RETURN_DTYPE_ATTR` from the method, but not `MAX_RETRIES_ATTR` / `ON_ERROR_ATTR` — it uses the values that `wrap_cls` passed in, which always come from the class-level `@daft.cls` decorator.

Result: a user who writes

```python
@daft.cls(max_retries=0)
class Pipeline:
@daft.method(max_retries=5, on_error="log")
def call_flaky_api(self, x): ...
```

gets `max_retries=0` and `on_error="raise"`, contrary to what the kwargs in the public signature suggest.

Expected

Either:

  1. `Func._from_method` reads `MAX_RETRIES_ATTR` / `ON_ERROR_ATTR` from the method and uses them when set (preserving class-level defaults when unset), or
  2. The kwargs are removed from the `@daft.method` / `@daft.method.batch` public signatures so callers don't think they work.

Option 1 matches the existing pattern for `unnest` / `batch_size` / `return_dtype` and matches what users are likely to expect (per-method override of class defaults).

Test gap

`tests/udf/test_cls.py` has no coverage for `max_retries` / `on_error` at all — class-level or per-method. Both should be added when this is fixed.

Doc impact

Docs in #6702 originally claimed per-method overrides worked; that has been corrected to flag the kwargs as currently no-op pending this fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions