Skip to content

Collapse map-data onto a single writer API - #307

Open
amc-corey-cox wants to merge 1 commit into
mainfrom
writer-api-collapse
Open

Collapse map-data onto a single writer API#307
amc-corey-cox wants to merge 1 commit into
mainfrom
writer-api-collapse

Conversation

@amc-corey-cox

Copy link
Copy Markdown
Contributor

Closes #302.

map-data used two different writer stacks depending on whether -O was passed. The -O path built a MultiStreamWriter; the single-output path used the function-based writers and a hand-rolled header-rewrite block that only worked because tsv_stream/csv_stream stash their headers on the function object.

This routes both through MultiStreamWriter — the single-output case is just the multi-output case with no additional outputs. MultiStreamWriter already handles the tabular header rewrite generically, so the inline rewrite and the get_stream_writer / rewrite_header_and_pad / nullcontext / os imports drop out.

The function-based API (yaml_stream, json_stream, jsonl_stream, tsv_stream, csv_stream, get_stream_writer, and the TabularStreamWriter.stream alias) is now unused internally. Per the decision on #302 it is deprecated, not deleted: the names stay exported from linkml_map.writers, now emitting a DeprecationWarning that matches the repo's existing "removed in a future version" wording. The raw line reduction lands later when the deprecation cycle completes.

Testing

  • New CLI-level test for tabular output over heterogeneous records (records with different keys) — a path that previously had no coverage. It asserts the resulting table order-independently, since output column order varies by environment.
  • TestDeprecatedFunctionAPI asserts each deprecated name warns; the redundant function-API tests are marked to ignore the warning so the suite stays quiet.
  • Full suite: 1078 passed, 4 skipped, 1 xfailed. Lint, format, and doctests clean.

Note for reviewers

While writing the CLI test I hit a pre-existing quirk unrelated to this change: tabular column order varies by environment (subprocess CLI emits insertion order, in-process CliRunner under pytest emits sorted order). The object keys and values are identical either way — only column ordering differs, which is cosmetic. The new test is order-independent because of it. Possibly worth its own issue.

map-data used two different writer stacks depending on whether -O was
passed: the multi-output path built a MultiStreamWriter, while the
single-output path used the function-based writers plus a hand-rolled
header-rewrite block that depended on tsv_stream/csv_stream stashing
state on the function object.

Route both through MultiStreamWriter — the single-output case is just
the multi-output case with no additional outputs. MultiStreamWriter
already handles the tabular header rewrite generically, so the inline
rewrite and the get_stream_writer/rewrite_header_and_pad/nullcontext/os
imports are gone.

The function-based API (yaml_stream, json_stream, jsonl_stream,
tsv_stream, csv_stream, get_stream_writer, TabularStreamWriter.stream)
is now unused internally. Rather than delete exported names outright,
deprecate them with DeprecationWarning matching the repo's existing
"removed in a future version" style.

Adds a CLI-level test for tabular output over heterogeneous records,
which previously had no coverage.
Copilot AI review requested due to automatic review settings July 29, 2026 21:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Unifies the map-data CLI output pipeline by routing both single-output and -O/--additional-output cases through the class-based MultiStreamWriter, while keeping the legacy function-based writer API available but deprecated (per #302). This reduces duplicated writer logic and centralizes tabular header-rewrite behavior.

Changes:

  • Refactor map-data streaming output to always use MultiStreamWriter (single output becomes the degenerate multi-output case).
  • Deprecate function-based writer entry points (*_stream, get_stream_writer, TabularStreamWriter.stream) by emitting DeprecationWarning, and adjust tests to either assert or silence warnings as appropriate.
  • Add a CLI-level TSV regression test covering heterogeneous records (late-discovered tabular columns) with order-independent assertions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/linkml_map/cli/cli.py Removes the separate single-output writer path and routes all streaming output through MultiStreamWriter.
src/linkml_map/writers/output_streams.py Adds a shared deprecation-warning helper and deprecates legacy function API + TabularStreamWriter.stream.
tests/test_cli/test_cli_tabular.py Adds an integration test ensuring TSV output rewrites/pads correctly when columns are discovered late.
tests/test_writers/test_output_streams.py Updates tests to handle deprecations (silence in legacy tests, assert warnings in deprecation tests).
Comments suppressed due to low confidence (1)

src/linkml_map/writers/output_streams.py:469

  • csv_stream() retains key_name for API compatibility but no longer uses it after switching to writer.process(chunks). To prevent Ruff unused-argument warnings (ARG001), mark the parameter with # noqa: ARG001 (or rename to _key_name in a future breaking release).
    """
    Convert chunks of objects to CSV format.

    .. deprecated::
        Use ``make_stream_writer(OutputFormat.CSV)`` and drive it via

Comment on lines 440 to +444
"""
Convert chunks of objects to TSV format.

.. deprecated::
Use ``make_stream_writer(OutputFormat.TSV)`` and drive it via
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

map-data uses a different writer stack depending on whether -O was passed

2 participants