Skip to content

Add handling for global parameters - #15

Merged
ghanse merged 2 commits into
mainfrom
global-parameters
Jul 23, 2026
Merged

Add handling for global parameters#15
ghanse merged 2 commits into
mainfrom
global-parameters

Conversation

@ghanse

@ghanse ghanse commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Changes

Add handler methods to resolve global parameters from ADF resources.

Parameter is controlled by the user via prompting. The choice is applied to an entire ADF resource during conversion. Users can choose to:

  • Inject resolved parameter values directly into the code or configuration during conversion
  • Create bundle variables and parameterize code or configuration with bundle variable references

Linked issues

N/A

Tests

  • manually tested
  • added unit tests
  • added integration tests

Copilot AI review requested due to automatic review settings July 23, 2026 15:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end support for Azure Data Factory (ADF) factory global parameters in flowx, letting users choose whether global references are baked in as literals or hoisted to Databricks Asset Bundle (DAB) variables that can be overridden at deploy time.

Changes:

  • Added a global_parameter_resolution policy (literal vs bundle_variable) that affects expression resolution, IR rewriting, bundling, and setup guidance.
  • Extended ARM template ingestion to load /globalparameters resources and resolve parameters('X') references from the sibling parameters file.
  • Threaded hoisted globals through the workflow/bundler so databricks.yml declares variables and SETUP.md documents overrides (including plaintext-secret caveats), with unit tests covering the behavior.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/flowx/translator/engine.py Adds the global-parameter resolution policy, passes it through rewrite passes, and hoists referenced globals into Pipeline.bundle_variables.
src/flowx/parser/expression_parser.py Updates global parameter expression resolution and extends notebook interpolation lowering to bridge ${var.X} via widgets.
src/flowx/parser/ir_rewriter.py Ensures whole-IR @{...} rewriting has access to factory globals and the resolution policy.
src/flowx/parser/adf_loader.py Adds ARM parameters-file discovery + substitution and loads standalone /globalparameters resources.
src/flowx/models/ir.py Adds Pipeline.bundle_variables and threads TranslationContext.global_parameter_resolution through context reconstruction.
src/flowx/translator/activity_translators/for_each.py Threads the policy through ForEach child contexts.
src/flowx/preparer/workflow_preparer.py Copies Pipeline.bundle_variables onto PreparedWorkflow for downstream bundling.
src/flowx/bundler/dab_writer.py Declares hoisted globals in databricks.yml and binds hoisted widget defaults to ${var.X} during base-parameter augmentation.
src/flowx/bundler/prereqs_writer.py Adds a SETUP.md section listing hoisted globals + deploy-time override commands and a security note.
src/flowx/adapter/constants.py Adds global_parameter_resolution input constant.
src/flowx/adapter/session.py Adds a convert-phase input option prompting users for global parameter resolution mode.
skills/flowx-convert/SKILL.md Documents the new --global-parameter-resolution CLI flag and behavior.
docs/content/docs/options.mdx Documents the new global_parameter_resolution option and its security implications.
AGENTS.md Adds repo guidance for naming/docstrings/comments conventions.
tests/unit/test_translators.py Adds engine-level tests asserting literal vs hoisted-global behavior, including ForEach and whole-IR rewrite cases.
tests/unit/test_expression_parser.py Adds expression parser tests for bundle-variable behavior and policy propagation through context rebuilds.
tests/unit/test_bundler.py Adds tests asserting hoisted globals are declared in databricks.yml, show up in SETUP.md, and bind widgets correctly.
tests/unit/test_adf_loader.py Adds tests for ARM template globalparameters ingestion and parameters-file resolution helpers.
tests/unit/test_adapter.py Updates adapter tests to assert the new option is present and defaults to literal.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 271 to 273
if result.kind == "literal":
return result.value
if result.kind == "dab_ref":

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved

@ghanse
ghanse merged commit 87829c3 into main Jul 23, 2026
3 checks passed
@ghanse
ghanse deleted the global-parameters branch July 23, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants