Skip to content

fix(terraform): batch provider schema retrieval - #126

Merged
Jared Holgate (jaredfholgate) merged 1 commit into
mainfrom
jaredfholgate-provider-schema-batching
Sep 4, 2026
Merged

fix(terraform): batch provider schema retrieval#126
Jared Holgate (jaredfholgate) merged 1 commit into
mainfrom
jaredfholgate-provider-schema-batching

Conversation

@jaredfholgate

Copy link
Copy Markdown
Member

Summary

  • Batch statically evaluable provider_schema requests into one temporary Terraform configuration and one terraform init -upgrade / terraform providers schema cycle.
  • Reuse a config-scoped, concurrency-safe in-memory cache with provider-source normalization and duplicate request elimination.
  • Isolate conflicting constraints for the same provider source and fall back to individual retrieval after batch failures so valid providers and request-specific errors are preserved.
  • Keep plan-time/dynamic requests on the existing lazy path and preserve cancellation and CLI behavior.

Performance

Repeatable local benchmark (fake Terraform runner, no network):

individual: 3.000 terraform_cycles/op, 11.66-12.03 ms/op
batched:    1.000 terraform_cycles/op,  4.10-4.29 ms/op

A warmed end-to-end transform fixture using the AVM constraints (azurerm ~> 4.0, azapi ~> 2.4, random ~> 3.0) measured:

v0.1.10: 24.104 s
batched:  15.024 s
change:   -37.7%

The transformed main.tf files were byte-identical (matching SHA-256).

Persistent cache follow-up

A cross-process schema cache is intentionally not included. AVM uses open constraints, so caching by constraint could return stale schemas and violate -upgrade. A safe follow-up should initially cache exact versions only, key by normalized source + exact selected version + Terraform version + platform + schema format, and use inter-process locking with atomic publication.

Validation

  • go build github.com/Azure/mapotf
  • go test -v github.com/Azure/mapotf/...
  • golangci-lint run --timeout=3600s
  • gosec ./... reports the same pre-existing G204 finding on cmd/terraform_cmd_wrapper.go:26 as origin/main

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jaredfholgate
Jared Holgate (jaredfholgate) merged commit fef18fe into main Sep 4, 2026
7 checks passed
@jaredfholgate
Jared Holgate (jaredfholgate) deleted the jaredfholgate-provider-schema-batching branch September 4, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants