Skip to content

Make local compile backend configurable - #11

Closed
ryan-li-ryanllli-aws wants to merge 1 commit into
mainfrom
ryanllli/configurable-local-compile-backend
Closed

ryan-li-ryanllli-aws wants to merge 1 commit into
mainfrom
ryanllli/configurable-local-compile-backend

Conversation

@ryan-li-ryanllli-aws

Copy link
Copy Markdown
Collaborator

Summary

  • add model.local_compile_backend with an inductor default
  • use that backend for every enabled local compile region
  • late-bind FlexAttention and block-mask helpers so custom device backends do not require a process-wide torch.compile patch
  • preserve inductor-specific options only for inductor and keep the deterministic ROCm eager fallback

Why

TorchTitan-owned compile sites currently select the default backend before model configuration is available. A custom device backend therefore has to patch torch.compile process-wide, which also affects user and third-party compile calls. This change gives those sites one scoped backend setting while preserving current default behavior.

Test plan

  • Python 3.11 plus nightly CPU PyTorch: 53 focused CPU tests passed
  • formatting, flake8, pydoclint, codespell, AST, whitespace, merge-conflict, license, and link checks passed
  • git diff --check passed

@ryan-li-ryanllli-aws

Copy link
Copy Markdown
Collaborator Author

Superseded by #13, which retains the lazy helper reset/rebinding design and includes HF config initialization, typed torch.compile calls, compatibility defaults, and engine propagation coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant