From NVIDIA Megatron-LM for visibility - #18

Open

RaymondLi0 wants to merge 7580 commits into

bigcode-project:multi-query-attentionfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Jul 14, 2026

build: Update Transformer Engine to 2.17 (#5680)
balasaajay
authored
97b25d9
Mamba prefix caching fixes (#5502)

santhnm2
and
claude
authored
e5344ab
Exercise nested MFSDP CUDA graph capture (#5796)
wujingyue
authored
a327943

Commits on Jul 15, 2026

Implement Quantile Balancing in MoE (#5349)
Mellonta
authored
33706f9
Unset NCCL overrides for MFSDP v2 tests (#5794)
wujingyue
authored
f8c9911
Fix MegatronFSDP root module hook dispatch (#5808)
wujingyue
authored
4bf7fca
Pin cudnn-fe and cuTeDSL version (#5812)
balasaajay
authored
834abc1
chore: rotate oncall schedule
github-actions[bot]
committed
5abab93
Inference: Add the nemotron_v3 reasoning parser (#5634)
sidsingh-nvidia
authored
82e9dc6
Missing moe_router_dtype causes unexpected downcast in ModelOpt example (#5810)

jinhangchoi
and
cspades
authored
ccdfa7d
Avoid FSDP unit terminology in MFSDP v2 (#5793)

wujingyue
and
svcnvidia-nemo-ci
authored
5e45ccf
Fix configured norm epsilon in MambaLayer (#5750)
shanhaoli
authored
7711346
[refactor] Common combined-1F1B schedule-plan base (1/4 of #4798) (#4941)

Connor-XY
and
claude
authored
ffbe018
chore(tests): AUT-851 move NCCL defaults from run_ci_test.sh to conftest (#5826)
svcnemo-autobot
authored
ecf5547
Set num_splits to 0 for FA4 inference (#5804)
santhnm2
authored
8bf7365
ci: Enhance nightly/mr/weekly error reporting (#5831)
balasaajay
authored
06bb399
feat(docker): Add NCCL installation script and install NCCL 2.30.4 (#5815)
balasaajay
authored
3927d9b
IMA fix by making the copy of book keeping buffer to GPU blocking (#5715)
shanmugamr1992
authored
53a2dd5

Commits on Jul 16, 2026

Add GPTModel to HybridModel migration guide (#5698)
Phlip79
authored
802da56
Pair frozen FSDP backward hooks (#5710)
wujingyue
authored
38626c2
Clarify NVIDIA email signing guidance (#5699)
wujingyue
authored
c2f0df5
Overlap FSDP communication with compute (#5719)
wujingyue
authored
d981f66
chore(skills): add Regent Open Plugin manifest (#5840)

ko3n1g
and
claude
authored
7b2a97b
chore(skills): remove Open Plugin manifest (superseded) (#5842)
ko3n1g
authored
2aa3645
return prefix cache hits data from the chat completions api (#5609)
sidsingh-nvidia
authored
edd4562
Refactor data parallel coordinator to enable modular handlers (#5550)
santhnm2
authored
f8e1ac6
Pass device IDs to cleanup barrier (#5702)
wujingyue
authored
407ea4f
Fix averaging for MoE z-loss metric tracking (#3199)

Marks101
and
Markus Schnös
authored
4abeecc
test(mfsdp): AUT-881 mark test_overlaps_communication_and_compute flaky (#5848)
svcnemo-autobot
authored
9a67ea0
Inference: Extend default cuda-graph coverage to 512 tokens (#5797)
sidsingh-nvidia
authored
740c16e

Commits on Jul 17, 2026

Commits on Jul 18, 2026

Print important dependencies (#5814)
gautham-kollu
authored
dc57c2b
Test zero-CTA copy-engine all-gather (#5858)

wujingyue
and
claude
authored
f258d4f
Route Lion through DistributedOptimizer and support single-moment checkpointing (#5742)

deepakn94
and
claude
authored
b4ad280

Commits on Jul 19, 2026

Fix broken remove_sharded_tensors public API and re-enable its unit test (#5759)

authored
3219f38

Commits on Jul 20, 2026

Commits on Jul 21, 2026

Commits on Jul 22, 2026

Commits on Jul 23, 2026

Make LRU prefix caching eviction policy only evict child blocks (#5822)
santhnm2
authored
52cf20d
Inference: Reduce mamba scratch space size by an order of magnitude. (#5863)
sidsingh-nvidia
authored
411a5d8
Fix TE grouped MLP fused main-grad setup (#5209)
Wong4j
authored
c5398de
fix(ci): AUT-957 support golden checks in merge queue (#5989)

authored
bb5647a
Fix formatting error in qwen3_30b_a3b config (#5978)

jon-barker
and
Jon Barker
authored
6cd6ea5
Batch-invariant train/inference logprob parity (#5897)
wdykas
authored
51e915a
Move FSDP model weight sync to optimizer post-step (#5949)
wujingyue
authored
15d4b34
fix: allow mtp_num_layers=0 with overlap_moe_expert_parallel_comm (#5912)
cuichenx
authored
acd0e35
Port Multi-Latent Attention to `HybridModel` (#4452)
janEbert
authored
4464d1c
docs(skills): clarify container::lts is the older LTS PyTorch base (#6008)

ko3n1g
and
claude
authored
4f72cc0
[main] add thd sequence packing dispatcher support for main (#5008)
HaochenYuan
authored
81770cb

Commits on Jul 24, 2026

Commits on Jul 25, 2026

Commits on Jul 26, 2026

fix(inference): MCORE-536 report dropped prompt token lengths (#6051)
svcnemo-autobot
authored
7ec1bce

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Commits on Jul 30, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
542ff53
Reorganize a few docs (#6109)
Phlip79
authored
9436656
Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back (#6094)
fanshiqing
authored
8e57bb6
test: add async scheduling inference coverage (#6104)

authored
354b5d2
test(ci): AUT-1173 set functional tests to ft_launcher (#6128)
svcnemo-autobot
authored
ab16414
Fix --freeze-all-layers for new model builder path + unit tests (#5926)
AAnoosheh
authored
ff6b92a
Fix inference overflow into dummy blocks (#5950)

tdene
and
jalbericiola
authored
3fd8f33
fix(tests): prefer staged release assets (#6149)
ko3n1g
authored
175b4cc
Fix Megatron FSDP optimizer under-update (#5976)

wujingyue
and
svcnvidia-nemo-ci
authored
ce8a8a7
Rebind MFSDP v2 sharded grads once per gradient reduction (#6041)

wujingyue
and
claude
authored
9d416ee
Fix MLA dynamic inference decode flag (#4902)
cuichenx
authored
0081c84
Normalize MFSDP data-parallel axes at fully_shard (#6067)
wujingyue
authored
dc3c9ab
Add Agent Compose package and runtime interface skeleton (#5937)
ISEEKYAN
authored
ab20a60
Fix GTP full-iteration CUDA-graph capture regression (#6077)

prajwal1210
and
claude
authored
ec53920
Harden dynamic prefix cache allocation - mamba mostly (#6091)
wdykas
authored
d04e7a0
ci(cache): AUT-1188 harden container cache selection (#6160)
svcnemo-autobot
authored
8543ae4

Commits on Jul 31, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
6513e3e
ci(nvskills): AUT-1188 fix reusable workflow startup failure (#6162)
svcnemo-autobot
authored
b19b1f4
Add MFSDP runtime schedule design (#6031)
wujingyue
authored
d3519d5
Feature/enable inference optimized qwen moe (#5700)

authored
8339200
Bs invar moe (#4871)

authored
57deb43
chore: update pytest version to 9.1.1 in pyproject.toml and uv.lock (#6170)
balasaajay
authored
4ac5ad0
Fix LRU block accounting for resumed requests (#5995)

lmcafee-nvidia
and
santhnm2
authored
ee38649
[Main] update moe single weight unit test (#6035)
zhongbozhu
authored
d378bfb
fix ckpt conversion issue (#6151)
dimapihtar
authored
801d7e6
Add determinism developer documentation (#5718)
Connor-XY
authored
07fdc08
Add MFSDP v2 MCore integration with FullyShardedOptimizer (#5865)

wujingyue
and
svcnvidia-nemo-ci
authored
313f6e9

Commits on Aug 1, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
4d129d8
Add streaming replies for dynamic engine (#5727)
nvcsathe
authored
1addfbd
deps: Update urllib3 to version 2.7.0 (#6182)
balasaajay
authored
2cbe71d
deps: Update black dependency to version 26.3.0 (#6180)
balasaajay
authored
f2d4dfa
Add meta-parameter support (#5369)
wujingyue
authored
1678db5
Fix tied parameters (#6164)
wujingyue
authored
714c24a

Commits on Aug 2, 2026

chore(ci): AUT-1231 bump preflight workflow template (#6198)
svcnemo-autobot
authored
9829b3f

Commits on Aug 3, 2026

Commits on Aug 4, 2026

[feat] Add reduce-scatter-with-fp32-accumulation support for GTP (#6200)
fanshiqing
authored
23b2ff2
chore(ci): AUT-1298 bump community workflow to v1.8.7 (#6219)
svcnemo-autobot
authored
5adbee2
fix: pretrain_hybrid.py --help (#6211)

paulwambergue
and
ilml
authored
0896be0
refactor(ci): migrate triage consumer to Cerno (#6250)

thomasdhc
and
codex
authored
22dfc28
Validate NIXL CUDA transports and share transfer agents (#6221)
wdykas
authored
46fe6d2
Add prefix-cache lifecycle hooks (#5860)
nvcsathe
authored
3fe1519

Commits on Aug 5, 2026

Commits on Aug 6, 2026

Commits on Aug 7, 2026

chore(ci): bump community workflow to v1.8.8 (#6322)
ko3n1g
authored
d62ff04
fix(moe): free activations immediately in full recompute mode (#5197)

authored
07a0de6
Allocate main_grad on the reduce-scatter stream (#6187)
wujingyue
authored
fa0758c
Encapsulate symmetric-memory AVG in DBuffer (#6169)
wujingyue
authored
cde3b5c
fix(ci): remove community workflow PAT (#6350)
ko3n1g
authored
c4968aa
Add multimodal argument and config plumbing (#6332)

matthieule
and
trintamaki
authored
628c2a5
Fix FSDP activation recompute prefetch (#6153)

shjwudp
and
wujingyue
authored
07a2319
Add Nemotron 3.5 Lightning GB200 nightly test (#6312)
Phlip79
authored
d2715d7

Commits on Aug 8, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
a6fb162
Exit non-zero when --exit-on-missing-checkpoint triggers (#6321)

shanmugamr1992
and
claude
authored
e04ed19
Divide expert gradients by the expert-parallel size (#5956)

wujingyue
and
claude
authored
f8ec15e
[feat] GTP+MTP (#6242)
fanshiqing
authored
79e6b6a
fix(mtp): scatter the MTP embedding when the model's embedding does not (#6256)

DanialTaheri
and
claude
authored
67700f7
Apply the layerwise param layout on the ModelBuilder DDP path (#6286)

deepakn94
and
claude
authored
0ced206
test: AUT-1363 prevent recipe fetch-token exposure (#6364)
svcnemo-autobot
authored
14499ef
fix(ci): AUT-1396 use one global external queue worker (#6366)
svcnemo-autobot
authored
ccf686e
Add generic interface for SSM inference (#5382)

santhnm2
and
shanmugamr1992
authored
d12f6c8

Commits on Aug 9, 2026

Add PyTorch DCP checkpoint save/load for MFSDP v2 (#6024)

ahmadki
and
wujingyue
authored
6518b75

Commits on Aug 10, 2026

Commits on Aug 11, 2026

Bump FLA to 0.5.1 and integrate GDN2 kernels (#5765)
xuantengh
authored
0c1fc4c
Update docstring for moe_router_topk_scaling_factor (#2178)

JavaZeroo
and
guihong-nv
authored
67a355e
Add construction-scoped MFSDP v2 fully shard options (#6127)
wujingyue
authored
78901d8
Support explicit Hugging Face finetune data files in finetune.sh (#6413)
jenchen13
authored
5e86dfa
Add megatron.core.telemetry base layer and observability guide (#6415)
deepakn94
authored
d7d7a6a
[NCCL EP] Support eager mode and overflow handling (#6229)
YangFei1990
authored
8971d19
Gate expensive work behind log levels (#5805)

shurkat-nvidia
and
claude
authored
db57c15
Fixes for SP + MTP all-gathers (#5748)

authored
8018459
TE Layernorm dtype guard with fp32 residuals (#6272)
mkhona-nvidia
authored
7a76dff
Treat LayerWise bucket_size as a soft minimum and fill would-be padding with real parameters (#5415)

deepakn94
and
claude
authored
e0ccd26
Gated Delta Product (GDP) implementation (#6074)

authored
8e70f69
fix(mfsdp): refresh V2 compute weights after optimizer step (#6336)

wujingyue
and
claude
authored
1d402ad
[core] Clarify GPT output_processor callback contract (#5739)

authored
c3d44dc
Validate MFSDP module phase transitions (#6359)
wujingyue
authored
03318a8
Refine MFSDP v2 configuration validation (#6333)
wujingyue
authored
80cda5d
Add hybrid layer config definitions (#6410)
Phlip79
authored
f3eb0df
Propagate RL runtime config updates to module configs (#6423)
Phlip79
authored
da18ed6
[fix] GTP+recompute: keep adjacent GTP recompute weights off the same gather buffer (#6407)
fanshiqing
authored
e22bcb0
Add MFSDP v2 design document (#6431)
wujingyue
authored
49a49a9

Commits on Aug 12, 2026