Skip to content

From NVIDIA Megatron-LM for visibility#18

Open
RaymondLi0 wants to merge 7254 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7254 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0 RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0 RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
chtruong814 and others added 27 commits May 22, 2026 11:37
@chtruong814
fix: Fix multi-node functional test phase sync (#4924)
4db6fa4 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@shanmugamr1992 @claude
Perf tests (#4917)
686aa8c 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@balasaajay
fix(cuda_graphs): handle TE 2.15 removal of FP8GlobalStateManager.set…
0beaa53 
…_skip_fp8_weight_update_tensor (#4874)
@Phlip79
Fix paged stashing test submodules lookup (#4925)
6c1bd6e
@sraman-rgb @yaox12 @gautham-kollu
Add TEFusedDenseMLP for Dense+Grouped GEMM fusion on SM100+ (#4318) (#…
fa7a23b 
…4786)

Co-authored-by: Siddhartha Raman S <sraman@login-lyris02.lyris.clusters.nvidia.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: gautham-kollu <gkollu@nvidia.com>
Co-authored-by: Siddhartha Raman S <sraman@login-lyris01.lyris.clusters.nvidia.com>
@WanZzzzzz @Phlip79
Fix mxfp8 param gather numerical issue when DP overlap is off (#4800)
5f79118 
Signed-off-by: qiyuw <qiyuw@nvidia.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@WanZzzzzz
[MXFP8/FP4-param-gather] Post processing after forced param AG in eval (
08bad7a 
#4562)

Signed-off-by: qiyuw <qiyuw@nvidia.com>
@balasaajay
ci: Update training script paths in BERT and T5 (#4939)
3fb34c6
@maanug-nv
Various training utils (#4872)
34560c4 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@chtruong814
ci: restore perf test torchrun logs (#4951)
f7f584d 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@deepakn94 @claude
Fix get_batch return order to ignore BlendedDataset provenance fiel…
4bd8bb3 
…ds (#4952)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ko3n1g
test(release): add release goldens for deepseekv3/nemotron3 and set t…
be2b2cd 
…p2pp2 exit-interval (#4932)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@github-actions
chore(beep boop 🤖): Bump (main) (2026-05-25)
3b2b6e7
@ko3n1g
test: enable NVTE_CUTEDSL_FUSED_GROUPED_MLP via pytest fixture (#4931)
2f754f4 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@chtruong814
Avoid offsetting functional test master port (#4973)
2bd9fd5 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@Devil1716
Fix elastification unwrap_model import (#4972)
4415119
@ko3n1g
test: re-enable paged stashing MoE tests (#4978)
432d76b 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g @claude
test(ci): re-enable 8experts2parallel_multi_dist_optimizer_instances_…
ff64743 
…1node (#4984)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@balasaajay
ci: Add support for MBridge job gating based on PR labels (#4926)
0e5cd0e
@ko3n1g @claude
test: re-enable test_pp2_create_cudagraphs_first_stage on TE 2.15+ (#…
08c368a 
…4985)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ko3n1g
fix(tests): initialize num_microbatches calculator in vision cudagrap…
6ce6fac 
…h tests (#4986)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@balasaajay
ci: Add allow_failure flag to gpt and moe recipes that are failing in…
859b719 
… nightlies (#4905)
@deepakn94 @claude
Drain predecessor reduce-scatter at dispatch time (#4940)
88e7ab0 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@balasaajay
nightly(ci): Update golden values for functional t5 tests (#4995)
e6b2bd8
@github-actions
chore: rotate oncall schedule
7521ecb
@yanring @buptzyb
@Victarry @Phlip79 @shifangx
[main] Refactor and Improve MoE Logginginit commit (#3431)
873678a 
Co-authored-by: Robin Zhang <robinz@nvidia.com>
Co-authored-by: Dennis Liu <denliu@nvidia.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: Shifang Xu <shifangx@nvidia.com>
@ko3n1g
ci: validate release branch-rules (#4929)
4e52a9e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
github-actions Bot and others added 30 commits June 24, 2026 10:06
@github-actions
chore: rotate oncall schedule
cc0c960
@shanmugamr1992 @claude
Add inference functions to support MCore-/MBridge- training refactor …
4d44e37 
…and remove legacy modelbuilder functions (#5169)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ko3n1g @claude
ci: launch GB200 unit tests via launch_on_gb200 marker (#5477)
5a256f3 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ko3n1g @claude
build: install flash_mla from source in the CI image (#5481)
82de1b8 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@HollowMan6
[split 2/4] Scale DSA indexer loss in pipeline schedules (#5244)
0b0d985 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
@ko3n1g
ci: check megatron.training imports in installation test (#5458)
0938eb7 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@muyihao
Fix merges_file kwarg name in HuggingFaceTokenizer (#5406)
239959b 
Signed-off-by: yanghao.666 <yanghao.666@jd.com>
@Phlip79
Automated community request assignment (#5147)
3330d12 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@ilml
Clean up training.py module header (dedupe + reorganize imports/globa…
311416f 
…ls) (#5469)

Signed-off-by: ilml <tolong@nvidia.com>
@yashaswikarnati
Thread process groups through training checkpoint paths (#5486)
9038381 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@Phlip79
Narrow oncall responsibilities (#5490)
5863721 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@yashaswikarnati
Add MIMO forward step and per-token loss for hetero training (#5376)
1c1d6b5 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@yashaswikarnati @claude
Add Nemotron6-MoE VLM model provider for MIMO example (#5374)
ea967a7 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@ko3n1g @claude
ci: auto-retry test-data download in container-build job (#5498)
7168714 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tdene
Force RL inference to CP=1 (#5423)
3bfd87b 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@deepakn94 @claude
Merge cu_seqlens across micro-batch for THD attention (#5454)
2a43e0d 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@HollowMan6
[split 4/4] Enable DSA CP and THD hooks (#5246)
da482cf 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
@sraman-rgb
Fix fused MLA down projection with tensor parallelism (#5383)
e1b8454 
Signed-off-by: Siddhartha Raman <sraman@nvidia.com>
@adityasingh2400
Fix NameError in is_flashinfer_min_version when check_equality=False (#…
8bafe7c 
…4961)

Signed-off-by: Aditya Singh <adisin650@gmail.com>
@Phlip79
Add hybrid FSDP unit module support (#4329)
da42015 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@ko3n1g @claude
fix: set DATA_PATH for moe-dynamic-inference recipe (#5506)
476228d 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yeyu-nvidia @claude
Add --qad-train-target {base|mtp|both} for QAD / MTP QAT (frozen-base…
c0d7848 
…, frozen-MTP, or co-train) (#4785)

Signed-off-by: Ye Yu <yeyu@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@zhongbozhu
[Main] Generalized fix for mxfp8 param gather (#5236)
0552f29 
Signed-off-by: Zhongbo Zhu <zhongboz@nvidia.com>
@wujingyue
Add CUDA graph training iteration test (#5417)
5949478 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
@lauradang
test: restore G/G + lag=19 for gpt_grpo_tp4_pp1_dp2_8b throughput tes…
847de23 
…ts (#5514)

Signed-off-by: Laura Dang <laurad@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
990ced9
@ko3n1g @claude
ci: cache-from a single coherent buildcache donor (#5509)
cbaa6eb 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@chtruong814
ci: Use GB300 for Github CI tests (#5520)
ed59a0a 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@ko3n1g @claude
ci: pin HF_HUB_CACHE to bind-mounted cache for gpt-oss-20b inference …
0ff7226 
…test (#5512)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@deepakn94 @claude
Add inter-document attention masking to GPTDataset (#5298)
f88b85f 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.