Skip to content

Add a backwards linear function to be used with the fused mla q up-proj - #3330

Open
chaseblock wants to merge 4 commits into
NVIDIA:mainfrom
chaseblock:lin_bwd
Open

Add a backwards linear function to be used with the fused mla q up-proj#3330
chaseblock wants to merge 4 commits into
NVIDIA:mainfrom
chaseblock:lin_bwd

Conversation

@chaseblock

Copy link
Copy Markdown
Contributor

Description

Please include a brief summary of the changes, relevant motivation and context.

Fixes # (issue)

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Signed-off-by: Chase Block <cblock@nvidia.com>
@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Aug 7, 2026
chaseblock added a commit to chaseblock/Megatron-LM that referenced this pull request Aug 7, 2026
Requires TE PR: NVIDIA/TransformerEngine#3330

Signed-off-by: Chase Block <cblock@nvidia.com>
@greptile-apps

greptile-apps Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds a reusable linear-backward helper for the fused MLA Q up-projection and exposes dgrad, wgrad, and bias-gradient results.

  • Adds an attention-layer delegate into the linear module.
  • Builds the backward arguments for BF16 and MXFP8 projection paths.
  • The BF16 bias-gradient path remains incomplete because the fused bgrad GEMM is invoked without a bias tensor.

Confidence Score: 4/5

The PR is not yet safe to merge because BF16 fused MLA training still loses the projection bias gradient.

The non-FP8 path delegates bias-gradient computation to the weight-gradient GEMM, but the new wrapper supplies no bias tensor, so the binding does not allocate or return db.

Files Needing Attention: transformer_engine/pytorch/module/linear.py

Important Files Changed

Filename Overview
transformer_engine/pytorch/module/linear.py Adds the shared backward helper, but its BF16 use_bias path returns no bias gradient because bias is always configured as None.
transformer_engine/pytorch/attention/fused_mla_q_uproj.py Adds a thin classmethod that forwards fused MLA backward arguments and return values to the new linear helper.

Reviews (3): Last reviewed commit: "Handle biad gradient in lin bwd wrapper." | Re-trigger Greptile

Comment thread transformer_engine/pytorch/module/linear.py Outdated
Signed-off-by: Chase Block <cblock@nvidia.com>
Signed-off-by: Chase Block <cblock@nvidia.com>
inputmat=x_saved,
weight_fp8=w_q,
saved_weight=w_q,
bias=None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 BF16 bias gradient remains missing

When a BF16 fused MLA backward call sets use_bias=True, this wrapper still passes bias=None; the non-FP8 path delegates bgrad to the weight-gradient GEMM, which does not allocate db without a bias tensor, causing backward_linear to return grad_bias=None and preventing the projection bias from being updated.

Knowledge Base Used: PyTorch Fused Modules (transformer_engine/pytorch/module)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant