Skip to content

feat(convolution): add accelerated split weight gradients - #1083

Merged
laggui merged 3 commits into
tracel-ai:mainfrom
EthanAlmloff:pr/accelerated-split-wgrad
Nov 6, 2026
Merged

laggui merged 3 commits into
tracel-ai:mainfrom
EthanAlmloff:pr/accelerated-split-wgrad

Conversation

@EthanAlmloff

Copy link
Copy Markdown
Contributor

Instructions

  • Create a new branch or fork of the burn repo
  • Update the main Cargo.toml with this PR hash.
  • Fix any broken tests or compilation errors in burn.
  •  Submit a PR in burn with your fixes and link it here.

Changes

Extend the split-weight gradient pipeline to support an accelerated kernel

Burn PR

tracel-ai/burn#6319

@EthanAlmloff

Copy link
Copy Markdown
Contributor Author

@laggui here is the cubek change for the accelerated split wgrad burn pr

Comment on lines +480 to +481
#[cube(comptime)]
channels: u32,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

channels receives padded_channels, which is a problem extent rather than a kernel setting. Marking it as comptime creates separate JIT entries for channel counts that could otherwise a kernel.

This field should be runtime, consistent with other problem sizes.

/// The output has shape `[out_channels, ...kernel, in_channels]`, with rank `N + 2`.
/// `SPATIAL_LAST` stores partial columns with spatial axes inside the channel axis.
/// F16 and BF16 partial sums stay F32 until the final output write.
pub fn launch_accelerated<const N: usize, const SPATIAL_LAST: bool>(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be a named strategy value instead of a boolean const generic, following the existing convolution APIs?

Passing it as a cubecl comptime argument would retain specialized GPU code and let callers select the ordering through a single launch call.

@EthanAlmloff
EthanAlmloff requested a review from laggui November 5, 2026 20:14

@laggui laggui left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM!

Can you update the burn PR?

@laggui laggui changed the title add accelerated split wgrad feat(convolution): add accelerated split weight gradients Nov 6, 2026
@laggui
laggui merged commit e669b06 into tracel-ai:main Nov 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants