Skip to content

Specify kernel version when loading CPU gemm_4bit_forward from the Hub#1972

Merged
matthewdouglas merged 2 commits into
bitsandbytes-foundation:mainfrom
jiqing-feng:kernel
Jun 12, 2026
Merged

Specify kernel version when loading CPU gemm_4bit_forward from the Hub#1972
matthewdouglas merged 2 commits into
bitsandbytes-foundation:mainfrom
jiqing-feng:kernel

Conversation

@jiqing-feng

Copy link
Copy Markdown
Contributor

Description

Loading the CPU gemm_4bit_forward kernel from kernels-community fails with newer
kernels (>= 0.11.1), which now require an explicit kernel API version (or a Hub
revision) to be passed to get_kernel:

A kernel version or revision must be specified. Use `version=<major>` for a stable
kernel API version or `revision=<branch/tag/commit>` for an explicit Hub revision.

Without it, get_kernel("kernels-community/quantization-bitsandbytes") raises and the
CPU 4-bit fused path silently falls back to the slower native implementation.

Fix

Pass the stable kernel API version=1 (an int, matching the repo's v1 build
branch) to get_kernel:

- gemm_4bit_forward_kernel = get_kernel(
-     "kernels-community/quantization-bitsandbytes"
- ).gemm_4bit_forward
+ gemm_4bit_forward_kernel = get_kernel(
+     "kernels-community/quantization-bitsandbytes", version=1
+ ).gemm_4bit_forward

Notes

  • version must be an integer (1), not a string; it resolves to the v1 build
    branch published for this kernel.
  • The load remains wrapped in try/except, so environments without kernels
    installed continue to fall back to the native CPU kernel.

Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
@jiqing-feng jiqing-feng marked this pull request as ready for review June 12, 2026 05:53

@matthewdouglas matthewdouglas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was on my TODO list, thank you! LGTM!

@github-actions

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@matthewdouglas matthewdouglas merged commit 936f60b into bitsandbytes-foundation:main Jun 12, 2026
99 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants