Hugging Face's logo

yetter-ai
/
sage-attention

Kernels
Model card Files
xet
 Community
How to use from the
Use from the
Kernels library
# !pip install kernels

from kernels import get_kernel

kernel = get_kernel("yetter-ai/sage-attention")

This is the repository card of yetter-ai/sage-attention that has been pushed on the Hub. It was built to be used with the kernels library.

Build source: https://github.com/thu-ml/SageAttention Upstream commit: d1a57a546c3d395b1ffcbeecc66d81db76f3b4b5

How to use 

# make sure `kernels` is installed: `pip install -U kernels`
from kernels import get_kernel

kernel_module = get_kernel("yetter-ai/sage-attention")
sageattn = kernel_module.sageattn

sageattn(...)

Available functions

  • per_block_int8
  • per_warp_int8
  • sub_mean
  • per_channel_fp8
  • sageattn
  • sageattn_varlen
  • sageattn_qk_int8_pv_fp16_triton
  • sageattn_qk_int8_pv_fp16_cuda
  • sageattn_qk_int8_pv_fp8_cuda
  • sageattn_qk_int8_pv_fp8_cuda_sm90
  • sageattn3_blackwell

Benchmarks

No benchmark available yet.

Downloads last month
27
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support