Attention Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

flashinfer-python

FlashInfer: Kernel Library for LLM Serving

flashinfer-cubin

FlashInfer: Kernel Library for LLM Serving

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sageattention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

dreamer4

Implementation of Danijar's latest iteration for his Dreamer line of work

performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

keras-transformer

Transformer implemented in Keras

keras-position-wise-feed-forward

Feed forward layer implemented in Keras

keras-cv-attention-models

Keras beit,caformer,CMT,CoAtNet,convnext,davit,dino,efficientdet,edgenext,efficientformer,efficientnet,eva,fasternet,fastervit,fastvit,flexivit,gcvit,ghostnet,gpvit,hornet,hiera,iformer,inceptionnext,lcnet,levit,maxvit,mobilevit,moganet,nat,nfnets,pvt,swin,tinynet,tinyvit,uniformer,volo,vanillanet,yolor,yolov7,yolov8,yolox,gpt2,llama2, alias kecam

lightning-core

Lightning Core: macOS-first CUDA-style runtime with Metal backend

rl4co

A PyTorch library for all things Reinforcement Learning (RL) for Combinatorial Optimization (CO)

activations

AReLU: Attention-based-Rectified-Linear-Unit

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

transfusion-pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

qwen

My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't released model code yet sooo...

native-sparse-attention-pytorch

Native Sparse Attention

labml-nn

🧑‍🏫 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit), optimizers (adam, radam, adabelief), gans(dcgan, cyclegan, stylegan2), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, diffusion, etc. 🧠

spear-tts-pytorch

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

longnet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

h-transformer-1d

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

numpy-ml

Machine learning, in numpy

genie2-pytorch

Genie2