Search Packages

local-attention

An implementation of local windowed attention for language modeling

colt5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

alphafold3-pytorch

Implementation of Alphafold 3 from Google Deepmind in Pytorch

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

palm-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

bs-roformer

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

alphagenome-pytorch

Implementation of AlphaGenome, Deepmind's updated genomic attention model

calm-pytorch

Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind

locoformer

LocoFormer - Generalist Locomotion via Long-Context Adaptation

megabyte-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

tri-lbm

Implementation of the Large Behavioral Model architecture for dexterous manipulation from Toyota Research Institute

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

q-transformer

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

itransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

coconut-pytorch

Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch

taylor-series-linear-attention

Explorations into the recently proposed Taylor Series Linear Attention

block-recurrent-transformer-pytorch

Implementation of Block Recurrent Transformer - Pytorch

muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

longnet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

robotic-transformer-pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch