https://github.com/Dao-AILab/flash-attention

Score: 29.668073443613654

Last synced: 1 day ago
JSON representation

Repository metadata:

Fast and memory-efficient exact attention

Host: GitHub
URL: https://github.com/Dao-AILab/flash-attention
Owner: Dao-AILab
License: bsd-3-clause
Created: 2022-05-19T21:22:06.000Z (over 4 years ago)
Default Branch: main
Last Pushed: 2026-10-29T03:51:33.000Z (5 days ago)
Last Synced: 2026-10-29T18:17:20.525Z (4 days ago)
Language: Python
Homepage:
Size: 22.2 MB
Stars: 25,221
Watchers: 145
Forks: 3,194
Open Issues: 1,394
Metadata Files:
- Readme: README.md
- License: LICENSE
- Authors: AUTHORS
- Agents: AGENTS.md
- Claude: CLAUDE.md

Owner metadata:

Name: Dao AI Lab
Login: Dao-AILab
Email:
Kind: organization
Description: We are an AI research group led by Prof. Tri Dao
Website: https://tridao.me/
Location: United States of America
Twitter:
Company:
Icon url: https://avatars.githubusercontent.com/u/139507659?v=4
Repositories: 1
Last Synced at: 2026-10-31T17:34:39.243Z
Profile URL: https://github.com/Dao-AILab

GitHub Events

Total

Commit comment event: 10
Create event: 82
Delete event: 51
Discussion event: 2
Fork event: 683
Issue comment event: 2170
Issues event: 584
Member event: 3
Pull request event: 379
Pull request review comment event: 206
Pull request review event: 271
Push event: 876
Release event: 17
Watch event: 6030
Total: 11364

Last Year

Create event: 58
Delete event: 44
Discussion event: 2
Fork event: 141
Issue comment event: 392
Issues event: 85
Member event: 2
Pull request event: 173
Pull request review comment event: 124
Pull request review event: 185
Push event: 473
Release event: 1
Watch event: 1104
Total: 2784

Committers metadata

Last synced: 2 days ago

Total Commits: 1,577
Total Committers: 219
Avg Commits per committer: 7.201
Development Distribution Score (DDS): 0.413

Commits in past year: 488
Committers in past year: 98
Avg Commits per committer in past year: 4.98
Development Distribution Score (DDS) in past year: 0.742

Name	Email	Commits
Tri Dao	t**q@g**m	926
Driss Guessous	3****g	92
Johnsonms	l**u@g**m	35
jayhshah	j**h@g**m	32
Pierce Freeman	p**n@g**m	25
Reuben Stern	1****s	23
Henry Tsang	h**g@m**m	21
Guilherme Leobas	g**s@g**m	18
rocking	C**i@a**m	15
Kirthi Shankar Sivamani	k**i@n**m	15
Ying Zhang	y**z@m**m	12
Dan Fu	d**9@g**m	11
Michael Melesse	m**e@g**m	9
Lequn Chen	l**n@p**i	9
Johnny	j**4@g**m	9
dan_the_3rd	4****d	7
Yunwei Li	y**3@g**m	7
Alkaid	g**o@m**m	6
oliver könig	o**g@n**m	6
Jean-Luc Duprat	j**d@a**g	5
Jane (Yuan) Xu	3****9	5
liangel-02	l**l@m**m	5
Markus Hoehnerbach	m**h@m**m	5
Ted Zadouri	t**i@g**m	5
timmy-feng	7****g	5
Aman Gupta Karmani	a**n@t**t	5
Xuechen Li	1****n	4
Grigory Sizov	g**v@f**m	4
NanoCode012	n**o@a**i	4
Phil Wang	l**s@g**m	4
and 189 more...

Issue and Pull Request metadata

Last synced: 2 days ago

Total issues: 1,063
Total pull requests: 771
Average time to close issues: 30 days
Average time to close pull requests: 19 days
Total issue authors: 846
Total pull request authors: 311
Average comments per issue: 2.19
Average comments per pull request: 1.21
Merged pull request: 307
Bot issues: 0
Bot pull requests: 0

Past year issues: 94
Past year pull requests: 394
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 20 days
Past year issue authors: 80
Past year pull request authors: 148
Past year average comments per issue: 2.38
Past year average comments per pull request: 1.15
Past year merged pull request: 108
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/Dao-AILab/flash-attention

Top Issue Authors

ziyuhuang123 (33)
phantaurus (10)
complexfilter (8)
endurehero (6)
FurkanGozukara (6)
TyGu1 (5)
gaodaheng (5)
DefTruth (5)
goldhuang (5)
janelu9 (5)
DianCh (4)
eric-tc-wong (4)
ArtificialZeng (4)
Johnsonms (4)
foreverpiano (4)

Top Pull Request Authors

drisspg (122)
Johnsonms (24)
rocking5566 (20)
imitater-dou (19)
johnnynunez (14)
guoriyue (13)
ipiszy (11)
yunweili3 (11)
ksivaman (10)
LucasWilkinson (9)
jayhshah (8)
reubenconducts (7)
micmelesse (7)
abcdabcd987 (7)
NanoCode012 (6)

Top Issue Labels

Top Pull Request Labels

size/M (1)
bug (1)
cla:yes (1)

Package metadata

Total packages: 21
Total downloads:
- pypi: 1,194,691 last-month
- conda: 177 total
Total docker downloads: 120,271
Total dependent packages: 44 (may contain duplicates)
Total dependent repositories: 317 (may contain duplicates)
Total versions: 305
Total maintainers: 12

pypi.org: flash-attn

Flash Attention: Fast and Memory-Efficient Exact Attention

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://flash-attn.readthedocs.io/
Licenses: BSD License
Latest release: 2.8.3.post2 (published 20 days ago)
Last Synced: 2026-10-31T17:39:35.383Z (2 days ago)
Versions: 79
Dependent Packages: 44
Dependent Repositories: 317
Downloads: 292,988 Last month
Docker Downloads: 120,271
Rankings:
- Stargazers count: 0.335%
- Dependent packages count: 0.475%
- Dependent repos count: 0.836%
- Downloads: 0.894%
- Average: 1.371%
- Forks count: 2.111%
- Docker downloads count: 3.575%
Maintainers (1)
- tridao

proxy.golang.org: github.com/Dao-AILab/flash-attention

Homepage:
Documentation: https://pkg.go.dev/github.com/Dao-AILab/flash-attention#section-documentation
Licenses: bsd-3-clause
Latest release: v2.8.3+incompatible (published about 1 year ago)
Last Synced: 2026-10-31T17:38:27.475Z (2 days ago)
Versions: 71
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 9.558%
- Average: 10.17%
- Dependent repos count: 10.783%

proxy.golang.org: github.com/dao-ailab/flash-attention

Homepage:
Documentation: https://pkg.go.dev/github.com/dao-ailab/flash-attention#section-documentation
Licenses: bsd-3-clause
Latest release: v2.8.3+incompatible (published about 1 year ago)
Last Synced: 2026-10-31T17:38:01.033Z (2 days ago)
Versions: 71
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 9.558%
- Average: 10.17%
- Dependent repos count: 10.783%

pypi.org: pi-flash-attn

Flash Attention: Fast and Memory-Efficient Exact Attention

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://pi-flash-attn.readthedocs.io/
Licenses: BSD License
Latest release: 2.8.2 (published about 1 year ago)
Last Synced: 2026-10-31T17:39:10.783Z (2 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 4 Last month
Rankings:
- Stargazers count: 0.482%
- Forks count: 0.771%
- Dependent packages count: 8.696%
- Average: 14.739%
- Dependent repos count: 49.007%
Maintainers (1)
- Jackmin

pypi.org: flash-attn-wheels-test

Flash Attention: Fast and Memory-Efficient Exact Attention

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://flash-attn-wheels-test.readthedocs.io/
Licenses: BSD License
Latest release: 2.0.8.post17 (published about 3 years ago)
Last Synced: 2026-10-31T17:38:26.209Z (2 days ago)
Versions: 14
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 41 Last month
Rankings:
- Stargazers count: 0.36%
- Forks count: 2.344%
- Dependent packages count: 7.565%
- Average: 19.918%
- Dependent repos count: 69.402%
Maintainers (1)
- tridao

pypi.org: fa4

Flash Attention CUTE (CUDA Template Engine) implementation

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://fa4.readthedocs.io/
Licenses: BSD 3-Clause License
Latest release: 4.0.0b3 (published 8 months ago)
Last Synced: 2026-10-31T17:38:42.896Z (2 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 28 Last month
Rankings:
- Dependent packages count: 7.716%
- Average: 25.672%
- Dependent repos count: 43.628%
Maintainers (1)
- tridao

pypi.org: sgl-fa4

Flash Attention CUTE (CUDA Template Engine) implementation

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://sgl-fa4.readthedocs.io/
Licenses: BSD 3-Clause License
Latest release: 4.0.5 (published 8 months ago)
Last Synced: 2026-10-31T17:38:00.911Z (2 days ago)
Versions: 5
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 40,421 Last month
Rankings:
- Dependent packages count: 7.729%
- Average: 25.717%
- Dependent repos count: 43.704%
Maintainers (2)
- BBuf
- rainj

pypi.org: tokenspeed-fa4

Flash Attention CUTE (CUDA Template Engine) implementation

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://tokenspeed-fa4.readthedocs.io/
Licenses: BSD 3-Clause License
Latest release: 4.0.0.post20260923 (published about 1 month ago)
Last Synced: 2026-10-31T17:38:43.435Z (2 days ago)
Versions: 9
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 1,989 Last month
Rankings:
- Dependent packages count: 7.555%
- Average: 25.857%
- Downloads: 27.293%
- Dependent repos count: 42.723%
Maintainers (1)
- lightseek

pypi.org: flash-attn-4

Flash Attention CUTE (CUDA Template Engine) implementation

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://flash-attn-4.readthedocs.io/
Licenses: BSD 3-Clause License
Latest release: 4.0.0b37 (published 6 days ago)
Last Synced: 2026-10-31T17:39:44.902Z (2 days ago)
Versions: 35
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 859,207 Last month
Rankings:
- Dependent packages count: 7.872%
- Average: 26.189%
- Dependent repos count: 44.505%
Maintainers (2)
- tridao
- drisspg

pypi.org: qifa

flash-attn quick installer

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://qifa.readthedocs.io/
Licenses: BSD-3-Clause
Latest release: 0.1.1 (published 11 months ago)
Last Synced: 2026-10-31T17:39:35.190Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 13 Last month
Rankings:
- Dependent packages count: 8.34%
- Average: 27.74%
- Dependent repos count: 47.14%
Maintainers (1)
- voidful

pypi.org: crossing-flash-attn

Flash Attention: Fast and Memory-Efficient Exact Attention

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://crossing-flash-attn.readthedocs.io/
Licenses: BSD License
Latest release: 2.6.3 (published about 2 years ago)
Last Synced: 2026-10-31T17:40:03.040Z (2 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 10.467%
- Average: 34.696%
- Dependent repos count: 58.925%
Maintainers (1)
- zhuping

pypi.org: hstu-attn

HSTU Attention: The HSTU structure mainly serves generative recommendation

Homepage: https://github.com/Dao-AILab/flash-attention
Documentation: https://hstu-attn.readthedocs.io/
Licenses: BSD License
Latest release: 2.5.9.post1 (published over 2 years ago)
Last Synced: 2026-10-31T17:39:51.016Z (2 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 10.542%
- Average: 34.954%
- Dependent repos count: 59.365%
Maintainers (1)
- binc

anaconda.org: flash-attn

FlashAttention is a fast and memory-efficient implementation of the exact attention mechanism for Transformer models. It supports Ampere, Ada Lovelace, Hopper, and Blackwell NVIDIA GPUs with fp16/bf16 precision.

Homepage: https://github.com/Dao-AILab/flash-attention
Licenses: BSD-3-Clause
Latest release: 2.8.3 (published 8 months ago)
Last Synced: 2026-06-23T15:01:31.411Z (4 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 121 Total
Rankings:
- Dependent packages count: 38.177%
- Average: 41.173%
- Dependent repos count: 41.627%
- Downloads: 43.716%

anaconda.org: flash-attn-layer-norm

Homepage: https://github.com/Dao-AILab/flash-attention
Licenses: BSD-3-Clause
Latest release: 2.8.3 (published 8 months ago)
Last Synced: 2026-07-27T23:01:27.198Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 28 Total
Rankings:
- Dependent packages count: 38.177%
- Average: 41.173%
- Dependent repos count: 41.627%
- Downloads: 43.716%

anaconda.org: flash-attn-fused-dense

Homepage: https://github.com/Dao-AILab/flash-attention
Licenses: BSD-3-Clause
Latest release: 2.8.3 (published 8 months ago)
Last Synced: 2026-07-27T23:01:27.501Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 28 Total
Rankings:
- Dependent packages count: 38.177%
- Average: 41.173%
- Dependent repos count: 41.627%
- Downloads: 43.716%

nixpkgs-unstable: python313Packages.flash-attn-4

CuTeDSL-based implementation of FlashAttention for Hopper and Blackwell GPUs

Homepage: https://github.com/Dao-AILab/flash-attention/tree/main/flash_attn/cute
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn-4/default.nix#L71
Licenses: BSD-3-Clause
Latest release: 4.0.0.beta37 (published 2 days ago)
Last Synced: 2026-11-01T10:53:23.372Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- GaetanLepage
- Prince213

nixpkgs-unstable: python313Packages.flash-attn

Official implementation of FlashAttention and FlashAttention-2

Homepage: https://github.com/Dao-AILab/flash-attention/
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn/default.nix#L159
Licenses: BSD-3-Clause
Latest release: 2.8.2 (published 9 months ago)
Last Synced: 2026-11-01T10:53:22.609Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- jherland

nixpkgs-unstable: python313Packages.flash-attn-3

Official implementation of FlashAttention-3

Homepage: https://github.com/Dao-AILab/flash-attention/blob/main/hopper
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn-3/default.nix#L107
Licenses: BSD-3-Clause
Latest release: 3.0.0-unstable-2026-06-10 (published 2 days ago)
Last Synced: 2026-11-01T10:53:22.973Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- GaetanLepage

nixpkgs-unstable: python314Packages.flash-attn-4

CuTeDSL-based implementation of FlashAttention for Hopper and Blackwell GPUs

Homepage: https://github.com/Dao-AILab/flash-attention/tree/main/flash_attn/cute
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn-4/default.nix#L71
Licenses: BSD-3-Clause
Latest release: 4.0.0.beta37 (published 1 day ago)
Last Synced: 2026-11-01T11:28:39.007Z (1 day ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- GaetanLepage
- Prince213

nixpkgs-unstable: python314Packages.flash-attn-3

Official implementation of FlashAttention-3

Homepage: https://github.com/Dao-AILab/flash-attention/blob/main/hopper
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn-3/default.nix#L107
Licenses: BSD-3-Clause
Latest release: 3.0.0-unstable-2026-06-10 (published 1 day ago)
Last Synced: 2026-11-01T11:28:38.673Z (1 day ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- GaetanLepage

nixpkgs-unstable: python314Packages.flash-attn

Official implementation of FlashAttention and FlashAttention-2

Homepage: https://github.com/Dao-AILab/flash-attention/
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/flash-attn/default.nix#L159
Licenses: BSD-3-Clause
Latest release: 2.8.2 (published 9 months ago)
Last Synced: 2026-11-01T11:28:38.339Z (1 day ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- jherland

Dependencies

.github/workflows/publish.yml actions

Jimver/cuda-toolkit v0.2.11 composite
actions/checkout v3 composite
actions/create-release v1 composite
actions/setup-python v4 composite
actions/upload-release-asset v1 composite
joutvhu/get-release v1 composite

training/Dockerfile docker

base latest build
env-${PERSONAL} latest build
env-0 latest build
nvcr.io/nvidia/pytorch 22.12-py3 build

csrc/fused_dense_lib/setup.py pypi

csrc/layer_norm/setup.py pypi

flash_attn/pyproject.toml pypi

setup.py pypi

einops *
ninja *
packaging *
torch *

tests/pyproject.toml pypi

flash_attn/cute/pyproject.toml pypi

apache-tvm-ffi >=0.1.5,<0.2
einops *
nvidia-cutlass-dsl ==4.3.4
torch *
torch-c-dlpack-ext *
typing-extensions *

.github/workflows/_build.yml actions

Jimver/cuda-toolkit v0.2.29 composite
actions/cache/restore v4 composite
actions/cache/save v4 composite
actions/checkout v5 composite
actions/setup-python v5 composite
actions/upload-release-asset v1 composite
joutvhu/get-release v1 composite
pierotofy/set-swap-space v1.0 composite

.github/workflows/build.yml actions

.github/workflows/pre-commit.yaml actions

actions/checkout v5 composite
actions/setup-python v6 composite
pre-commit/action v3.0.1 composite

hopper/setup.py pypi

einops *
ninja *
packaging *
torch *

.github/workflows/ci.yml actions

./.github/actions/gpu-test * composite
actions/checkout v4 composite
docker/login-action v3 composite

.github/workflows/publish-fa4.yml actions

actions/checkout v4 composite
actions/download-artifact v4 composite
actions/setup-python v5 composite
actions/upload-artifact v4 composite
pypa/gh-action-pypi-publish release/v1 composite
softprops/action-gh-release v2 composite

tools/ci/docker/Dockerfile docker

ubuntu 24.04 build

.github/actions/gpu-test/action.yml actions