https://github.com/huggingface/xet-core

huggingface-hub rust storage xet

Score: 29.295198497072395

Last synced: about 1 hour ago
JSON representation

Repository metadata:

xet client tech, used in huggingface_hub

Host: GitHub
URL: https://github.com/huggingface/xet-core
Owner: huggingface
License: apache-2.0
Created: 2024-09-10T20:57:29.000Z (almost 2 years ago)
Default Branch: main
Last Pushed: 2026-08-25T14:29:35.000Z (1 day ago)
Last Synced: 2026-08-25T14:37:02.178Z (1 day ago)
Topics: huggingface-hub, rust, storage, xet
Language: Rust
Homepage:
Size: 8.85 MB
Stars: 562
Watchers: 11
Forks: 88
Open Issues: 38
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Owner metadata:

Name: Hugging Face
Login: huggingface
Email:
Kind: organization
Description: The AI community building the future.
Website: https://huggingface.co/
Location: NYC + Paris
Twitter: huggingface
Company:
Icon url: https://avatars.githubusercontent.com/u/25720743?v=4
Repositories: 474
Last Synced at: 2026-08-22T02:26:21.821Z
Profile URL: https://github.com/huggingface

GitHub Events

Total

Create event: 468
Delete event: 382
Fork event: 27
Issue comment event: 404
Issues event: 95
Member event: 5
Public event: 1
Pull request event: 622
Pull request review comment event: 730
Pull request review event: 999
Push event: 1560
Release event: 65
Watch event: 230
Total: 5588

Last Year

Create event: 162
Delete event: 154
Fork event: 12
Issue comment event: 128
Issues event: 43
Pull request event: 147
Pull request review comment event: 270
Pull request review event: 311
Push event: 644
Release event: 16
Watch event: 100
Total: 1987

Committers metadata

Last synced: 1 day ago

Total Commits: 618
Total Committers: 28
Avg Commits per committer: 22.071
Development Distribution Score (DDS): 0.717

Commits in past year: 276
Committers in past year: 23
Avg Commits per committer in past year: 12.0
Development Distribution Score (DDS) in past year: 0.688

Name	Email	Commits
Hoyt Koepke	h**k@h**o	175
Di Xiao	d**i@h**o	134
Assaf Vayner	a**f@h**o	125
Rajat Arya	r****a	61
Joseph Godlewski	2****w	30
Brian Ronan	b**n@h**o	23
Sam Horradarn	s**n@h**o	16
Adrien	a**0@g**m	15
Yucheng Low	2****w	7
dependabot[bot]	4****]	4
Eliott C.	c**8@p**m	4
Ajit Banerjee	a**t@h**o	3
Hugo Larcher	h**r@g**m	2
Salman Chishti	1**i@g**m	2
Zach Nation	z**h@h**o	2
Krisztián Szűcs	s**n@g**m	2
Jared Sulzdorf	j**f@g**m	2
Arpit Jain	3****9	1
Erik Cederstrand	e**k@c**k	1
Expyron	5****n	1
Guspan Tanadi	3****i	1
Pauline Bailly-Masson	1****m	1
Qubitium-ModelCloud	q**m@m**i	1
Rupesh	w**h@g**m	1
SuperKenVery	3****y	1
crStiv	c**7@g**m	1
omahs	7****s	1
tison	w**6@g**m	1

Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 63
Total pull requests: 370
Average time to close issues: 19 days
Average time to close pull requests: 6 days
Total issue authors: 44
Total pull request authors: 24
Average comments per issue: 2.29
Average comments per pull request: 0.36
Merged pull request: 220
Bot issues: 0
Bot pull requests: 17

Past year issues: 29
Past year pull requests: 121
Past year average time to close issues: 22 days
Past year average time to close pull requests: 6 days
Past year issue authors: 23
Past year pull request authors: 16
Past year average comments per issue: 1.55
Past year average comments per pull request: 0.49
Past year merged pull request: 60
Past year bot issues: 0
Past year bot pull requests: 10

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/huggingface/xet-core

Top Issue Authors

jsulz (6)
FurkanGozukara (3)
assafvayner (3)
seanses (3)
danielhanchen (3)
coyotte508 (2)
kszucs (2)
ghchris2021 (2)
Wrufesh (2)
rajatarya (2)
Manamama (2)
cquil11 (1)
the01 (1)
yoooooolx (1)
clin1234 (1)

Top Pull Request Authors

hoytak (99)
assafvayner (90)
seanses (50)
rajatarya (27)
bpronan (19)
dependabot[bot] (16)
sirahd (14)
jgodlew (13)
XciD (11)
ylow (9)
coyotte508 (7)
znation (2)
port8080 (2)
marked23 (1)
ArthurZucker (1)

Top Issue Labels

bug (32)
client (6)
xet-core (6)
enhancement (4)
rust (4)
documentation (2)
hf-xet (2)
wasm (1)
cas (1)
good first issue (1)
performance (1)

Top Pull Request Labels

dependencies (16)
rust (15)
dependabot (1)

Package metadata

Total packages: 15
Total downloads:
- cargo: 4,275,903 total
- pypi: 309,316,693 last-month
- conda: 5,320 total
- homebrew: 637 last-month
Total docker downloads: 766,135
Total dependent packages: 0 (may contain duplicates)
Total dependent repositories: 0 (may contain duplicates)
Total versions: 224
Total maintainers: 9

proxy.golang.org: github.com/huggingface/xet-core

Homepage:
Documentation: https://pkg.go.dev/github.com/huggingface/xet-core#section-documentation
Licenses: apache-2.0
Latest release: v1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:59:34.163Z (1 day ago)
Versions: 86
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 4.881%
- Average: 5.045%
- Dependent repos count: 5.209%

crates.io: hf-xet

Client library and tooling for the Hugging Face Xet data storage system.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://docs.rs/hf-xet/
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:34:45.191Z (1 day ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 850,532 Total
Rankings:
- Downloads: 15.079%
- Dependent repos count: 15.496%
- Average: 17.014%
- Dependent packages count: 20.469%
Maintainers (2)
- assafvayner
- seanses

spack.io: py-hf-xet

Fast transfer of large files with the Hugging Face Hub.

Homepage: https://github.com/huggingface/xet-core
Licenses: []
Latest release: 1.4.3 (published 3 months ago)
Last Synced: 2026-08-25T14:34:40.734Z (1 day ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Average: 27.337%
- Dependent packages count: 54.674%

pypi.org: hf-xet

Fast transfer of large files with the Hugging Face Hub.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://huggingface.co/docs/hub/xet/index
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:35:03.411Z (1 day ago)
Versions: 90
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 309,316,693 Last month
Docker Downloads: 766,135
Rankings:
- Dependent packages count: 9.783%
- Average: 32.425%
- Dependent repos count: 55.067%
Maintainers (5)
- chaumond
- hf-xet
- rajatarya1
- seanses
- xcid

formulae.brew.sh: git-xet

Git LFS plugin that uploads and downloads using the Xet protocol

Homepage: https://github.com/huggingface/xet-core
Licenses: Apache-2.0
Latest release: 0.2.1 (published 6 months ago)
Last Synced: 2026-08-25T14:34:39.196Z (1 day ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 637 Last month
Rankings:
- Dependent packages count: 15.353%
- Average: 33.575%
- Downloads: 40.696%
- Dependent repos count: 44.676%

crates.io: xet-client

Client library for communicating with Hugging Face Xet storage servers. Use through the hf-xet crate.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://docs.rs/xet-client/
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:35:16.541Z (1 day ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 856,670 Total
Rankings:
- Dependent repos count: 15.91%
- Dependent packages count: 21.016%
- Average: 38.287%
- Downloads: 77.936%
Maintainers (2)
- assafvayner
- seanses

crates.io: xet-core-structures

Core data structures including MerkleHash, metadata shards, and Xorb objects.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://docs.rs/xet-core-structures/
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:34:49.061Z (1 day ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 856,965 Total
Rankings:
- Dependent repos count: 16.069%
- Dependent packages count: 21.227%
- Average: 42.831%
- Downloads: 91.196%
Maintainers (2)
- assafvayner
- seanses

crates.io: xet-data

Data processing pipeline for chunking, deduplication, and file reconstruction; used in the Hugging Face Xet client tools. Intended to be used through the API in the hf-xet package.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://docs.rs/xet-data/
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:35:04.167Z (1 day ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 854,471 Total
Rankings:
- Dependent repos count: 16.069%
- Dependent packages count: 21.227%
- Average: 42.831%
- Downloads: 91.196%
Maintainers (2)
- assafvayner
- seanses

crates.io: xet-runtime

Async runtime, configuration, logging, and utility infrastructure for the Hugging Face Xet client tools.

Homepage: https://github.com/huggingface/xet-core
Documentation: https://docs.rs/xet-runtime/
Licenses: Apache-2.0
Latest release: 1.6.0 (published 23 days ago)
Last Synced: 2026-08-25T14:34:44.466Z (1 day ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 857,265 Total
Rankings:
- Dependent repos count: 16.069%
- Dependent packages count: 21.227%
- Average: 42.831%
- Downloads: 91.196%
Maintainers (2)
- assafvayner
- seanses

anaconda.org: hf-xet

xet-core enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub. hf_xet - Python integration with Rust code, uses maturin to build hfxet Python package. Main integration with HF Hub Python package.

Homepage: https://github.com/huggingface/xet-core
Licenses: Apache-2.0
Latest release: 1.4.3 (published 2 months ago)
Last Synced: 2026-06-23T20:04:04.800Z (2 months ago)
Versions: 3
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 5,320 Total
Rankings:
- Dependent packages count: 42.313%
- Average: 44.416%
- Dependent repos count: 46.519%

gentoo-portage: sci-ml/hf_xet

xet client tech, used in huggingface_hub

Homepage: https://github.com/huggingface/xet-core
Documentation: https://packages.gentoo.org/packages/sci-ml/hf_xet
Licenses: Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD CDDL CDLA-Permissive-2.0 ISC MIT MPL-2.0 Unicode-3.0 ZLIB
Latest release: 1.5.2 (published 20 days ago)
Last Synced: 2026-08-09T01:32:12.422Z (18 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

nixpkgs-unstable: python314Packages.hf-xet

Xet client tech, used in huggingface_hub

Homepage: https://github.com/huggingface/xet-core/tree/main/hf_xet
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/hf-xet/default.nix#L52
Licenses: Apache-2.0
Latest release: 1.2.0 (published 7 months ago)
Last Synced: 2026-07-29T08:45:07.622Z (28 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- GaetanLepage

guix: python-hf-xet

Transfer large files with the Hugging Face Hub

Homepage: https://github.com/huggingface/xet-core
Documentation: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/python-web.scm#n2020
Licenses: asl2.0
Latest release: 1.5.1 (published 15 days ago)
Last Synced: 2026-08-12T00:22:37.247Z (15 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

nixpkgs-unstable: python313Packages.hf-xet

Xet client tech, used in huggingface_hub

Homepage: https://github.com/huggingface/xet-core/tree/main/hf_xet
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/hf-xet/default.nix#L52
Licenses: Apache-2.0
Latest release: 1.2.0 (published 7 months ago)
Last Synced: 2026-05-13T14:09:00.040Z (4 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- GaetanLepage

nixpkgs-unstable: git-xet

Git LFS plugin that uploads and downloads using the Xet protocol

Homepage: https://github.com/huggingface/xet-core/blob/main/git_xet/README.md
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/by-name/gi/git-xet/package.nix#L51
Licenses: Apache-2.0
Latest release: 0.2.0 (published 7 months ago)
Last Synced: 2026-03-26T11:10:20.300Z (5 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- cybardev

Dependencies

xet_client/Cargo.toml cargo

xet-core-structures 1.4.0
xet-runtime 1.4.0

xet_data/Cargo.toml cargo

pyo3 0.26
xet-client 1.4.0
xet-core-structures 1.4.0
xet-runtime 1.4.0

simulation/Cargo.toml cargo

simulation/chunk_cache_bench/Cargo.lock cargo

581 dependencies

simulation/chunk_cache_bench/Cargo.toml cargo

criterion 0.4 development
rand 0.8 development
async-trait 0.1
base64 0.22
clap 4
r2d2 0.8.10
r2d2_postgres 0.18.1
sccache 0.8
tempfile 3
tokio 1.44

wasm/hf_xet_thin_wasm/Cargo.lock cargo

416 dependencies

wasm/hf_xet_thin_wasm/Cargo.toml cargo

wasm/hf_xet_wasm/Cargo.toml cargo

wasm-bindgen-test 0.3.50 development
async-trait 0.1
bytes 1.11
console_error_panic_hook 0.1.7
console_log 1.0.0
env_logger 0.11
futures 0.3
getrandom 0.3
http 1
js-sys 0.3.77
log 0.4
serde 1
serde-wasm-bindgen 0.6.5
sha2 0.10
thiserror 2.0
tokio 1.49
tokio_with_wasm 0.8
uuid 1
wasm-bindgen =0.2.100
wasm-bindgen-futures 0.4.50
wasm_thread 0.3
web-sys 0.3.77

xet_core_structures/Cargo.toml cargo

xet-runtime 1.4.0

xet_pkg/Cargo.toml cargo

xet-client 1.4.0
xet-core-structures 1.4.0
xet-data 1.4.0
xet-runtime 1.4.0

wasm/hf_xet_wasm/Cargo.lock cargo

430 dependencies

.github/workflows/ci.yml actions

actions/checkout v4 composite
dtolnay/rust-toolchain stable composite
dtolnay/rust-toolchain 1.79.0 composite

.github/workflows/release.yml actions

PyO3/maturin-action v1 composite
actions/attest-build-provenance v1 composite
actions/checkout v4 composite
actions/download-artifact v4 composite
actions/setup-python v5 composite
actions/upload-artifact v4 composite

Cargo.lock cargo

392 dependencies

Cargo.toml cargo

hf_xet/Cargo.lock cargo

339 dependencies

hf_xet/Cargo.toml cargo

hf_xet/pyproject.toml pypi

.github/actions/cache-rust-build/action.yml actions

actions/cache v4 composite

.github/actions/set-build-profile/action.yml actions

.github/actions/windows-codesign/action.yml actions

azure/trusted-signing-action v0 composite

git_xet/Cargo.toml cargo

.github/workflows/git-xet-release.yml actions

./.github/actions/cache-rust-build * composite
./.github/actions/windows-codesign * composite
actions/checkout v4 composite
actions/download-artifact v5 composite
actions/upload-artifact v4 composite
dtolnay/rust-toolchain 1.89.0 composite
lando/code-sign-action v3 composite

.github/workflows/pre-release-testing.yml actions

actions/checkout v4 composite

.github/workflows/hf-xet-tests.yml actions

PyO3/maturin-action v1 composite
actions/checkout v4 composite
actions/setup-python v5 composite

xet_runtime/Cargo.toml cargo

.github/workflows/bump-crates-version.yml actions

./.github/actions/build-wasm * composite
./.github/actions/cache-rust-build * composite
PyO3/maturin-action 04ac600d27cdf7a9a280dadf7147097c42b757ad composite
actions/checkout de0fac2e4500dabe0009e67214ff5f5447ce83dd composite
actions/setup-python a309ff8b426b58ec0e2a45f0f869d46889d02405 composite
dtolnay/rust-toolchain 3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 composite
huggingface/hf-workflows/.github/actions/setup-hf-registry-proxies 21845492ef47d4e46b07a3b2a6e14fd4748c39cc composite

.github/workflows/crates-release.yml actions

actions/checkout de0fac2e4500dabe0009e67214ff5f5447ce83dd composite
dtolnay/rust-toolchain 3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 composite
rust-lang/crates-io-auth-action bbd81622f20ce9e2dd9622e3218b975523e45bbe composite

examples/xet_pkg_napi/Cargo.lock cargo

348 dependencies

simulation/sha256_bench/Cargo.lock cargo

block-buffer 0.10.4
block-buffer 0.12.1
cc 1.2.67
cfg-if 1.0.4
const-oid 0.10.2
cpufeatures 0.2.17
cpufeatures 0.3.0
crypto-common 0.1.7
crypto-common 0.2.2
digest 0.10.7
digest 0.11.3
find-msvc-tools 0.1.9
generic-array 0.14.7
hybrid-array 0.4.13
libc 0.2.186
sha2 0.10.9
sha2 0.11.0
sha2-asm 0.6.4
shlex 2.0.1
typenum 1.20.1
version_check 0.9.5

.github/actions/build-wasm/action.yml actions

./.github/actions/cache-rust-build * composite
actions/cache 668228422ae6a00e4ad889ee87cd7109ec5666a7 composite
dtolnay/rust-toolchain 3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 composite

.github/actions/macos-codesign-notarize/action.yml actions

examples/xet_pkg_napi/Cargo.toml cargo

simulation/sha256_bench/Cargo.toml cargo

examples/xet_pkg_napi/package-lock.json npm

@napi-rs/cli 2.18.4 development

examples/xet_pkg_napi/package.json npm

@napi-rs/cli ^2.18.4 development

wasm/ci-smoke/package-lock.json npm

fsevents 2.3.2
playwright 1.60.0
playwright-core 1.60.0

wasm/ci-smoke/package.json npm

playwright 1.60.0

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary