text-generation-inference

bloom deep-learning falcon gpt inference nlp pytorch starcoder transformer

Score: 25.838746516363642

Last synced: about 17 hours ago
JSON representation

Repository metadata:

Large Language Model Text Generation Inference

Host: GitHub
URL: https://github.com/huggingface/text-generation-inference
Owner: huggingface
License: apache-2.0
Created: 2022-10-08T10:26:28.000Z (over 3 years ago)
Default Branch: main
Last Pushed: 2025-09-17T00:46:42.000Z (5 months ago)
Last Synced: 2025-10-27T00:47:16.633Z (3 months ago)
Topics: bloom, deep-learning, falcon, gpt, inference, nlp, pytorch, starcoder, transformer
Language: Python
Homepage: http://hf.co/docs/text-generation-inference
Size: 13.5 MB
Stars: 10,597
Watchers: 101
Forks: 1,236
Open Issues: 316
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Owner metadata:

Name: Hugging Face
Login: huggingface
Email:
Kind: organization
Description: The AI community building the future.
Website: https://huggingface.co/
Location: NYC + Paris
Twitter: huggingface
Company:
Icon url: https://avatars.githubusercontent.com/u/25720743?v=4
Repositories: 370
Last Synced at: 2025-11-13T02:39:46.174Z
Profile URL: https://github.com/huggingface

GitHub Events

Total

Create event: 291
Delete event: 234
Fork event: 189
Issue comment event: 840
Issues event: 241
Member event: 6
Pull request event: 742
Pull request review comment event: 501
Pull request review event: 760
Push event: 1396
Release event: 16
Watch event: 1492
Total: 6708

Last Year

Create event: 291
Delete event: 234
Fork event: 189
Issue comment event: 840
Issues event: 241
Member event: 6
Pull request event: 742
Pull request review comment event: 501
Pull request review event: 760
Push event: 1396
Release event: 16
Watch event: 1492
Total: 6708

Committers metadata

Last synced: 2 months ago

Total Commits: 1,444
Total Committers: 145
Avg Commits per committer: 9.959
Development Distribution Score (DDS): 0.747

Commits in past year: 280
Committers in past year: 38
Avg Commits per committer in past year: 7.368
Development Distribution Score (DDS) in past year: 0.711

Name	Email	Commits
Nicolas Patry	p**s@p**m	365
OlivierDehaene	o**r@h**o	360
Daniël de Kok	m**e@d**u	167
drbh	d**z@g**m	143
Wang, Yi	y**g@i**m	74
Mohit Sharma	m**s@g**m	16
fxmarty	9****y	15
Yuan Wu	y**u@i**m	13
Nick Hill	n**l@u**m	13
Merve Noyan	m**n@g**m	13
Alvaro Bartolome	3****t	12
Omar Sanseviero	o**o@g**m	11
Funtowicz Morgan	m****z	10
Hugo Larcher	h**r@h**o	10
David Corvoysier	d**r@g**m	9
Baptiste Colle	3****e	8
Erik Kaunismäki	e**m@g**m	8
oOraph	1****h	6
Lucain	l**n@h**o	6
regisss	1****s	5
Adrien	a**n@h**o	5
Mishig	m**j@c**u	5
Alvaro Moran	6****o	4
Vaibhav Srivastav	v**0@g**m	4
icyboy™	x**g@p**m	3
zhangsibo1129	1****9	3
lewtun	l**l@g**m	3
Vincent Brouwers	v**s@i**m	3
Nicholas Broad	n**4@g**m	3
Guspan Tanadi	3****i	3
and 115 more...

Issue and Pull Request metadata

Last synced: 3 months ago

Total issues: 1,244
Total pull requests: 2,156
Average time to close issues: about 2 months
Average time to close pull requests: 10 days
Total issue authors: 833
Total pull request authors: 229
Average comments per issue: 3.03
Average comments per pull request: 0.99
Merged pull request: 1,454
Bot issues: 0
Bot pull requests: 0

Past year issues: 192
Past year pull requests: 662
Past year average time to close issues: 16 days
Past year average time to close pull requests: 6 days
Past year issue authors: 163
Past year pull request authors: 66
Past year average comments per issue: 0.89
Past year average comments per pull request: 0.78
Past year merged pull request: 452
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/huggingface/text-generation-inference

Top Issue Authors

RonanKMcGovern (17)
flozi00 (13)
paulcx (9)
martinigoyanes (8)
josephrocca (8)
angel-luis (7)
SinanAkkoyun (7)
fxmarty (7)
sam-h-bean (7)
icyxp (6)
poojitharamachandra (6)
philschmid (6)
amihalik (6)
vitalyshalumov (6)
calycekr (5)

Top Pull Request Authors

Narsil (498)
drbh (307)
danieldk (297)
sywangyi (154)
OlivierDehaene (130)
mht-sharma (43)
fxmarty (31)
mfuntowicz (31)
alvarobartt (29)
Hugoch (28)
ErikKaum (27)
yuanwu2017 (24)
dacorvo (19)
baptistecolle (15)
oOraph (15)

Top Issue Labels

Stale (423)
documentation (6)
bug (5)
feature request (4)
enhancement (2)
new model (2)

Top Pull Request Labels

Stale (48)
gaudi (4)
documentation (2)
Release tests (2)

Package metadata

Total packages: 3
Total downloads:
- pypi: 98,463 last-month
Total docker downloads: 6,543
Total dependent packages: 18 (may contain duplicates)
Total dependent repositories: 231 (may contain duplicates)
Total versions: 86
Total maintainers: 2
Total advisories: 1

pypi.org: text-generation

Hugging Face Text Generation Python Client

Homepage: https://github.com/huggingface/text-generation-inference
Documentation: https://text-generation.readthedocs.io/
Licenses: Apache-2.0
Latest release: 0.7.0 (published almost 2 years ago)
Last Synced: 2025-12-09T10:30:06.664Z (about 2 months ago)
Versions: 13
Dependent Packages: 18
Dependent Repositories: 231
Downloads: 97,199 Last month
Docker Downloads: 6,543
Rankings:
- Stargazers count: 0.367%
- Dependent repos count: 1.007%
- Downloads: 1.304%
- Dependent packages count: 1.4%
- Average: 1.609%
- Forks count: 1.804%
- Docker downloads count: 3.774%
Maintainers (1)
- olivierdehaene
Advisories:
- code injection vulnerability exists in the huggingface/text-generation-inference repository

proxy.golang.org: github.com/huggingface/text-generation-inference

Homepage:
Documentation: https://pkg.go.dev/github.com/huggingface/text-generation-inference#section-documentation
Licenses: apache-2.0
Latest release: v3.3.6+incompatible (published 5 months ago)
Last Synced: 2025-12-05T13:25:03.545Z (2 months ago)
Versions: 66
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 9.196%
- Average: 9.782%
- Dependent repos count: 10.369%

pypi.org: tgi

Nightly release of Hugging Face Text Generation Python Client

Homepage: https://github.com/huggingface/text-generation-inference
Documentation: https://tgi.readthedocs.io/
Licenses: Apache-2.0
Latest release: 2.4.2 (published over 1 year ago)
Last Synced: 2025-12-02T07:33:47.030Z (2 months ago)
Versions: 7
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 1,264 Last month
Rankings:
- Dependent packages count: 9.681%
- Average: 36.779%
- Dependent repos count: 63.876%
Maintainers (1)
- maziyar

Dependencies

Cargo.lock cargo

258 dependencies

server/pyproject.toml pypi

accelerate ^0.12.0
bitsandbytes ^0.35.1
grpcio ^1.49.1
grpcio-reflection ^1.49.1
protobuf ^4.21.7
python ^3.9
safetensors ^0.2.4
typer ^0.6.1

.github/workflows/build.yaml actions

actions/checkout v2 composite
actions/checkout v3 composite
actions/setup-python v4 composite
aquasecurity/trivy-action master composite
aws-actions/configure-aws-credentials v1 composite
docker/build-push-action v4 composite
docker/login-action v2 composite
docker/login-action v2.1.0 composite
docker/metadata-action v4.3.0 composite
docker/setup-buildx-action v2.0.0 composite
github/codeql-action/upload-sarif v2 composite
philschmid/philschmid-ec2-github-runner main composite
rlespinasse/github-slug-action v4.4.1 composite
sigstore/cosign-installer f3c664df7af409cb4873aa5068053ba9d61a57b6 composite
tailscale/github-action 7bd8039bf25c23c4ab1b8d6e2cc2da2280601966 composite

.github/workflows/client-tests.yaml actions

actions/checkout v2 composite
actions/setup-python v1 composite

.github/workflows/load_test.yaml actions

actions/checkout v3 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/tests.yaml actions

actions-rs/toolchain v1 composite
actions/cache v3 composite
actions/checkout v2 composite
actions/github-script v6 composite
actions/setup-python v1 composite
arduino/setup-protoc v1 composite

Cargo.toml cargo

benchmark/Cargo.toml cargo

launcher/Cargo.toml cargo

float_eq 1.0.1 development
reqwest 0.11.14 development
clap 4.1.4
ctrlc 3.2.5
nix 0.26.2
serde 1.0.152
serde_json 1.0.93
tracing 0.1.37
tracing-subscriber 0.3.16

router/Cargo.toml cargo

Dockerfile docker

base latest build
chef latest build
debian bullseye-slim build
kernel-builder latest build
lukemathwalker/cargo-chef latest-rust-1.71 build
nvidia/cuda 11.8.0-base-ubuntu20.04 build
pytorch-install latest build

clients/python/poetry.lock pypi

aiohttp 3.8.5
aiosignal 1.3.1
async-timeout 4.0.3
asynctest 0.13.0
atomicwrites 1.4.1
attrs 23.1.0
certifi 2023.7.22
charset-normalizer 3.2.0
colorama 0.4.6
coverage 7.2.7
filelock 3.12.2
frozenlist 1.3.3
fsspec 2023.1.0
huggingface-hub 0.16.4
idna 3.4
importlib-metadata 6.7.0
iniconfig 2.0.0
multidict 6.0.4
packaging 23.1
pluggy 1.2.0
py 1.11.0
pydantic 1.10.12
pytest 6.2.5
pytest-asyncio 0.17.2
pytest-cov 3.0.0
pyyaml 6.0.1
requests 2.31.0
toml 0.10.2
tomli 2.0.1
tqdm 4.66.1
typing-extensions 4.7.1
urllib3 2.0.4
yarl 1.9.2
zipp 3.15.0

clients/python/pyproject.toml pypi

pytest ^6.2.5 develop
pytest-asyncio ^0.17.2 develop
pytest-cov ^3.0.0 develop
aiohttp ^3.8
huggingface-hub >= 0.12, < 1.0
pydantic > 1.10, < 3
python ^3.7

integration-tests/pyproject.toml pypi

docker ^6.1.3
pytest ^7.4.0
pytest-asyncio ^0.21.1
python >=3.9,<3.13
syrupy 4.0.1
text-generation ^0.6.0

integration-tests/requirements.txt pypi

aiohttp ==3.8.5 test
aiosignal ==1.3.1 test
async-timeout ==4.0.3 test
attrs ==23.1.0 test
certifi ==2023.7.22 test
charset-normalizer ==3.2.0 test
colorama ==0.4.6 test
colored ==1.4.4 test
docker ==6.1.3 test
exceptiongroup ==1.1.3 test
filelock ==3.12.3 test
frozenlist ==1.4.0 test
fsspec ==2023.6.0 test
huggingface-hub ==0.16.4 test
idna ==3.4 test
iniconfig ==2.0.0 test
multidict ==6.0.4 test
packaging ==23.1 test
pluggy ==1.3.0 test
pydantic ==1.10.12 test
pytest ==7.4.0 test
pytest-asyncio ==0.21.1 test
pywin32 ==306 test
pyyaml ==6.0.1 test
requests ==2.31.0 test
syrupy ==4.0.1 test
text-generation ==0.6.0 test
tomli ==2.0.1 test
tqdm ==4.66.1 test
typing-extensions ==4.7.1 test
urllib3 ==2.0.4 test
websocket-client ==1.6.2 test
yarl ==1.9.2 test

server/custom_kernels/setup.py pypi

server/exllama_kernels/setup.py pypi

.github/workflows/build_documentation.yaml actions

.github/workflows/nix_build.yaml actions

actions/checkout v4 composite
cachix/cachix-action v14 composite
cachix/install-nix-action v27 composite
docker/login-action v3 composite
docker/setup-buildx-action v3 composite
rlespinasse/github-slug-action v4.4.1 composite

.github/workflows/nix_cache.yaml actions

actions/checkout v4 composite
cachix/cachix-action v14 composite
cachix/install-nix-action v27 composite

.github/workflows/nix_tests.yaml actions

actions/checkout v4 composite
cachix/cachix-action v14 composite
cachix/install-nix-action v27 composite

server/requirements_cuda.txt pypi

accelerate ==1.6.0
aiohappyeyeballs ==2.6.1
aiohttp ==3.11.18
aiosignal ==1.3.2
airportsdata ==20250224
annotated-types ==0.7.0
attrs ==25.3.0
bitsandbytes ==0.45.5
certifi ==2025.4.26
charset-normalizer ==3.4.2
click ==8.1.8
cloudpickle ==3.1.1
compressed-tensors ==0.9.4
datasets ==2.21.0
deprecated ==1.2.18
dill ==0.3.8
diskcache ==5.6.3
einops ==0.8.1
filelock ==3.18.0
frozenlist ==1.6.0
fsspec ==2024.6.1
genson ==1.3.0
googleapis-common-protos ==1.70.0
grpc-interceptor ==0.15.4
grpcio ==1.71.0
grpcio-reflection ==1.71.0
grpcio-status ==1.71.0
hf-transfer ==0.1.9
hf-xet ==1.1.0
huggingface-hub ==0.31.1
idna ==3.10
importlib-metadata ==8.6.1
interegular ==0.3.3
iso3166 ==2.1.1
jinja2 ==3.1.6
jsonschema ==4.23.0
jsonschema-specifications ==2025.4.1
kernels ==0.5.0
lark ==1.2.2
loguru ==0.7.3
markdown-it-py ==3.0.0
markupsafe ==3.0.2
mdurl ==0.1.2
mpmath ==1.3.0
multidict ==6.4.3
multiprocess ==0.70.16
nest-asyncio ==1.6.0
networkx ==3.4.2
numpy ==2.2.5
nvidia-cublas-cu12 ==12.6.4.1
nvidia-cuda-cupti-cu12 ==12.6.80
nvidia-cuda-nvrtc-cu12 ==12.6.77
nvidia-cuda-runtime-cu12 ==12.6.77
nvidia-cudnn-cu12 ==9.5.1.17
nvidia-cufft-cu12 ==11.3.0.4
nvidia-cufile-cu12 ==1.11.1.6
nvidia-curand-cu12 ==10.3.7.77
nvidia-cusolver-cu12 ==11.7.1.2
nvidia-cusparse-cu12 ==12.5.4.2
nvidia-cusparselt-cu12 ==0.6.3
nvidia-nccl-cu12 ==2.26.2
nvidia-nvjitlink-cu12 ==12.6.85
nvidia-nvtx-cu12 ==12.6.77
opentelemetry-api ==1.33.0
opentelemetry-exporter-otlp ==1.33.0
opentelemetry-exporter-otlp-proto-common ==1.33.0
opentelemetry-exporter-otlp-proto-grpc ==1.33.0
opentelemetry-exporter-otlp-proto-http ==1.33.0
opentelemetry-instrumentation ==0.54b0
opentelemetry-instrumentation-grpc ==0.54b0
opentelemetry-proto ==1.33.0
opentelemetry-sdk ==1.33.0
opentelemetry-semantic-conventions ==0.54b0
outlines ==0.2.3
outlines-core ==0.1.26
packaging ==25.0
pandas ==2.2.3
peft ==0.15.2
pillow ==11.2.1
prometheus-client ==0.21.1
propcache ==0.3.1
protobuf ==5.29.4
psutil ==7.0.0
py-cpuinfo ==9.0.0
pyarrow ==20.0.0
pydantic ==2.11.4
pydantic-core ==2.33.2
pygments ==2.19.1
python-dateutil ==2.9.0.post0
pytz ==2025.2
pyyaml ==6.0.2
referencing ==0.36.2
regex ==2024.11.6
requests ==2.32.3
rich ==14.0.0
rpds-py ==0.24.0
safetensors ==0.5.3
scipy ==1.15.3
sentencepiece ==0.2.0
setuptools ==80.4.0
shellingham ==1.5.4
six ==1.17.0
sympy ==1.14.0
texttable ==1.7.0
tokenizers ==0.21.1
torch ==2.7.0
tqdm ==4.67.1
transformers ==4.51.3
triton ==3.3.0
typer ==0.15.3
typing-extensions ==4.13.2
typing-inspection ==0.4.0
tzdata ==2025.2
urllib3 ==2.4.0
wrapt ==1.17.2
xxhash ==3.5.0
yarl ==1.20.0
zipp ==3.21.0

.github/workflows/upload_pr_documentation.yaml actions

backends/client/Cargo.toml cargo

backends/grpc-metadata/Cargo.toml cargo

backends/gaudi/server/pyproject.toml pypi

grpcio-tools * develop
pytest ^8.3.5 develop
accelerate ^1.7.0
grpc-interceptor ^0.15.0
grpcio ^1.71.1
grpcio-reflection *
grpcio-status *
hf-transfer ^0.1.9
loguru ^0.7.3
numpy ^1.26
opentelemetry-api ^1.32.0
opentelemetry-exporter-otlp ^1.32.0
opentelemetry-instrumentation-grpc ^0.53b0
outlines ^0.0.36
peft ^0.15
prometheus-client ^0.21.1
protobuf ^5.0
py-cpuinfo ^9.0.0
python >=3.9,<3.13
sentencepiece ^0.2.0
transformers ^4.52.4
typer ^0.15.0

backends/v2/Cargo.toml cargo

backends/llamacpp/Cargo.toml cargo

backends/trtllm/Cargo.toml cargo

load_tests/pyproject.toml pypi

docker ^7.1.0
gputil ^1.4.0
loguru ^0.7.2
pandas ^2.2.3
psutil ^6.0.0
pyarrow ^17.0.0
python ^3.11

.github/workflows/ci_build.yaml actions

backends/neuron/server/pyproject.toml pypi

grpc-interceptor == 0.15.2
grpcio == 1.57.0
grpcio-reflection == 1.48.2
grpcio-status == 1.48.2
loguru == 0.6.0
optimum-neuron [neuronx] >= 0.0.28
protobuf > 3.20.1, < 4
safetensors *
typer == 0.6.1

backends/neuron/tests/requirements.txt pypi

Levenshtein * test
docker >=6.1.3 test
pytest >=7.4.0 test
pytest-asyncio >=0.21.1 test
requests <2.32.0 test
text-generation >=0.6.0 test

server/requirements_rocm.txt pypi

accelerate ==1.6.0
aiohappyeyeballs ==2.6.1
aiohttp ==3.11.18
aiosignal ==1.3.2
airportsdata ==20250224
annotated-types ==0.7.0
attrs ==25.3.0
certifi ==2025.4.26
charset-normalizer ==3.4.2
click ==8.1.8
cloudpickle ==3.1.1
compressed-tensors ==0.9.4
datasets ==2.21.0
deprecated ==1.2.18
dill ==0.3.8
diskcache ==5.6.3
einops ==0.8.1
filelock ==3.18.0
frozenlist ==1.6.0
fsspec ==2024.6.1
genson ==1.3.0
googleapis-common-protos ==1.70.0
grpc-interceptor ==0.15.4
grpcio ==1.71.0
grpcio-reflection ==1.71.0
grpcio-status ==1.71.0
hf-transfer ==0.1.9
hf-xet ==1.1.0
huggingface-hub ==0.31.1
idna ==3.10
importlib-metadata ==8.6.1
interegular ==0.3.3
iso3166 ==2.1.1
jinja2 ==3.1.6
jsonschema ==4.23.0
jsonschema-specifications ==2025.4.1
kernels ==0.5.0
lark ==1.2.2
loguru ==0.7.3
markdown-it-py ==3.0.0
markupsafe ==3.0.2
mdurl ==0.1.2
mpmath ==1.3.0
multidict ==6.4.3
multiprocess ==0.70.16
nest-asyncio ==1.6.0
networkx ==3.4.2
numpy ==2.2.5
nvidia-cublas-cu12 ==12.6.4.1
nvidia-cuda-cupti-cu12 ==12.6.80
nvidia-cuda-nvrtc-cu12 ==12.6.77
nvidia-cuda-runtime-cu12 ==12.6.77
nvidia-cudnn-cu12 ==9.5.1.17
nvidia-cufft-cu12 ==11.3.0.4
nvidia-cufile-cu12 ==1.11.1.6
nvidia-curand-cu12 ==10.3.7.77
nvidia-cusolver-cu12 ==11.7.1.2
nvidia-cusparse-cu12 ==12.5.4.2
nvidia-cusparselt-cu12 ==0.6.3
nvidia-nccl-cu12 ==2.26.2
nvidia-nvjitlink-cu12 ==12.6.85
nvidia-nvtx-cu12 ==12.6.77
opentelemetry-api ==1.33.0
opentelemetry-exporter-otlp ==1.33.0
opentelemetry-exporter-otlp-proto-common ==1.33.0
opentelemetry-exporter-otlp-proto-grpc ==1.33.0
opentelemetry-exporter-otlp-proto-http ==1.33.0
opentelemetry-instrumentation ==0.54b0
opentelemetry-instrumentation-grpc ==0.54b0
opentelemetry-proto ==1.33.0
opentelemetry-sdk ==1.33.0
opentelemetry-semantic-conventions ==0.54b0
outlines ==0.2.3
outlines-core ==0.1.26
packaging ==25.0
pandas ==2.2.3
peft ==0.15.2
pillow ==11.2.1
prometheus-client ==0.21.1
propcache ==0.3.1
protobuf ==5.29.4
psutil ==7.0.0
py-cpuinfo ==9.0.0
pyarrow ==20.0.0
pydantic ==2.11.4
pydantic-core ==2.33.2
pygments ==2.19.1
python-dateutil ==2.9.0.post0
pytz ==2025.2
pyyaml ==6.0.2
referencing ==0.36.2
regex ==2024.11.6
requests ==2.32.3
rich ==14.0.0
rpds-py ==0.24.0
safetensors ==0.5.3
scipy ==1.15.3
sentencepiece ==0.2.0
setuptools ==80.4.0
shellingham ==1.5.4
six ==1.17.0
sympy ==1.14.0
texttable ==1.7.0
tokenizers ==0.21.1
torch ==2.7.0
tqdm ==4.67.1
transformers ==4.51.3
triton ==3.3.0
typer ==0.15.3
typing-extensions ==4.13.2
typing-inspection ==0.4.0
tzdata ==2025.2
urllib3 ==2.4.0
wrapt ==1.17.2
xxhash ==3.5.0
yarl ==1.20.0
zipp ==3.21.0

server/uv.lock pypi

135 dependencies

.github/workflows/build_pr_documentation.yaml actions

backends/llamacpp/requirements.txt pypi

hf-transfer ==0.1.9
huggingface-hub ==0.28.1
torch ==2.6.0
transformers ==4.49

backends/gaudi/server/poetry.lock pypi

colorama 0.4.6 develop
exceptiongroup 1.2.2 develop
grpcio 1.72.0rc1 develop
grpcio-tools 1.71.0 develop
iniconfig 2.1.0 develop
packaging 24.2 develop
pluggy 1.5.0 develop
protobuf 5.29.4 develop
pytest 8.3.5 develop
setuptools 78.1.0 develop
tomli 2.2.1 develop
accelerate 0.33.0
annotated-types 0.7.0
attrs 25.3.0
certifi 2025.1.31
charset-normalizer 3.4.1
click 8.1.8
cloudpickle 3.1.1
colorama 0.4.6
deprecated 1.2.18
diffusers 0.31.0
diskcache 5.6.3
filelock 3.18.0
fsspec 2025.3.2
googleapis-common-protos 1.70.0
grpc-interceptor 0.15.4
grpcio 1.72.0rc1
grpcio-reflection 1.71.0
grpcio-status 1.71.0
hf-transfer 0.1.9
huggingface-hub 0.30.2
idna 3.10
importlib-metadata 8.6.1
interegular 0.3.3
jinja2 3.1.6
joblib 1.4.2
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
lark 1.2.2
llvmlite 0.43.0
loguru 0.7.3
markdown-it-py 3.0.0
markupsafe 3.0.2
mdurl 0.1.2
mpmath 1.3.0
nest-asyncio 1.6.0
networkx 3.2.1
numba 0.60.0
numpy 1.26.4
opentelemetry-api 1.32.0
opentelemetry-exporter-otlp 1.32.0
opentelemetry-exporter-otlp-proto-common 1.32.0
opentelemetry-exporter-otlp-proto-grpc 1.32.0
opentelemetry-exporter-otlp-proto-http 1.32.0
opentelemetry-instrumentation 0.53b0
opentelemetry-instrumentation-grpc 0.53b0
opentelemetry-proto 1.32.0
opentelemetry-sdk 1.32.0
opentelemetry-semantic-conventions 0.53b0
optimum 1.24.0
optimum-habana 1.17.0
outlines 0.0.36
packaging 24.2
peft 0.15.1
pillow 11.2.1
prometheus-client 0.21.1
protobuf 5.29.4
psutil 7.0.0
py-cpuinfo 9.0.0
pydantic 2.11.3
pydantic-core 2.33.1
pygments 2.19.1
pyyaml 6.0.2
referencing 0.36.2
regex 2024.11.6
requests 2.32.3
rich 14.0.0
rpds-py 0.24.0
safetensors 0.5.3
scikit-learn 1.6.1
scipy 1.13.1
sentence-transformers 3.3.1
sentencepiece 0.2.0
setuptools 78.1.0
shellingham 1.5.4
sympy 1.13.1
threadpoolctl 3.6.0
tokenizers 0.21.1
tqdm 4.67.1
transformers 4.49.0
triton 3.2.0
typer 0.15.2
typing-extensions 4.13.2
typing-inspection 0.4.0
urllib3 2.4.0
win32-setctime 1.2.0
wrapt 1.17.2
zipp 3.21.0

backends/neuron/server/build-requirements.txt pypi

build *
grpcio-tools ==1.53.0
mypy-protobuf *

.github/workflows/integration_tests.yaml actions

actions/checkout v4 composite
actions/setup-python v4 composite
rlespinasse/github-slug-action v4.4.1 composite

server/exllamav2_kernels/setup.py pypi

server/requirements_gen.txt pypi

certifi ==2025.4.26
charset-normalizer ==3.4.2
click ==8.1.8
deprecated ==1.2.18
einops ==0.8.1
filelock ==3.18.0
fsspec ==2025.3.2
googleapis-common-protos ==1.70.0
grpc-interceptor ==0.15.4
grpcio ==1.71.0
grpcio-reflection ==1.71.0
grpcio-status ==1.71.0
grpcio-tools ==1.71.0
hf-transfer ==0.1.9
hf-xet ==1.1.0
huggingface-hub ==0.31.1
idna ==3.10
importlib-metadata ==8.6.1
kernels ==0.5.0
loguru ==0.7.3
markdown-it-py ==3.0.0
mdurl ==0.1.2
mypy-protobuf ==3.6.0
numpy ==2.2.5
opentelemetry-api ==1.33.0
opentelemetry-exporter-otlp ==1.33.0
opentelemetry-exporter-otlp-proto-common ==1.33.0
opentelemetry-exporter-otlp-proto-grpc ==1.33.0
opentelemetry-exporter-otlp-proto-http ==1.33.0
opentelemetry-instrumentation ==0.54b0
opentelemetry-instrumentation-grpc ==0.54b0
opentelemetry-proto ==1.33.0
opentelemetry-sdk ==1.33.0
opentelemetry-semantic-conventions ==0.54b0
packaging ==25.0
pillow ==11.2.1
prometheus-client ==0.21.1
protobuf ==5.29.4
py-cpuinfo ==9.0.0
pygments ==2.19.1
pyyaml ==6.0.2
regex ==2024.11.6
requests ==2.32.3
rich ==14.0.0
safetensors ==0.5.3
scipy ==1.15.3
sentencepiece ==0.2.0
setuptools ==80.4.0
shellingham ==1.5.4
tokenizers ==0.21.1
tqdm ==4.67.1
transformers ==4.51.3
typer ==0.15.3
types-protobuf ==6.30.2.20250506
typing-extensions ==4.13.2
urllib3 ==2.4.0
wrapt ==1.17.2
zipp ==3.21.0

server/requirements_intel.txt pypi

accelerate ==1.6.0
aiohappyeyeballs ==2.6.1
aiohttp ==3.11.18
aiosignal ==1.3.2
airportsdata ==20250224
annotated-types ==0.7.0
attrs ==25.3.0
certifi ==2025.4.26
charset-normalizer ==3.4.2
click ==8.1.8
cloudpickle ==3.1.1
compressed-tensors ==0.9.4
datasets ==2.21.0
deprecated ==1.2.18
dill ==0.3.8
diskcache ==5.6.3
einops ==0.8.1
filelock ==3.18.0
frozenlist ==1.6.0
fsspec ==2024.6.1
genson ==1.3.0
googleapis-common-protos ==1.70.0
grpc-interceptor ==0.15.4
grpcio ==1.71.0
grpcio-reflection ==1.71.0
grpcio-status ==1.71.0
hf-transfer ==0.1.9
hf-xet ==1.1.0
huggingface-hub ==0.31.1
idna ==3.10
importlib-metadata ==8.6.1
interegular ==0.3.3
iso3166 ==2.1.1
jinja2 ==3.1.6
jsonschema ==4.23.0
jsonschema-specifications ==2025.4.1
kernels ==0.5.0
lark ==1.2.2
loguru ==0.7.3
markdown-it-py ==3.0.0
markupsafe ==3.0.2
mdurl ==0.1.2
mpmath ==1.3.0
multidict ==6.4.3
multiprocess ==0.70.16
nest-asyncio ==1.6.0
networkx ==3.4.2
numpy ==2.2.5
nvidia-cublas-cu12 ==12.6.4.1
nvidia-cuda-cupti-cu12 ==12.6.80
nvidia-cuda-nvrtc-cu12 ==12.6.77
nvidia-cuda-runtime-cu12 ==12.6.77
nvidia-cudnn-cu12 ==9.5.1.17
nvidia-cufft-cu12 ==11.3.0.4
nvidia-cufile-cu12 ==1.11.1.6
nvidia-curand-cu12 ==10.3.7.77
nvidia-cusolver-cu12 ==11.7.1.2
nvidia-cusparse-cu12 ==12.5.4.2
nvidia-cusparselt-cu12 ==0.6.3
nvidia-nccl-cu12 ==2.26.2
nvidia-nvjitlink-cu12 ==12.6.85
nvidia-nvtx-cu12 ==12.6.77
opentelemetry-api ==1.33.0
opentelemetry-exporter-otlp ==1.33.0
opentelemetry-exporter-otlp-proto-common ==1.33.0
opentelemetry-exporter-otlp-proto-grpc ==1.33.0
opentelemetry-exporter-otlp-proto-http ==1.33.0
opentelemetry-instrumentation ==0.54b0
opentelemetry-instrumentation-grpc ==0.54b0
opentelemetry-proto ==1.33.0
opentelemetry-sdk ==1.33.0
opentelemetry-semantic-conventions ==0.54b0
outlines ==0.2.3
outlines-core ==0.1.26
packaging ==25.0
pandas ==2.2.3
peft ==0.15.2
pillow ==11.2.1
prometheus-client ==0.21.1
propcache ==0.3.1
protobuf ==5.29.4
psutil ==7.0.0
py-cpuinfo ==9.0.0
pyarrow ==20.0.0
pydantic ==2.11.4
pydantic-core ==2.33.2
pygments ==2.19.1
python-dateutil ==2.9.0.post0
pytz ==2025.2
pyyaml ==6.0.2
referencing ==0.36.2
regex ==2024.11.6
requests ==2.32.3
rich ==14.0.0
rpds-py ==0.24.0
safetensors ==0.5.3
scipy ==1.15.3
sentencepiece ==0.2.0
setuptools ==80.4.0
shellingham ==1.5.4
six ==1.17.0
sympy ==1.14.0
texttable ==1.7.0
tokenizers ==0.21.1
torch ==2.7.0
tqdm ==4.67.1
transformers ==4.51.3
triton ==3.3.0
typer ==0.15.3
typing-extensions ==4.13.2
typing-inspection ==0.4.0
tzdata ==2025.2
urllib3 ==2.4.0
wrapt ==1.17.2
xxhash ==3.5.0
yarl ==1.20.0
zipp ==3.21.0

.github/workflows/autodocs.yaml actions

actions-rs/toolchain v1 composite
actions/checkout v2 composite
actions/setup-node v4 composite
actions/setup-python v2 composite

backends/gaudi/server/requirements.txt pypi

accelerate ==1.7.0
annotated-types ==0.7.0
attrs ==25.3.0
certifi ==2025.1.31
charset-normalizer ==3.4.1
click ==8.1.8
cloudpickle ==3.1.1
colorama ==0.4.6
deprecated ==1.2.18
diffusers ==0.31.0
diskcache ==5.6.3
filelock ==3.18.0
fsspec ==2025.3.2
googleapis-common-protos ==1.70.0
grpc-interceptor ==0.15.4
grpcio ==1.72.0rc1
grpcio-reflection ==1.71.0
grpcio-status ==1.71.0
hf-transfer ==0.1.9
huggingface-hub ==0.30.2
idna ==3.10
importlib-metadata ==8.6.1
interegular ==0.3.3
jinja2 ==3.1.6
joblib ==1.4.2
jsonschema ==4.23.0
jsonschema-specifications ==2024.10.1
lark ==1.2.2
llvmlite ==0.43.0
loguru ==0.7.3
markdown-it-py ==3.0.0
markupsafe ==3.0.2
mdurl ==0.1.2
mpmath ==1.3.0
nest-asyncio ==1.6.0
networkx ==3.2.1
numba ==0.60.0
numpy ==1.26.4
opentelemetry-api ==1.32.0
opentelemetry-exporter-otlp ==1.32.0
opentelemetry-exporter-otlp-proto-common ==1.32.0
opentelemetry-exporter-otlp-proto-grpc ==1.32.0
opentelemetry-exporter-otlp-proto-http ==1.32.0
opentelemetry-instrumentation ==0.53b0
opentelemetry-instrumentation-grpc ==0.53b0
opentelemetry-proto ==1.32.0
opentelemetry-sdk ==1.32.0
opentelemetry-semantic-conventions ==0.53b0
optimum ==1.24.0
outlines ==0.0.36
packaging ==24.2
peft ==0.15.1
pillow ==11.2.1
prometheus-client ==0.21.1
protobuf ==5.29.4
psutil ==7.0.0
py-cpuinfo ==9.0.0
pydantic ==2.11.3
pydantic-core ==2.33.1
pygments ==2.19.1
pyyaml ==6.0.2
referencing ==0.36.2
regex ==2024.11.6
requests ==2.32.3
rich ==14.0.0
rpds-py ==0.24.0
safetensors ==0.5.3
scikit-learn ==1.6.1
scipy ==1.13.1
sentence-transformers ==3.3.1
sentencepiece ==0.2.0
setuptools ==78.1.0
shellingham ==1.5.4
sympy ==1.13.1
threadpoolctl ==3.6.0
tokenizers ==0.21.1
tqdm ==4.67.1
transformers ==4.52.4
triton ==3.2.0
typer ==0.15.2
typing-extensions ==4.13.2
typing-inspection ==0.4.0
urllib3 ==2.4.0
win32-setctime ==1.2.0
wrapt ==1.17.2
zipp ==3.21.0

.github/workflows/stale.yaml actions

actions/stale v8 composite

.github/workflows/trufflehog.yaml actions

actions/checkout v4 composite
trufflesecurity/trufflehog 853e1e8d249fd1e29d0fcc7280d29b03df3d643d composite

backends/v3/Cargo.toml cargo

criterion 0.3 development
itertools 0.13 development
rustc-hash 2 development
async-stream 0.3.5
async-trait 0.1.74
axum 0.7
axum-tracing-opentelemetry 0.16
clap 4.4.5
futures 0.3.28
futures-util 0.3.30
image 0.25.1
init-tracing-opentelemetry 0.14.1
jsonschema 0.28.0
nohash-hasher 0.2.0
once_cell 1.19.0
opentelemetry 0.20.0
opentelemetry-otlp 0.13.0
prost ^0.12
rand 0.8.5
regex 1.10.3
reqwest 0.11.20
serde 1.0.188
serde_json 1.0.107
slotmap 1.0.7
thiserror 1.0.48
tokio 1.32.0
tokio-stream 0.1.14
tonic ^0.10
tower ^0.4
tower-http 0.5.1
tracing 0.1.37
tracing-opentelemetry 0.21.0
tracing-subscriber 0.3.17
utoipa 4.2.0
utoipa-swagger-ui 6.0.0

integration-tests/uv.lock pypi

aiohappyeyeballs 2.4.6
aiohttp 3.11.12
aiosignal 1.3.2
annotated-types 0.7.0
anyio 4.8.0
async-timeout 5.0.1
attrs 25.1.0
certifi 2025.1.31
charset-normalizer 3.4.1
colorama 0.4.6
distro 1.9.0
docker 7.1.0
exceptiongroup 1.2.2
filelock 3.17.0
frozenlist 1.5.0
fsspec 2025.2.0
h11 0.14.0
httpcore 1.0.7
httpx 0.28.1
huggingface-hub 0.29.0
idna 3.10
iniconfig 2.0.0
jiter 0.9.0
multidict 6.1.0
numpy 2.2.3
openai 1.66.3
packaging 24.2
pillow 11.1.0
pluggy 1.5.0
propcache 0.2.1
pydantic 2.10.6
pydantic-core 2.27.2
pytest 8.3.4
pytest-asyncio 0.25.3
pywin32 308
pyyaml 6.0.2
requests 2.32.3
sniffio 1.3.1
syrupy 4.8.1
text-generation 0.7.0
text-generation-integration-tests 2.0.1
tomli 2.2.1
tqdm 4.67.1
typing-extensions 4.12.2
urllib3 2.3.0
yarl 1.18.3

load_tests/poetry.lock pypi

certifi 2024.8.30
charset-normalizer 3.3.2
colorama 0.4.6
docker 7.1.0
gputil 1.4.0
idna 3.10
loguru 0.7.2
numpy 2.1.1
pandas 2.2.3
psutil 6.0.0
pyarrow 17.0.0
python-dateutil 2.9.0.post0
pytz 2024.2
pywin32 306
requests 2.32.3
six 1.16.0
tzdata 2024.2
urllib3 2.2.3
win32-setctime 1.1.0

backends/neuron/Cargo.toml cargo

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary