OpenLLM

bentoml fine-tuning llama llama2 llama3-1 llama3-2 llama3-2-vision llm llm-inference llm-ops llm-serving llmops mistral mlops model-inference open-source-llm openllm vicuna

Score: 22.200493686843345

Last synced: about 6 hours ago
JSON representation

Repository metadata:

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Host: GitHub
URL: https://github.com/bentoml/OpenLLM
Owner: bentoml
License: apache-2.0
Created: 2023-04-19T00:27:52.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2026-03-30T16:54:12.000Z (2 months ago)
Last Synced: 2026-03-30T18:28:26.953Z (2 months ago)
Topics: bentoml, fine-tuning, llama, llama2, llama3-1, llama3-2, llama3-2-vision, llm, llm-inference, llm-ops, llm-serving, llmops, mistral, mlops, model-inference, open-source-llm, openllm, vicuna
Language: Python
Homepage: https://bentoml.com
Size: 40.8 MB
Stars: 12,244
Watchers: 71
Forks: 806
Open Issues: 12
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Codeowners: .github/CODEOWNERS
- Security: .github/SECURITY.md

Owner metadata:

Name: BentoML
Login: bentoml
Email:
Kind: organization
Description: The most flexible way to serve AI models in production
Website: https://bentoml.com
Location: San Francisco
Twitter: bentomlai
Company:
Icon url: https://avatars.githubusercontent.com/u/49176046?v=4
Repositories: 76
Last Synced at: 2024-04-14T02:46:22.271Z
Profile URL: https://github.com/bentoml

GitHub Events

Total

Create event: 94
Delete event: 85
Fork event: 138
Issue comment event: 45
Issues event: 38
Pull request event: 167
Pull request review comment event: 1
Pull request review event: 13
Push event: 140
Release event: 17
Watch event: 1739
Total: 2477

Last Year

Create event: 73
Delete event: 71
Fork event: 122
Issue comment event: 33
Issues event: 30
Pull request event: 134
Pull request review comment event: 1
Pull request review event: 13
Push event: 111
Release event: 13
Watch event: 1423
Total: 2024

Committers metadata

Last synced: 3 months ago

Total Commits: 1,917
Total Committers: 32
Avg Commits per committer: 59.906
Development Distribution Score (DDS): 0.331

Commits in past year: 42
Committers in past year: 3
Avg Commits per committer in past year: 14.0
Development Distribution Score (DDS) in past year: 0.571

Name	Email	Commits
Aaron Pham	2****m	1282
dependabot[bot]	4****]	314
bojiang	b**_@o**m	146
pre-commit-ci[bot]	6****]	84
Sherlock Xu	6****3	14
Zhao Shenyang	d**v@z**m	13
Jian Shen	j**2@g**m	8
Rick Zhou	r**u@g**m	8
Chaoyu	p**g@g**m	7
xianxian.zhang	1****l	7
XunchaoZ	6****Z	5
agent	a**t@S**l	3
Fazli Sapuan	f**i@s**g	2
Frost Ming	m**g@g**m	2
GutZuFusss	l**r@g**m	2
HeTaoPKU	4****d	2
MingLiangDai	9****i	2
github-actions[bot]	g****]	2
Abhishek	5****2	1
Alan Poulain	c**t@a**u	1
Dennis Rall	5****l	1
Ikko Eltociear Ashimine	e**r@g**m	1
Kuan-Chun Wang	j**o@g**m	1
Matt Hoffner	m**r@g**m	1
Miorel-Lucian Palii	m**i@g**m	1
RichardScottOZ	7****Z	1
Sauyon Lee	2****n	1
Sean Sheng	s**g@g**m	1
Steve Guo	4****o	1
Yuchen Cheng	r**c@g**m	1
and 2 more...

Issue and Pull Request metadata

Last synced: 6 months ago

Total issues: 140
Total pull requests: 471
Average time to close issues: 4 months
Average time to close pull requests: 5 days
Total issue authors: 117
Total pull request authors: 26
Average comments per issue: 2.66
Average comments per pull request: 0.22
Merged pull request: 352
Bot issues: 2
Bot pull requests: 353

Past year issues: 9
Past year pull requests: 159
Past year average time to close issues: about 21 hours
Past year average time to close pull requests: 4 days
Past year issue authors: 8
Past year pull request authors: 7
Past year average comments per issue: 1.11
Past year average comments per pull request: 0.18
Past year merged pull request: 125
Past year bot issues: 0
Past year bot pull requests: 135

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/bentoml/OpenLLM

Top Issue Authors

aarnphm (12)
hahmad2008 (4)
Said-Ikki (2)
andrewjwaggoner (2)
Nanguage (2)
zhangxinyang97 (2)
meanwo (2)
foxxxx001 (2)
yalattas (2)
mfournioux (2)
stoneLee81 (2)
dependabot[bot] (1)
espdev (1)
andysingal (1)
Lightwave234 (1)

Top Pull Request Authors

dependabot[bot] (252)
pre-commit-ci[bot] (100)
aarnphm (60)
Sherlock113 (16)
larme (8)
bojiang (6)
parano (4)
fuzzie360 (2)
dowithless (2)
Oscarjia (2)
charlod (2)
matthoffner (2)
rickzx (2)
GutZuFusss (1)
mantrakp04 (1)

Top Issue Labels

performance (2)
enhancement (1)
dependencies (1)
python (1)

Top Pull Request Labels

dependencies (252)
github_actions (132)
python (117)
documentation (8)
javascript (3)
no_release (1)

Package metadata

Total packages: 21
Total downloads:
- pypi: 10,698 last-month
- npm: 4 last-month
Total docker downloads: 141
Total dependent packages: 7 (may contain duplicates)
Total dependent repositories: 311 (may contain duplicates)
Total versions: 769
Total maintainers: 6

pypi.org: openllm

OpenLLM: Self-hosting LLMs Made Easy.

Homepage: https://bentoml.com
Documentation: https://github.com/bentoml/OpenLLM#readme
Licenses: Apache-2.0
Latest release: 0.6.30 (published about 1 year ago)
Last Synced: 2026-03-25T18:20:06.537Z (2 months ago)
Versions: 195
Dependent Packages: 3
Dependent Repositories: 295
Downloads: 7,125 Last month
Docker Downloads: 47
Rankings:
- Dependent repos count: 0.868%
- Stargazers count: 0.981%
- Downloads: 1.87%
- Average: 2.175%
- Forks count: 3.001%
- Docker downloads count: 3.154%
- Dependent packages count: 3.173%
Maintainers (3)
- parano
- ssheng
- aar0npham

pypi.org: openllm-core

OpenLLM Core: Core components for OpenLLM.

Homepage: https://bentoml.com
Documentation: https://github.com/bentoml/OpenLLM/blob/main/openllm-core/README.md
Licenses: Apache Software License
Latest release: 0.5.7 (published almost 2 years ago)
Last Synced: 2026-03-11T02:47:50.876Z (3 months ago)
Versions: 85
Dependent Packages: 2
Dependent Repositories: 8
Downloads: 2,545 Last month
Docker Downloads: 47
Rankings:
- Downloads: 2.675%
- Docker downloads count: 3.154%
- Dependent packages count: 3.173%
- Average: 3.54%
- Dependent repos count: 5.156%
Maintainers (1)
- aar0npham

pypi.org: openllm-client

OpenLLM Client: Interacting with OpenLLM HTTP/gRPC server, or any BentoML server.

Homepage: https://bentoml.com
Documentation: https://github.com/bentoml/OpenLLM/blob/main/openllm-client/README.md
Licenses: Apache Software License
Latest release: 0.5.7 (published almost 2 years ago)
Last Synced: 2026-03-15T14:36:04.698Z (3 months ago)
Versions: 85
Dependent Packages: 2
Dependent Repositories: 8
Downloads: 1,028 Last month
Docker Downloads: 47
Rankings:
- Downloads: 2.697%
- Docker downloads count: 3.154%
- Average: 3.95%
- Dependent packages count: 4.791%
- Dependent repos count: 5.156%
Maintainers (1)
- aar0npham

proxy.golang.org: github.com/bentoml/OpenLLM

Homepage:
Documentation: https://pkg.go.dev/github.com/bentoml/OpenLLM#section-documentation
Licenses: apache-2.0
Latest release: v0.6.30 (published about 1 year ago)
Last Synced: 2026-03-04T22:03:23.280Z (3 months ago)
Versions: 194
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 6.528%
- Average: 6.747%
- Dependent repos count: 6.966%

proxy.golang.org: github.com/bentoml/openllm

Homepage:
Documentation: https://pkg.go.dev/github.com/bentoml/openllm#section-documentation
Licenses: apache-2.0
Latest release: v0.6.30 (published about 1 year ago)
Last Synced: 2026-03-22T13:04:01.690Z (2 months ago)
Versions: 194
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 6.528%
- Average: 6.747%
- Dependent repos count: 6.966%

npmjs.org: openllm

TS/JS binding for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM#readme
Licenses: Apache 2.0
Latest release: 0.0.2 (published almost 3 years ago)
Last Synced: 2026-03-13T01:33:55.441Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 2 Last month
Rankings:
- Stargazers count: 1.307%
- Forks count: 1.925%
- Average: 23.86%
- Dependent repos count: 37.487%
- Dependent packages count: 54.722%
Maintainers (1)
- aarnphm

npmjs.org: openllm_client

TS/JS binding for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM#readme
Licenses: Apache 2.0
Latest release: 0.0.2 (published almost 3 years ago)
Last Synced: 2026-03-12T19:47:04.394Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 2 Last month
Rankings:
- Stargazers count: 1.307%
- Forks count: 1.925%
- Average: 23.86%
- Dependent repos count: 37.487%
- Dependent packages count: 54.722%
Maintainers (1)
- aarnphm

nixpkgs-23.11: python310Packages.openllm

Operating LLMs in production

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-python
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm/default.nix#L184
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-08T23:30:07.918Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- natsukium
- happysalada

nixpkgs-24.05: python312Packages.openllm-core

Core components for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-core
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm-core/default.nix#L103
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-09T05:01:04.103Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- natsukium

nixpkgs-24.05: python312Packages.openllm

Operating LLMs in production

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-python
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm/default.nix#L193
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-09T05:01:02.910Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- happysalada
- natsukium

nixpkgs-23.11: python310Packages.openllm-client

Interacting with OpenLLM HTTP/gRPC server, or any BentoML server

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-client
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm-client/default.nix#L60
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-03T22:35:58.518Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
Maintainers (1)
- natsukium

nixpkgs-unstable: openllm

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud

Homepage: https://github.com/bentoml/OpenLLM
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/by-name/op/openllm/package.nix#L64
Licenses: Apache-2.0
Latest release: 0.6.30 (published 4 months ago)
Last Synced: 2026-03-03T13:13:05.015Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
Maintainers (2)
- happysalada
- natsukium

nixpkgs-23.11: python311Packages.openllm-client

Interacting with OpenLLM HTTP/gRPC server, or any BentoML server

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-client
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm-client/default.nix#L60
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-09T00:22:15.998Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- natsukium

nixpkgs-23.11: python310Packages.openllm-core

Core components for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-core
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm-core/default.nix#L84
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-08T23:30:30.360Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- natsukium

nixpkgs-24.05: python311Packages.openllm

Operating LLMs in production

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-python
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm/default.nix#L193
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-07T04:05:43.925Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
Maintainers (2)
- happysalada
- natsukium

nixpkgs-24.05: python312Packages.openllm-client

Interacting with OpenLLM HTTP/gRPC server, or any BentoML server

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-client
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm-client/default.nix#L64
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-07T13:01:36.430Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- natsukium

nixpkgs-24.11: openllm

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud

Homepage: https://github.com/bentoml/OpenLLM
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.11/pkgs/by-name/op/openllm/package.nix#L66
Licenses: Apache-2.0
Latest release: 0.6.10 (published 4 months ago)
Last Synced: 2026-03-08T00:18:01.004Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- natsukium
- happysalada

nixpkgs-23.11: python311Packages.openllm

Operating LLMs in production

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-python
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm/default.nix#L184
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-07T10:34:03.334Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (2)
- natsukium
- happysalada

nixpkgs-23.11: python311Packages.openllm-core

Core components for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-core
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-23.11/pkgs/development/python-modules/openllm-core/default.nix#L84
Licenses: Apache-2.0
Latest release: 0.4.22 (published 4 months ago)
Last Synced: 2026-03-07T10:34:17.285Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Maintainers (1)
- natsukium

nixpkgs-24.05: python311Packages.openllm-client

Interacting with OpenLLM HTTP/gRPC server, or any BentoML server

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-client
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm-client/default.nix#L64
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-07T04:01:26.982Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
Maintainers (1)
- natsukium

nixpkgs-24.05: python311Packages.openllm-core

Core components for OpenLLM

Homepage: https://github.com/bentoml/OpenLLM/tree/main/openllm-core
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/development/python-modules/openllm-core/default.nix#L103
Licenses: Apache-2.0
Latest release: 0.4.44 (published 4 months ago)
Last Synced: 2026-03-07T03:04:46.833Z (3 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
Maintainers (1)
- natsukium

Dependencies

.github/workflows/create-releases.yml actions

actions/checkout 08c6903cd8c0fde910a37f88322edcfb5dd907a8 composite
actions/download-artifact 634f93cb2916e3fdff6788551b99b062d0335ce0 composite
actions/setup-python e797f83bcb11b83ae66e0230d6156d7c80228e7c composite
actions/upload-artifact ea165f8d65b6e75b540449e92b4886f43607fa02 composite
pypa/gh-action-pypi-publish ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e composite
softprops/action-gh-release 6cbd405e2c4e67a21c47fa9e383d020e4e28b836 composite

.github/workflows/dependabot-auto-merge.yml actions

dependabot/fetch-metadata 08eff52bf64351f401fb50d4972fa95b9f2c2d1b composite

pyproject.toml pypi

attrs *
bentoml ==1.4.23
dulwich *
hf-xet *
huggingface-hub *
nvidia-ml-py *
openai ==1.90.0
pip-requirements-parser *
psutil *
pyaml *
questionary *
tabulate *
typer *
typing-extensions >=4.12.2
uv *

uv.lock pypi

105 dependencies