awesome-llama: https://github.com/alibaba/rtp-llm
gpt inference llama llm llm-serving llmops model-serving
Score: 11.992495688085175
Last synced: about 13 hours ago
JSON representation
Repository metadata:
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
- Host: GitHub
- URL: https://github.com/alibaba/rtp-llm
- Owner: alibaba
- License: apache-2.0
- Created: 2023-12-27T08:22:59.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2026-08-10T17:36:42.000Z (3 days ago)
- Last Synced: 2026-08-10T17:37:11.123Z (3 days ago)
- Topics: gpt, inference, llama, llm, llm-serving, llmops, model-serving
- Language: Cuda
- Homepage:
- Size: 567 MB
- Stars: 1,299
- Watchers: 19
- Forks: 255
- Open Issues: 183
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
- Support: docs/supported_models/embedding_models.md
- Notice: NOTICE
Owner metadata:
- Name: Alibaba
- Login: alibaba
- Email:
- Kind: organization
- Description: Alibaba Open Source
- Website: https://opensource.alibaba.com/
- Location: Hangzhou, China
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/1961952?v=4
- Repositories: 420
- Last Synced at: 2024-05-21T04:56:07.141Z
- Profile URL: https://github.com/alibaba
GitHub Events
Total
- Commit comment event: 1
- Create event: 115
- Delete event: 24
- Fork event: 26
- Issue comment event: 209
- Issues event: 31
- Member event: 13
- Pull request event: 131
- Pull request review comment event: 196
- Pull request review event: 204
- Push event: 1212
- Release event: 1
- Watch event: 321
- Total: 2484
Last Year
- Commit comment event: 1
- Create event: 115
- Delete event: 24
- Fork event: 25
- Issue comment event: 195
- Issues event: 19
- Member event: 13
- Pull request event: 129
- Pull request review comment event: 196
- Pull request review event: 204
- Push event: 1212
- Release event: 1
- Watch event: 294
- Total: 2428
Committers metadata
Last synced: 7 days ago
Total Commits: 4,712
Total Committers: 109
Avg Commits per committer: 43.229
Development Distribution Score (DDS): 0.315
Commits in past year: 1,719
Committers in past year: 83
Avg Commits per committer in past year: 20.711
Development Distribution Score (DDS) in past year: 0.787
| Name | Commits | |
|---|---|---|
| rtp-llm | r****m@a****a | 3228 |
| baowending.bwd | b****d@a****m | 96 |
| JackTan25 | t****y@a****m | 96 |
| liukan.lk | l****k@a****m | 89 |
| huzetao.hzt | h****t@a****m | 66 |
| luoli.hn | l****n@t****m | 64 |
| wangyin | w****x@a****m | 53 |
| wln | y****n@a****m | 51 |
| Nancheng | y****j@a****m | 42 |
| luoxi | l****q@a****m | 41 |
| wht21 | w****t@a****m | 38 |
| 蝉吟 | l****d@a****m | 38 |
| serina.wzq | s****q@a****m | 37 |
| brucelee.ly | b****y@a****m | 36 |
| feifei14119 | f****w@a****m | 36 |
| xinfei.sxf | x****f@a****m | 33 |
| liaochenzhi.lcz | l****z@a****m | 33 |
| Zhao An | z****n@a****m | 33 |
| lxq | l****2@a****m | 29 |
| zw193905 | z****5@a****m | 25 |
| silu.zsl | s****l@a****m | 23 |
| jacobwin-ai | 1****9@q****m | 23 |
| zhangzhi | 1****4@p****n | 20 |
| fanfengfeng.fff | f****f@a****m | 20 |
| hxy0118 | 1****9@q****m | 19 |
| chengshu-lcc | l****c@a****m | 18 |
| xingjunna.xjn | x****n@a****m | 18 |
| YilinZhao | y****o@a****m | 17 |
| zhangjianning.zjn | z****n@a****m | 17 |
| kedi | z****f@a****m | 17 |
| and 79 more... | ||
Issue and Pull Request metadata
Last synced: about 13 hours ago
Total issues: 102
Total pull requests: 414
Average time to close issues: 10 days
Average time to close pull requests: 11 days
Total issue authors: 62
Total pull request authors: 96
Average comments per issue: 1.52
Average comments per pull request: 5.96
Merged pull request: 150
Bot issues: 0
Bot pull requests: 1
Past year issues: 3
Past year pull requests: 348
Past year average time to close issues: N/A
Past year average time to close pull requests: 13 days
Past year issue authors: 3
Past year pull request authors: 84
Past year average comments per issue: 0.33
Past year average comments per pull request: 6.81
Past year merged pull request: 102
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- samaritan1998 (21)
- invisifire (7)
- frankang (4)
- Greatpanc (3)
- liucw2012 (3)
- cwlseu (3)
- xinge666 (2)
- Lzhang-hub (2)
- zf761 (2)
- skmkt (2)
- Vincent131499 (2)
- FelixZhang7 (1)
- yato1997 (1)
- leyiwang (1)
- supercocoa (1)
Top Pull Request Authors
- Oneydauh (17)
- feifei14119 (17)
- wanglining97 (16)
- Vinkle-hzt (16)
- jianglan89 (15)
- siluzhou (15)
- Proton02 (15)
- JackTan25 (14)
- ydshi0 (13)
- wht21 (12)
- zhangjianning-zjn (12)
- Reyfone (12)
- wenhuanh (10)
- Adrenaline-S (10)
- HoniiTro19 (10)
Top Issue Labels
- enhancement (1)
Top Pull Request Labels
- python (1)
- enhancement (1)
- dependencies (1)
Dependencies
- 32bit/debian latest build
- debian latest build
- grpctesting/rake-compiler-dock_53c22085d091183c528303791e7771359f699bcf latest build
- debian jessie build
- debian stretch build
- ubuntu latest build
- debian jessie build
- 32bit/debian jessie build
- debian jessie build
- debian stretch build
- debian jessie build
- centos 6.9 build
- quay.io/pypa/manylinux1_x86_64 latest build
- com.google.caliper:caliper 1.0-beta-2
- com.google.protobuf:protobuf-java ${protobuf.version} system
- junit:junit 4.4
- org.easymock:easymock 2.2
- org.easymock:easymockclassextension 2.2.1
- com.google.protobuf.compatibility:compatibility-protos 2.5.0
- com.google.protobuf:protobuf-java ${more_protos.protobuf.version}
- com.google.protobuf:protobuf-java ${protos.protobuf.version}
- com.google.protobuf.compatibility:compatibility-more-protos 2.5.0
- com.google.protobuf.compatibility:compatibility-protos 2.5.0
- com.google.protobuf:protobuf-java ${tests.protobuf.version}
- junit:junit 4.4
- org.easymock:easymock 2.2
- org.easymock:easymockclassextension 2.2.1
- com.google.guava:guava test
- junit:junit test
- org.easymock:easymock test
- org.easymock:easymockclassextension test
- com.google.guava:guava test
- junit:junit test
- org.easymock:easymock test
- org.easymock:easymockclassextension test
- com.google.code.gson:gson 2.7
- com.google.errorprone:error_prone_annotations 2.3.2
- com.google.guava:guava
- com.google.protobuf:protobuf-java
- junit:junit
- org.easymock:easymock
- org.easymock:easymockclassextension
- com.google.guava:guava-testlib test
- org.jruby:jruby-complete 1.7.13 provided
- com.fasterxml.jackson.core:jackson-core 2.4.3
- com.google.protobuf:protobuf-java 3.0.0
- glob ~6.0.4 development
- google-closure-compiler ~20190301.0.0 development
- google-closure-library ~20190301.0.0 development
- gulp ~4.0.1 development
- jasmine ~2.4.1 development
- NUnit 3.6.1
- NUnitLite 3.6.1
- Microsoft.SourceLink.GitHub 1.0.0-beta2-18618-05 development
- System.Memory 4.5.2
- BenchmarkDotNet 0.11.4
- Microsoft.NET.Test.Sdk 15.5.0
- NUnit 3.9.0
- NUnit3TestAdapter 3.9.0
- phpunit/phpunit >=4.8.0 development
- php >=5.5.0
- phpunit/phpunit ^5|^4.8.0 development
- php >=5.5.0
- protobuf
- setuptools *
- six >=1.9
- setuptools *
- six >=1.9
- rake-compiler ~> 0.9.5 development
- rake-compiler-dock ~> 0.6.0 development
- rubygems-tasks ~> 0.2.4 development
- test-unit ~> 3.0, >= 3.0.9 development
- actions/checkout v4 composite
- actions/checkout v3 composite
- 125 dependencies
- apache-tvm-ffi ==0.1.1
- auto_gptq *
- autoawq >=0.2.9
- datasets *
- flashinfer-python ==0.2.5
- nvidia-nvshmem-cu12 ==3.4.5
- tensorrt ==10.3.0
- tensorrt-cu12-bindings ==10.3.0
- tensorrt-cu12-libs ==10.3.0
- mysql-connector-python *
- safetensors ==0.7.0
- tabulate ==0.9.0
- pyrsmi ==0.2.0
- pyyaml ==6.0.2
- apache-tvm-ffi *
- auto_gptq *
- autoawq *
- datasets *
- mysql-connector-python *
- tabulate ==0.9.0
- tensorrt ==10.3.0
- tensorrt-cu12-bindings ==10.3.0
- tilelang ==0.1.6
- 169 dependencies
- Pillow *
- aiohttp *
- av ==16.1.0
- bitsandbytes >=0.41.0
- blobfile *
- concurrent_log_handler *
- cpm_kernels *
- dacite *
- dashscope >=1.11.0
- decord ==0.6.0
- einops *
- fastapi ==0.115.6
- filelock >=3.20.0
- grpcio ==1.62.0
- grpcio-tools ==1.57.0
- importlib_metadata *
- jieba *
- jinja2 *
- json5 *
- librosa *
- lru-dict *
- matplotlib *
- nest_asyncio *
- numpy <2.0a0,>=1.25
- onnx ==1.16.0
- openai *
- orjson *
- oss2 *
- partial_json_parser *
- pillow-avif-plugin ==1.5.2
- pillow-heif *
- portalocker *
- prettytable *
- protobuf ==4.25
- psutil *
- py-spy *
- pyOpenSSL ==24.1.0
- pybind11_stubgen *
- pydantic *
- pynvml *
- pyodps *
- safetensors *
- sentence-transformers ==5.2.0
- sentencepiece ==0.2.0
- setproctitle *
- setuptools ==60.5.0
- sympy *
- thrift *
- tiktoken ==0.7.0
- timm ==0.9.12
- transformers ==5.2.0
- typing-extensions *
- uvicorn ==0.30.0
- xfastertransformer_devel ==1.8.1.1
- xfastertransformer_devel_icx ==1.8.1.1
- 182 dependencies
- actions/checkout v4 composite
- mysql-connector-python *
- safetensors ==0.7.0
- tabulate ==0.9.0
- triton ==3.6.0
- aiohappyeyeballs ==2.7.1
- aiohttp ==3.14.3
- aiosignal ==1.4.0
- aliyun-python-sdk-core ==2.16.0
- aliyun-python-sdk-kms ==2.16.5
- annotated-doc ==0.0.4
- annotated-types ==0.7.0
- anyio ==4.14.2
- apache-tvm-ffi ==0.1.12
- async-timeout ==5.0.1
- attrs ==26.1.0
- audioread ==3.1.0
- backports-strenum ==1.3.1
- bitsandbytes ==0.49.2
- blobfile ==3.2.0
- certifi ==2026.7.22
- cffi ==2.1.0
- charset-normalizer ==3.4.9
- click ==8.4.2
- cloudpickle ==3.1.2
- concurrent-log-handler ==0.9.29
- contourpy ==1.3.2
- cpm-kernels ==1.0.11
- crcmod ==1.7
- cryptography ==42.0.8
- cuda-bindings ==13.3.1
- cuda-core ==1.0.1
- cuda-pathfinder ==1.6.0
- cuda-python ==13.3.1
- cuda-tile ==1.5.0
- cuda-toolkit ==13.0.2
- cycler ==0.12.1
- dacite ==1.9.2
- dashscope ==1.26.4
- decorator ==5.3.1
- distro ==1.9.0
- einops ==0.8.2
- exceptiongroup ==1.3.1
- fastapi ==0.115.6
- filelock ==3.32.0
- flashinfer-python ==0.6.15.post1
- fonttools ==4.63.0
- frozenlist ==1.8.0
- fsspec ==2026.6.0
- grpcio ==1.62.0
- grpcio-tools ==1.57.0
- h11 ==0.16.0
- hf-xet ==1.5.2
- httpcore ==1.0.9
- httpx ==0.28.1
- httpx-sse ==0.4.3
- huggingface-hub ==1.24.0
- idna ==3.18
- importlib-metadata ==9.0.0
- jieba ==0.42.1
- jinja2 ==3.1.6
- jiter ==0.16.0
- jmespath ==0.10.0
- joblib ==1.5.3
- json5 ==0.15.0
- kiwisolver ==1.5.0
- lazy-loader ==0.5
- librosa ==0.11.0
- llvmlite ==0.48.0
- lru-dict ==1.4.1
- lxml ==6.1.1
- markdown-it-py ==4.2.0
- markupsafe ==3.0.3
- matplotlib ==3.10.9
- mdurl ==0.1.2
- ml-dtypes ==0.5.4
- mpmath ==1.3.0
- msgpack ==1.2.1
- multidict ==6.7.1
- mysql-connector-python ==9.7.0
- nccl4py ==0.3.1
- nest-asyncio ==1.6.0
- networkx ==3.4.2
- ninja ==1.13.0
- numba ==0.66.0
- numpy ==1.26.4
- nvidia-cublas ==13.1.0.3
- nvidia-cuda-cupti ==13.0.85
- nvidia-cuda-nvrtc ==13.0.88
- nvidia-cuda-runtime ==13.0.96
- nvidia-cudnn-cu13 ==9.19.0.56
- nvidia-cudnn-frontend ==1.26.0
- nvidia-cufft ==12.0.0.61
- nvidia-cufile ==1.15.1.6
- nvidia-curand ==10.4.0.35
- nvidia-cusolver ==12.0.4.66
- nvidia-cusparse ==12.6.3.3
- nvidia-cusparselt-cu13 ==0.8.0
- nvidia-cutlass-dsl ==4.5.3
- nvidia-cutlass-dsl-libs-base ==4.5.3
- nvidia-ml-py ==13.610.43
- nvidia-nccl-cu13 ==2.28.9
- nvidia-nvjitlink ==13.0.88
- nvidia-nvshmem-cu13 ==3.4.5
- nvidia-nvtx ==13.0.85
- onnx ==1.16.0
- openai ==2.47.0
- orjson ==3.11.9
- oss2 ==2.19.1
- packaging ==26.2
- partial-json-parser ==0.2.1.1.post7
- pillow ==12.3.0
- pillow-avif-plugin ==1.5.2
- pillow-heif ==1.5.0
- platformdirs ==4.11.0
- pooch ==1.9.0
- portalocker ==3.2.0
- prettytable ==3.18.0
- propcache ==0.5.2
- protobuf ==4.25.0
- psutil ==7.2.2
- py-spy ==0.4.2
- pyarrow ==25.0.0
- pybind11-stubgen ==2.5.5
- pycparser ==3.0
- pycryptodome ==3.23.0
- pycryptodomex ==3.23.0
- pydantic ==2.13.4
- pydantic-core ==2.46.4
- pygments ==2.20.0
- pynvml ==13.0.1
- pyodps ==0.13.0
- pyopenssl ==24.1.0
- pyparsing ==3.3.2
- python-dateutil ==2.9.0.post0
- pyyaml ==6.0.3
- regex ==2026.7.19
- requests ==2.34.2
- rich ==15.0.0
- safetensors ==0.7.0
- scikit-learn ==1.7.2
- scipy ==1.15.3
- sentence-transformers ==5.2.0
- sentencepiece ==0.2.0
- setproctitle ==1.3.7
- setuptools ==60.5.0
- shellingham ==1.5.4
- six ==1.17.0
- sniffio ==1.3.1
- soundfile ==0.14.0
- soxr ==1.1.0
- starlette ==0.41.3
- sympy ==1.14.0
- tabulate ==0.9.0
- threadpoolctl ==3.6.0
- thrift ==0.24.0
- tiktoken ==0.7.0
- timm ==0.9.12
- tokenizers ==0.22.2
- torch-c-dlpack-ext ==0.1.5
- tqdm ==4.69.0
- transformers ==5.2.0
- triton ==3.6.0
- typer ==0.27.0
- typer-slim ==0.24.0
- typing-extensions ==4.16.0
- typing-inspection ==0.4.2
- urllib3 ==2.7.0
- uvicorn ==0.30.0
- wcwidth ==0.8.2
- websocket-client ==1.9.0
- yarl ==1.24.5
- zipp ==4.1.0
- flashinfer-python ==0.2.5
- mysql-connector-python *
- 150 dependencies
- 165 dependencies
- 145 dependencies
- 134 dependencies