awesome-llama: https://github.com/fcn94/llm_stream_endpoint

api candle cpu cuda cuda-kernels endpoint llama2 llm llm-server mistral-7b phi-2 quantized rest rest-api rust streaming

Score: 1.3862943611198906

Last synced: about 18 hours ago
JSON representation

Repository metadata:

Simple LLM Rest API using Rust, Warp and Candle. Dedicated for quantized version of either phi-2 ( default) , Mistral, or Llama. Work using CPU or CUDA

• Host: GitHub
• URL: https://github.com/fcn94/llm_stream_endpoint
• Owner: fcn94
• License: apache-2.0
• Created: 2023-11-02T17:25:46.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-02T20:43:26.000Z (about 2 years ago)
• Last Synced: 2025-10-28T21:48:01.226Z (7 months ago)
• Topics: api, candle, cpu, cuda, cuda-kernels, endpoint, llama2, llm, llm-server, mistral-7b, phi-2, quantized, rest, rest-api, rust, streaming
• Language: Rust
• Homepage:
• Size: 74.2 KB
• Stars: 4
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

---

Owner metadata:

• Name:
• Login: fcn94
• Email:
• Kind: user
• Description:
• Website:
• Location:
• Twitter:
• Company:
• Icon url: https://avatars.githubusercontent.com/u/116332106?v=4
• Repositories: 2
• Last Synced at: 2024-04-23T00:10:56.939Z
• Profile URL: https://github.com/fcn94

---

GitHub Events

Total

• Total: 0

Last Year

• Total: 0

---

Committers metadata

Last synced: 15 days ago

Name	Email	Commits
fcn94	f****t@g****m	51

---

Issue and Pull Request metadata

Last synced: 5 months ago

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/fcn94/llm_stream_endpoint

---

---

---

Dependencies