awesome-llama: https://github.com/Relaxed-System-Lab/HexGen
decentralized deep-learning generative-inference heterogeneous large-language-models llama2 machine-learning model-serving
Score: 5.075173815233827
Last synced: 1 day ago
JSON representation
Repository metadata:
[ICML 2024] Serving LLMs on heterogeneous decentralized clusters.
- Host: GitHub
- URL: https://github.com/Relaxed-System-Lab/HexGen
- Owner: Relaxed-System-Lab
- License: apache-2.0
- Created: 2023-11-30T11:17:54.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-06T09:19:34.000Z (over 2 years ago)
- Last Synced: 2026-08-01T14:06:45.042Z (12 days ago)
- Topics: decentralized, deep-learning, generative-inference, heterogeneous, large-language-models, llama2, machine-learning, model-serving
- Language: Python
- Homepage:
- Size: 959 KB
- Stars: 37
- Watchers: 3
- Forks: 3
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Owner metadata:
- Name: Relaxed System Lab
- Login: Relaxed-System-Lab
- Email: biyuan@cse.ust.hk
- Kind: organization
- Description: The research lab lead by Binhang Yuan @ CSE HKUST
- Website:
- Location: Hong Kong
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/143859082?v=4
- Repositories: 3
- Last Synced at: 2024-04-15T13:49:49.857Z
- Profile URL: https://github.com/Relaxed-System-Lab
GitHub Events
Total
- Fork event: 2
- Issues event: 2
- Watch event: 14
- Total: 18
Last Year
- Watch event: 1
- Total: 1
Committers metadata
Last synced: 3 days ago
Total Commits: 18
Total Committers: 4
Avg Commits per committer: 4.5
Development Distribution Score (DDS): 0.278
Commits in past year: 0
Committers in past year: 0
Avg Commits per committer in past year: 0.0
Development Distribution Score (DDS) in past year: 0.0
| Name | Commits | |
|---|---|---|
| Youhe Jiang | 8****g | 13 |
| jiangyouhe | j****e@j****l | 2 |
| ryan | r****n@r****l | 2 |
| ranyangit | r****u@g****m | 1 |
Issue and Pull Request metadata
Last synced: 2 days ago
Total issues: 4
Total pull requests: 1
Average time to close issues: about 4 hours
Average time to close pull requests: less than a minute
Total issue authors: 3
Total pull request authors: 1
Average comments per issue: 0.75
Average comments per pull request: 0.0
Merged pull request: 1
Bot issues: 0
Bot pull requests: 0
Past year issues: 3
Past year pull requests: 0
Past year average time to close issues: N/A
Past year average time to close pull requests: N/A
Past year issue authors: 2
Past year pull request authors: 0
Past year average comments per issue: 0.33
Past year average comments per pull request: 0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- AmitTalawar (2)
- siddhu1716 (1)
- Esthesia (1)
Top Pull Request Authors
- Youhe-Jiang (1)
Top Issue Labels
Top Pull Request Labels
Dependencies
third_party/ocf/src/ocf-core/Dockerfile
docker
- golang alpine build
third_party/ocf/src/ocf-core/go.mod
go
- aead.dev/minisign v0.2.0
- ariga.io/atlas v0.12.0
- entgo.io/ent v0.12.3
- github.com/agext/levenshtein v1.2.3
- github.com/apparentlymart/go-textseg/v13 v13.0.0
- github.com/authorizerdev/authorizer-go v0.0.0-20221101045102-34c6220aeb6c
- github.com/benbjohnson/clock v1.3.5
- github.com/beorn7/perks v1.0.1
- github.com/bytedance/sonic v1.9.1
- github.com/c-robinson/iplib v1.0.6
- github.com/cenkalti/backoff/v4 v4.2.0
- github.com/cespare/xxhash/v2 v2.2.0
- github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311
- github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be
- github.com/containerd/cgroups v1.1.0
- github.com/coreos/go-systemd/v22 v22.5.0
- github.com/cpuguy83/go-md2man/v2 v2.0.2
- github.com/creachadair/otp v0.4.0
- github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c
- github.com/deckarep/golang-set/v2 v2.1.0
- github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0
- github.com/docker/go-units v0.5.0
- github.com/elastic/gosigar v0.14.2
- github.com/ethereum/go-ethereum v1.12.0
- github.com/flynn/noise v1.0.0
- github.com/francoispqt/gojay v1.2.13
- github.com/fsnotify/fsnotify v1.6.0
- github.com/gabriel-vasile/mimetype v1.4.2
- github.com/getsentry/sentry-go v0.22.0
- github.com/gin-contrib/sse v0.1.0
- github.com/gin-gonic/gin v1.9.1
- github.com/go-co-op/gocron v1.28.3
- github.com/go-logr/logr v1.2.4
- github.com/go-logr/stdr v1.2.2
- github.com/go-ole/go-ole v1.2.6
- github.com/go-openapi/inflect v0.19.0
- github.com/go-playground/locales v0.14.1
- github.com/go-playground/universal-translator v0.18.1
- github.com/go-playground/validator/v10 v10.14.0
- github.com/go-stack/stack v1.8.1
- github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
- github.com/goccy/go-json v0.10.2
- github.com/godbus/dbus/v5 v5.1.0
- github.com/gofrs/uuid v4.4.0+incompatible
- github.com/gogo/protobuf v1.3.2
- github.com/golang/mock v1.6.0
- github.com/golang/protobuf v1.5.3
- github.com/google/btree v1.1.2
- github.com/google/go-cmp v0.5.9
- github.com/google/gopacket v1.1.19
- github.com/google/pprof v0.0.0-20230602150820-91b7bce49751
- github.com/google/uuid v1.3.0
- github.com/gorilla/websocket v1.5.0
- github.com/hashicorp/errwrap v1.1.0
- github.com/hashicorp/go-multierror v1.1.1
- github.com/hashicorp/golang-lru v0.5.4
- github.com/hashicorp/golang-lru/v2 v2.0.2
- github.com/hashicorp/hcl v1.0.0
- github.com/hashicorp/hcl/v2 v2.17.0
- github.com/holiman/uint256 v1.2.2-0.20230321075855-87b91420868c
- github.com/huin/goupnp v1.2.0
- github.com/inconshreveable/mousetrap v1.1.0
- github.com/ipfs/boxo v0.10.0
- github.com/ipfs/go-cid v0.4.1
- github.com/ipfs/go-datastore v0.6.0
- github.com/ipfs/go-ipfs-util v0.0.3
- github.com/ipfs/go-log v1.0.5
- github.com/ipfs/go-log/v2 v2.5.1
- github.com/ipld/go-ipld-prime v0.20.0
- github.com/jackpal/go-nat-pmp v1.0.2
- github.com/jbenet/go-temp-err-catcher v0.1.0
- github.com/jbenet/goprocess v0.1.4
- github.com/json-iterator/go v1.1.12
- github.com/klauspost/compress v1.16.6
- github.com/klauspost/cpuid/v2 v2.2.5
- github.com/koron/go-ssdp v0.0.4
- github.com/labstack/echo/v4 v4.10.2
- github.com/labstack/gommon v0.4.0
- github.com/leodido/go-urn v1.2.4
- github.com/libp2p/go-buffer-pool v0.1.0
- github.com/libp2p/go-cidranger v1.1.0
- github.com/libp2p/go-flow-metrics v0.1.0
- github.com/libp2p/go-libp2p v0.28.0
- github.com/libp2p/go-libp2p-asn-util v0.3.0
- github.com/libp2p/go-libp2p-gostream v0.6.0
- github.com/libp2p/go-libp2p-http v0.5.0
- github.com/libp2p/go-libp2p-kad-dht v0.24.1
- github.com/libp2p/go-libp2p-kbucket v0.6.3
- github.com/libp2p/go-libp2p-pubsub v0.9.3
- github.com/libp2p/go-libp2p-record v0.2.0
- github.com/libp2p/go-mplex v0.7.0
- github.com/libp2p/go-msgio v0.3.0
- github.com/libp2p/go-nat v0.2.0
- github.com/libp2p/go-netroute v0.2.1
- github.com/libp2p/go-reuseport v0.3.0
- github.com/libp2p/go-yamux/v4 v4.0.1
- github.com/libp2p/zeroconf/v2 v2.2.0
- github.com/magiconair/properties v1.8.7
- github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd
- github.com/mattn/go-colorable v0.1.13
- github.com/mattn/go-isatty v0.0.19
- github.com/matttproud/golang_protobuf_extensions v1.0.4
- github.com/miekg/dns v1.1.54
- github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b
- github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc
- github.com/minio/highwayhash v1.0.2
- github.com/minio/selfupdate v0.6.0
- github.com/minio/sha256-simd v1.0.1
- github.com/mitchellh/go-homedir v1.1.0
- github.com/mitchellh/go-wordwrap v1.0.1
- github.com/mitchellh/mapstructure v1.5.0
- github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
- github.com/modern-go/reflect2 v1.0.2
- github.com/mr-tron/base58 v1.2.0
- github.com/mudler/edgevpn v0.23.1
- github.com/mudler/water v0.0.0-20221010214108-8c7313014ce0
- github.com/multiformats/go-base32 v0.1.0
- github.com/multiformats/go-base36 v0.2.0
- github.com/multiformats/go-multiaddr v0.9.0
- github.com/multiformats/go-multiaddr-dns v0.3.1
- github.com/multiformats/go-multiaddr-fmt v0.1.0
- github.com/multiformats/go-multibase v0.2.0
- github.com/multiformats/go-multicodec v0.9.0
- github.com/multiformats/go-multihash v0.2.3
- github.com/multiformats/go-multistream v0.4.1
- github.com/multiformats/go-varint v0.0.7
- github.com/nakabonne/tstorage v0.3.6
- github.com/nats-io/jwt/v2 v2.4.1
- github.com/nats-io/nats-server/v2 v2.9.19
- github.com/nats-io/nats.go v1.27.1
- github.com/nats-io/nkeys v0.4.4
- github.com/nats-io/nuid v1.0.1
- github.com/onsi/ginkgo/v2 v2.10.0
- github.com/opencontainers/runtime-spec v1.0.2
- github.com/opentracing/opentracing-go v1.2.0
- github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
- github.com/pelletier/go-toml/v2 v2.0.8
- github.com/peterbourgon/diskv v2.0.1+incompatible
- github.com/pkg/errors v0.9.1
- github.com/polydawn/refmt v0.89.0
- github.com/prometheus/client_golang v1.16.0
- github.com/prometheus/client_model v0.4.0
- github.com/prometheus/common v0.44.0
- github.com/prometheus/procfs v0.10.1
- github.com/quic-go/qpack v0.4.0
- github.com/quic-go/qtls-go1-19 v0.3.2
- github.com/quic-go/qtls-go1-20 v0.2.2
- github.com/quic-go/quic-go v0.33.0
- github.com/quic-go/webtransport-go v0.5.3
- github.com/raulk/go-watchdog v1.3.0
- github.com/robfig/cron/v3 v3.0.1
- github.com/russross/blackfriday/v2 v2.1.0
- github.com/sethgrid/pester v1.2.0
- github.com/shirou/gopsutil v3.21.11+incompatible
- github.com/songgao/packets v0.0.0-20160404182456-549a10cd4091
- github.com/spaolacci/murmur3 v1.1.0
- github.com/spf13/afero v1.9.5
- github.com/spf13/cast v1.5.1
- github.com/spf13/cobra v1.7.0
- github.com/spf13/jwalterweatherman v1.1.0
- github.com/spf13/pflag v1.0.5
- github.com/spf13/viper v1.16.0
- github.com/subosito/gotenv v1.4.2
- github.com/tklauser/go-sysconf v0.3.11
- github.com/tklauser/numcpus v0.6.1
- github.com/twitchyliquid64/golang-asm v0.15.1
- github.com/ugorji/go/codec v1.2.11
- github.com/urfave/cli v1.22.10
- github.com/valyala/bytebufferpool v1.0.0
- github.com/valyala/fasttemplate v1.2.2
- github.com/vishvananda/netlink v1.1.0
- github.com/vishvananda/netns v0.0.0-20211101163701-50045581ed74
- github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1
- github.com/yusufpapurcu/wmi v1.2.3
- github.com/zclconf/go-cty v1.13.2
- go.opencensus.io v0.24.0
- go.opentelemetry.io/otel v1.16.0
- go.opentelemetry.io/otel/metric v1.16.0
- go.opentelemetry.io/otel/trace v1.16.0
- go.uber.org/atomic v1.11.0
- go.uber.org/dig v1.17.0
- go.uber.org/fx v1.20.0
- go.uber.org/multierr v1.11.0
- go.uber.org/zap v1.24.0
- golang.org/x/arch v0.3.0
- golang.org/x/crypto v0.10.0
- golang.org/x/exp v0.0.0-20230522175609-2e198f4a06a1
- golang.org/x/mod v0.11.0
- golang.org/x/net v0.11.0
- golang.org/x/sync v0.3.0
- golang.org/x/sys v0.9.0
- golang.org/x/text v0.10.0
- golang.org/x/time v0.3.0
- golang.org/x/tools v0.10.0
- golang.zx2c4.com/wintun v0.0.0-20211104114900-415007cec224
- golang.zx2c4.com/wireguard v0.0.0-20220703234212-c31a7b1ab478
- golang.zx2c4.com/wireguard/windows v0.5.3
- gonum.org/v1/gonum v0.13.0
- google.golang.org/protobuf v1.30.0
- gopkg.in/ini.v1 v1.67.0
- gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce
- gopkg.in/yaml.v2 v2.4.0
- gopkg.in/yaml.v3 v3.0.1
- lukechampine.com/blake3 v1.2.1
third_party/ocf/src/ocf-core/go.sum
go
- 899 dependencies
requirements.txt
pypi
- aiohttp *
- h5py *
- loguru *
- nats-py *
- ninja *
- packaging *
- pandas *
- protobuf *
- pynvml *
- regex *
- sentencepiece *
- six *
- tqdm *
- transformers *
third_party/megatron/megatron/core/requirements.txt
pypi
- torch *
third_party/ocf/docs/requirements.txt
pypi
- mkdocs *
- mkdocs-material *
- mkdocs-minify-plugin *
third_party/ocf/src/ocf-cli/pyproject.toml
pypi
- click *
- huggingface-hub *
- humanize *
- loguru *
- nats-py *
- netifaces *
- psutil *
- pynvml *
- requests *
- rich *
- typer *