llama-moe

continual-pre-training expert-partition llama llm mixture-of-experts moe

Score: 8.52912176228151

Last synced: about 9 hours ago
JSON representation

Repository metadata:

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

Host: GitHub
URL: https://github.com/pjlab-sys4nlp/llama-moe
Owner: pjlab-sys4nlp
License: apache-2.0
Created: 2023-07-24T06:15:51.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2024-12-06T04:47:07.000Z (about 1 year ago)
Last Synced: 2026-01-30T08:40:06.639Z (10 days ago)
Topics: continual-pre-training, expert-partition, llama, llm, mixture-of-experts, moe
Language: Python
Homepage: https://arxiv.org/abs/2406.16554
Size: 1.69 MB
Stars: 1,005
Watchers: 8
Forks: 65
Open Issues: 7
Metadata Files:
- Readme: README.md
- License: LICENSE

Owner metadata:

Name: pjlab-sys4nlp
Login: pjlab-sys4nlp
Email:
Kind: organization
Description:
Website:
Location:
Twitter:
Company:
Icon url: https://avatars.githubusercontent.com/u/124339599?v=4
Repositories: 1
Last Synced at: 2023-10-12T20:41:01.548Z
Profile URL: https://github.com/pjlab-sys4nlp

GitHub Events

Total

Fork event: 12
Issue comment event: 11
Issues event: 7
Pull request event: 1
Push event: 2
Watch event: 122
Total: 155

Last Year

Fork event: 9
Issue comment event: 5
Issues event: 2
Watch event: 74
Total: 90

Committers metadata

Last synced: 16 days ago

Total Commits: 142
Total Committers: 5
Avg Commits per committer: 28.4
Development Distribution Score (DDS): 0.472

Commits in past year: 0
Committers in past year: 0
Avg Commits per committer in past year: 0.0
Development Distribution Score (DDS) in past year: 0.0

Name	Email	Commits
zhutong	t**7@o**m	75
Daize Dong	d**9@g**m	58
XiaoYee	5**8@q**m	4
jcruan	j**n@1**m	3
tongjingqi	2**3@q**m	2

Issue and Pull Request metadata

Last synced: 3 months ago

Total issues: 25
Total pull requests: 50
Average time to close issues: 12 days
Average time to close pull requests: 2 days
Total issue authors: 23
Total pull request authors: 4
Average comments per issue: 2.28
Average comments per pull request: 0.14
Merged pull request: 46
Bot issues: 0
Bot pull requests: 0

Past year issues: 6
Past year pull requests: 1
Past year average time to close issues: 3 days
Past year average time to close pull requests: N/A
Past year issue authors: 6
Past year pull request authors: 1
Past year average comments per issue: 1.67
Past year average comments per pull request: 0.0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/pjlab-sys4nlp/llama-moe

Top Issue Authors

rzr002 (2)
Minami-su (2)
18600709862 (1)
theblackcat102 (1)
bulaikexiansheng (1)
qiuzh20 (1)
DoubleVII (1)
xyjsjruiliu (1)
exhyy (1)
abhinand5 (1)
WangRongsheng (1)
Tangkexian (1)
apoorvcodes (1)
Xingxiangrui (1)
ZeyuTeng96 (1)

Top Pull Request Authors

DaizeDong (29)
Spico197 (23)
tongjingqi (1)
JCruan519 (1)

Top Issue Labels

Top Pull Request Labels

Dependencies

requirements.txt pypi

Pillow ==9.4.0
PyYaml ==6.0.1
accelerate ==0.21.0
black ==23.7.0
coverage ==7.2.7
datasets ==2.14.1
debugpy ==1.6.7
deepspeed ==0.10.0
flake8 ==6.0.0
huggingface-hub ==0.16.4
isort ==5.12.0
k-means-constrained ==0.7.3
matplotlib ==3.7.2
ninja ==1.11.1
nltk ==3.8.1
numpy ==1.25.0
omegaconf ==2.0.6
opencv-python ==4.8.1.78
packaging ==23.1
pandas <2.1.0
pebble ==5.0.3
peft ==0.4.0
pre-commit ==3.3.3
pynvml ==11.5.0
pytest ==7.4.0
python-dotenv ==1.0.0
safetensors ==0.3.1
scikit-learn ==1.3.0
sentence-transformers ==2.2.2
sentencepiece ==0.1.99
tensorboard ==2.13.0
tokenizers ==0.13.3
torch ==2.0.1
torchaudio ==2.0.2
torchvision ==0.15.2
tqdm ==4.65.0
transformers ==4.31.0
triton ==2.0.0
trl ==0.4.7
wandb ==0.15.6
xformers ==0.0.20

setup.py pypi

datasets ==2.14.1
omegaconf ==2.0.6
peft ==0.4.0
scikit-learn ==1.3.0
tensorboard ==2.13.0
tqdm ==4.65.0
transformers ==4.31.0

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary