awesome-llama: https://github.com/jy-yuan/KIVI
inference large-language-models llama llm natural-language-processing quantization transformer
Score: 8.14002395246292
Last synced: about 13 hours ago
JSON representation
Repository metadata:
[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Host: GitHub
- URL: https://github.com/jy-yuan/KIVI
- Owner: jy-yuan
- License: mit
- Created: 2024-02-02T17:41:32.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-11-20T20:34:33.000Z (4 months ago)
- Last Synced: 2026-03-28T06:41:46.547Z (6 days ago)
- Topics: inference, large-language-models, llama, llm, natural-language-processing, quantization, transformer
- Language: Python
- Homepage: https://arxiv.org/abs/2402.02750
- Size: 16.7 MB
- Stars: 375
- Watchers: 7
- Forks: 46
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Owner metadata:
- Name: jiayi yuan
- Login: jy-yuan
- Email:
- Kind: user
- Description: CS PhD student @ Rice University, BS CS @ Tsinghua University
- Website:
- Location: USA
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/34369239?v=4
- Repositories: 2
- Last Synced at: 2023-03-03T16:47:00.181Z
- Profile URL: https://github.com/jy-yuan
GitHub Events
Total
- Fork event: 16
- Issue comment event: 20
- Issues event: 20
- Pull request event: 1
- Push event: 6
- Watch event: 104
- Total: 167
Last Year
- Fork event: 6
- Issue comment event: 6
- Issues event: 7
- Pull request event: 1
- Push event: 3
- Watch event: 42
- Total: 65
Committers metadata
Last synced: 3 days ago
Total Commits: 58
Total Committers: 9
Avg Commits per committer: 6.444
Development Distribution Score (DDS): 0.483
Commits in past year: 6
Committers in past year: 3
Avg Commits per committer in past year: 2.0
Development Distribution Score (DDS) in past year: 0.5
| Name | Commits | |
|---|---|---|
| jy-yuan | 1****8@q****m | 30 |
| Zirui Liu | z****5@d****u | 11 |
| Zirui Liu | 3****u | 10 |
| ray-liu | z****u@c****u | 2 |
| dependabot[bot] | 4****] | 1 |
| condy | c****9@g****m | 1 |
| Yifei Kong | k****g@y****e | 1 |
| QuiverDance | p****0@a****r | 1 |
| Luning Wang | w****2@g****m | 1 |
Issue and Pull Request metadata
Last synced: about 1 month ago
Total issues: 38
Total pull requests: 9
Average time to close issues: about 1 month
Average time to close pull requests: 1 day
Total issue authors: 34
Total pull request authors: 4
Average comments per issue: 1.34
Average comments per pull request: 0.11
Merged pull request: 6
Bot issues: 0
Bot pull requests: 0
Past year issues: 8
Past year pull requests: 0
Past year average time to close issues: about 2 months
Past year average time to close pull requests: N/A
Past year issue authors: 8
Past year pull request authors: 0
Past year average comments per issue: 1.13
Past year average comments per pull request: 0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- Felixvillas (4)
- dhjoo98 (2)
- Elycyx (1)
- JoesSattes (1)
- yifeikong (1)
- ascendpoet (1)
- xzwj1699 (1)
- ilur98 (1)
- riou-chen (1)
- ActiveSky (1)
- yuhuixu1993 (1)
- rayzr0123 (1)
- lky-violet (1)
- RalphMao (1)
- wuliangsixia (1)
Top Pull Request Authors
- Davids048 (4)
- wln20 (2)
- condy0919 (2)
- yifeikong (1)
Top Issue Labels
Top Pull Request Labels
Dependencies
- torch *
- absl-py =2.0.0=pypi_0
- accelerate =0.24.1=pypi_0
- aiohttp =3.9.1=pypi_0
- aiosignal =1.3.1=pypi_0
- annotated-types =0.6.0=pypi_0
- antlr4-python3-runtime =4.9.3=pypi_0
- antlr4-tools =0.2.1=pypi_0
- anyio =3.7.1=pypi_0
- asttokens =2.4.1=pypi_0
- async-timeout =4.0.3=pypi_0
- attributedict =0.3.0=pypi_0
- attrs =23.1.0=pypi_0
- awq =0.1.0=pypi_0
- bitsandbytes =0.41.2.post2=pypi_0
- blessed =1.20.0=pypi_0
- blessings =1.7=pypi_0
- bzip2 =1.0.8=h7b6447c_0
- ca-certificates =2023.08.22=h06a4308_0
- cachetools =5.3.2=pypi_0
- certifi =2023.11.17=pypi_0
- chardet =5.2.0=pypi_0
- charset-normalizer =3.3.2=pypi_0
- click =8.1.7=pypi_0
- codecov =2.1.13=pypi_0
- colorama =0.4.6=pypi_0
- coloredlogs =15.0.1=pypi_0
- colour-runner =0.1.1=pypi_0
- comm =0.2.0=pypi_0
- contourpy =1.2.0=pypi_0
- coverage =7.3.2=pypi_0
- cycler =0.12.1=pypi_0
- dataproperty =1.0.1=pypi_0
- datasets =2.15.0=pypi_0
- debugpy =1.8.0=pypi_0
- decorator =5.1.1=pypi_0
- deepdiff =6.7.1=pypi_0
- deepspeed =0.12.4=pypi_0
- dill =0.3.7=pypi_0
- distlib =0.3.7=pypi_0
- distro =1.8.0=pypi_0
- einops =0.7.0=pypi_0
- evaluate =0.4.1=pypi_0
- exceptiongroup =1.2.0=pypi_0
- executing =2.0.1=pypi_0
- filelock =3.13.1=pypi_0
- fonttools =4.45.1=pypi_0
- frozenlist =1.4.0=pypi_0
- fsspec =2023.10.0=pypi_0
- ftfy =6.1.3=pypi_0
- fuzzywuzzy =0.18.0=pypi_0
- gpustat =1.1.1=pypi_0
- h11 =0.14.0=pypi_0
- hjson =3.1.0=pypi_0
- httpcore =1.0.2=pypi_0
- httpx =0.25.2=pypi_0
- huggingface-hub =0.19.4=pypi_0
- humanfriendly =10.0=pypi_0
- idna =3.6=pypi_0
- importlib-resources =6.1.1=pypi_0
- inspecta =0.1.3=pypi_0
- install-jdk =1.1.0=pypi_0
- ipdb =0.13.13=pypi_0
- ipykernel =6.27.1=pypi_0
- ipython =8.18.1=pypi_0
- jedi =0.19.1=pypi_0
- jieba =0.42.1=pypi_0
- jinja2 =3.1.2=pypi_0
- joblib =1.3.2=pypi_0
- jsonlines =4.0.0=pypi_0
- jupyter-client =8.6.0=pypi_0
- jupyter-core =5.5.0=pypi_0
- kiwisolver =1.4.5=pypi_0
- ld_impl_linux-64 =2.38=h1181459_1
- libffi =3.4.4=h6a678d5_0
- libgcc-ng =11.2.0=h1234567_1
- libgomp =11.2.0=h1234567_1
- libstdcxx-ng =11.2.0=h1234567_1
- libuuid =1.41.5=h5eee18b_0
- lm-eval =0.3.0=dev_0
- markupsafe =2.1.3=pypi_0
- matplotlib =3.8.2=pypi_0
- matplotlib-inline =0.1.6=pypi_0
- mbstrdecoder =1.1.3=pypi_0
- mpmath =1.3.0=pypi_0
- multidict =6.0.4=pypi_0
- multiprocess =0.70.15=pypi_0
- ncurses =6.4=h6a678d5_0
- nest-asyncio =1.5.8=pypi_0
- networkx =3.2.1=pypi_0
- ninja =1.11.1.1=pypi_0
- nltk =3.8.1=pypi_0
- numexpr =2.8.7=pypi_0
- numpy =1.26.2=pypi_0
- nvidia-cublas-cu12 =12.1.3.1=pypi_0
- nvidia-cuda-cupti-cu12 =12.1.105=pypi_0
- nvidia-cuda-nvrtc-cu12 =12.1.105=pypi_0
- nvidia-cuda-runtime-cu12 =12.1.105=pypi_0
- nvidia-cudnn-cu12 =8.9.2.26=pypi_0
- nvidia-cufft-cu12 =11.0.2.54=pypi_0
- nvidia-curand-cu12 =10.3.2.106=pypi_0
- nvidia-cusolver-cu12 =11.4.5.107=pypi_0
- nvidia-cusparse-cu12 =12.1.0.106=pypi_0
- nvidia-ml-py =12.535.133=pypi_0
- nvidia-nccl-cu12 =2.18.1=pypi_0
- nvidia-nvjitlink-cu12 =12.3.101=pypi_0
- nvidia-nvtx-cu12 =12.1.105=pypi_0
- omegaconf =2.3.0=pypi_0
- openai =1.3.5=pypi_0
- openssl =3.0.12=h7f8727e_0
- ordered-set =4.1.0=pypi_0
- packaging =23.2=pypi_0
- pandas =2.1.3=pypi_0
- parso =0.8.3=pypi_0
- pathvalidate =3.2.0=pypi_0
- peft =0.6.2=pypi_0
- pexpect =4.9.0=pypi_0
- pillow =10.1.0=pypi_0
- pip =23.3=py310h06a4308_0
- platformdirs =4.0.0=pypi_0
- pluggy =1.3.0=pypi_0
- portalocker =2.8.2=pypi_0
- prompt-toolkit =3.0.41=pypi_0
- protobuf =4.25.1=pypi_0
- psutil =5.9.6=pypi_0
- ptyprocess =0.7.0=pypi_0
- pure-eval =0.2.2=pypi_0
- py-cpuinfo =9.0.0=pypi_0
- pyarrow =14.0.1=pypi_0
- pyarrow-hotfix =0.6=pypi_0
- pybind11 =2.11.1=pypi_0
- pycountry =22.3.5=pypi_0
- pydantic =2.5.2=pypi_0
- pydantic-core =2.14.5=pypi_0
- pygments =2.17.2=pypi_0
- pynvml =11.5.0=pypi_0
- pyparsing =3.1.1=pypi_0
- pyproject-api =1.6.1=pypi_0
- pytablewriter =1.2.0=pypi_0
- python =3.10.13=h955ad1f_0
- python-dateutil =2.8.2=pypi_0
- pytz =2023.3.post1=pypi_0
- pyyaml =6.0.1=pypi_0
- pyzmq =25.1.1=pypi_0
- readline =8.2=h5eee18b_0
- regex =2023.10.3=pypi_0
- requests =2.31.0=pypi_0
- responses =0.18.0=pypi_0
- rootpath =0.1.1=pypi_0
- rouge =1.0.1=pypi_0
- rouge-score =0.1.2=pypi_0
- sacrebleu =1.5.0=pypi_0
- safetensors =0.4.1=pypi_0
- scikit-learn =1.3.2=pypi_0
- scipy =1.11.4=pypi_0
- sentencepiece =0.1.99=pypi_0
- setuptools =68.0.0=py310h06a4308_0
- six =1.16.0=pypi_0
- sniffio =1.3.0=pypi_0
- sqlite =3.41.2=h5eee18b_0
- sqlitedict =2.1.0=pypi_0
- stack-data =0.6.3=pypi_0
- sympy =1.12=pypi_0
- tabledata =1.3.3=pypi_0
- tcolorpy =0.1.4=pypi_0
- termcolor =2.3.0=pypi_0
- texttable =1.7.0=pypi_0
- threadpoolctl =3.2.0=pypi_0
- tk =8.6.12=h1ccaba5_0
- tokenizers =0.15.0=pypi_0
- toml =0.10.2=pypi_0
- tomli =2.0.1=pypi_0
- torch =2.1.1=pypi_0
- torchvision =0.16.1=pypi_0
- tornado =6.4=pypi_0
- tox =4.11.4=pypi_0
- tqdm =4.66.1=pypi_0
- tqdm-multiprocess =0.0.11=pypi_0
- traitlets =5.14.0=pypi_0
- transformers =4.35.2=pypi_0
- triton =2.1.0=pypi_0
- typepy =1.3.2=pypi_0
- typing-extensions =4.8.0=pypi_0
- tzdata =2023.3=pypi_0
- urllib3 =2.1.0=pypi_0
- virtualenv =20.24.7=pypi_0
- wcwidth =0.2.12=pypi_0
- wheel =0.41.2=py310h06a4308_0
- xxhash =3.4.1=pypi_0
- xz =5.4.2=h5eee18b_0
- yarl =1.9.3=pypi_0
- zlib =1.2.13=h5eee18b_0
- zstandard =0.22.0=pypi_0
- accelerate ==0.25.0
- attributedict *
- ipdb *
- packaging ==24.0
- protobuf *
- sentencepiece *
- tokenizers >=0.15
- toml *
- torch ==2.1.2
- transformers ==4.36.2