llama2-tokenizer.js

chatgpt llama llama2 tokenizer

Score: 9.619266151713328

Last synced: about 10 hours ago
JSON representation

Repository metadata:

llama2 tokenizer for javascript

Host: GitHub
URL: https://github.com/lenML/llama2-tokenizer.js
Owner: lenML
License: mit
Created: 2024-01-07T12:03:03.000Z (about 2 years ago)
Default Branch: main
Last Pushed: 2024-04-20T10:59:45.000Z (almost 2 years ago)
Last Synced: 2026-01-12T15:48:47.397Z (26 days ago)
Topics: chatgpt, llama, llama2, tokenizer
Language: TypeScript
Homepage:
Size: 545 KB
Stars: 2
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Owner metadata:

Name: lenML
Login: lenML
Email:
Kind: organization
Description:
Website:
Location:
Twitter:
Company:
Icon url: https://avatars.githubusercontent.com/u/154655365?v=4
Repositories: 1
Last Synced at: 2023-12-31T10:24:24.070Z
Profile URL: https://github.com/lenML

GitHub Events

Total

Total: 0

Last Year

Total: 0

Committers metadata

Last synced: 4 days ago

Total Commits: 18
Total Committers: 1
Avg Commits per committer: 18.0
Development Distribution Score (DDS): 0.0

Commits in past year: 0
Committers in past year: 0
Avg Commits per committer in past year: 0.0
Development Distribution Score (DDS) in past year: 0.0

Name	Email	Commits
zhzluke96	z**1@g**m	18

Issue and Pull Request metadata

Last synced: 5 months ago

Total issues: 4
Total pull requests: 0
Average time to close issues: 5 days
Average time to close pull requests: N/A
Total issue authors: 4
Total pull request authors: 0
Average comments per issue: 2.25
Average comments per pull request: 0
Merged pull request: 0
Bot issues: 0
Bot pull requests: 0

Past year issues: 0
Past year pull requests: 0
Past year average time to close issues: N/A
Past year average time to close pull requests: N/A
Past year issue authors: 0
Past year pull request authors: 0
Past year average comments per issue: 0
Past year average comments per pull request: 0
Past year merged pull request: 0
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/lenML/llama2-tokenizer.js

Top Issue Authors

arvinxx (1)
Mgonand (1)
DreamGenX (1)
murtaza-malik (1)

Top Pull Request Authors

Top Issue Labels

enhancement (1)
bug (1)
help wanted (1)

Top Pull Request Labels

Package metadata

Total packages: 7
Total downloads:
- npm: 7,525 last-month
Total dependent packages: 0 (may contain duplicates)
Total dependent repositories: 0 (may contain duplicates)
Total versions: 18
Total maintainers: 1

npmjs.org: @lenml/llama2-tokenizer

Our library `@lenml/llama2-tokenizer` has been deprecated. We are excited to introduce our new library `@lenml/tokenizers` as its replacement, offering a broader set of features and an enhanced experience.

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.4 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:03.899Z (2 days ago)
Versions: 6
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 3,492 Last month
Rankings:
- Downloads: 1.846%
- Average: 21.814%
- Dependent repos count: 25.93%
- Dependent packages count: 37.667%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-baichuan2

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:06.196Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 16 Last month
Rankings:
- Dependent repos count: 33.313%
- Average: 40.32%
- Dependent packages count: 47.327%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-chatglm3

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:08.559Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 16 Last month
Rankings:
- Dependent repos count: 33.313%
- Average: 40.32%
- Dependent packages count: 47.327%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-yi

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:09.407Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 8 Last month
Rankings:
- Dependent repos count: 33.313%
- Average: 40.32%
- Dependent packages count: 47.327%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-internlm2

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:10.476Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 5 Last month
Rankings:
- Dependent repos count: 33.313%
- Average: 40.32%
- Dependent packages count: 47.327%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-llama2

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:12.948Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 3,979 Last month
Rankings:
- Dependent repos count: 33.313%
- Average: 40.32%
- Dependent packages count: 47.327%
Maintainers (1)
- luke_zhang

npmjs.org: @lenml/llama2-tokenizer-vocab-gemma

Homepage: https://github.com/lenML/llama2-tokenizer.js#readme
Licenses: MIT
Latest release: 1.1.1 (published almost 2 years ago)
Last Synced: 2026-02-04T22:30:02.785Z (2 days ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 9 Last month
Rankings:
- Dependent repos count: 33.643%
- Average: 40.875%
- Dependent packages count: 48.106%
Maintainers (1)
- luke_zhang

Dependencies

package.json npm

@types/jest ^29.5.11 development
@types/node ^20.10.6 development
jest ^29.7.0 development
ts-jest ^29.1.1 development
ts-node ^10.9.2 development
tslib ^2.6.2 development
tsup ^8.0.1 development
typescript ^5.3.3 development

pnpm-lock.yaml npm

400 dependencies

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary