https://github.com/niieani/gpt-tokenizer
bpe decoder encoder gpt-2 gpt-3 gpt-4 gpt-4o gpt-5 gpt-o1 machine-learning openai tokenizer
Score: 26.263496765139404
Last synced: about 21 hours ago
JSON representation
Repository metadata:
The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.
- Host: GitHub
- URL: https://github.com/niieani/gpt-tokenizer
- Owner: niieani
- License: mit
- Created: 2023-03-22T04:32:09.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2026-01-02T06:24:50.000Z (about 1 month ago)
- Last Synced: 2026-01-26T11:31:01.060Z (9 days ago)
- Topics: bpe, decoder, encoder, gpt-2, gpt-3, gpt-4, gpt-4o, gpt-5, gpt-o1, machine-learning, openai, tokenizer
- Language: TypeScript
- Homepage: https://gpt-tokenizer.dev
- Size: 13 MB
- Stars: 727
- Watchers: 3
- Forks: 52
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Agents: AGENTS.md
-
Funding:
- Github: niieani
Owner metadata:
- Name: Bazyli Brzóska
- Login: niieani
- Email:
- Kind: user
- Description: An inventor passionate about HCI workflows that help humanity innovate responsibly, for happiness and global thriving. Official @zendesk account: @bbrzoska
- Website: https://invent.life
- Location: San Francisco, CA
- Twitter: niieani
- Company: @zendesk
- Icon url: https://avatars.githubusercontent.com/u/563469?v=4
- Repositories: 206
- Last Synced at: 2026-01-25T21:32:44.093Z
- Profile URL: https://github.com/niieani
GitHub Events
Total
- Create event: 13
- Delete event: 5
- Fork event: 9
- Issue comment event: 42
- Issues event: 22
- Pull request event: 17
- Pull request review comment event: 1
- Pull request review event: 1
- Push event: 33
- Release event: 10
- Watch event: 180
- Total: 333
Last Year
- Create event: 4
- Fork event: 5
- Issue comment event: 15
- Issues event: 6
- Pull request event: 5
- Pull request review comment event: 1
- Pull request review event: 1
- Push event: 17
- Release event: 3
- Watch event: 120
- Total: 177
Committers metadata
Last synced: 7 months ago
Total Commits: 111
Total Committers: 12
Avg Commits per committer: 9.25
Development Distribution Score (DDS): 0.423
Commits in past year: 47
Committers in past year: 3
Avg Commits per committer in past year: 15.667
Development Distribution Score (DDS) in past year: 0.106
| Name | Commits | |
|---|---|---|
| Bazyli Brzoska | b****a@g****m | 64 |
| Nick Walton | n****0@g****m | 18 |
| Anar Kafkas | a****s@g****m | 8 |
| mbukeRepo | m****4@g****m | 7 |
| renovate[bot] | 2****] | 4 |
| Squibs | Z****n@g****m | 3 |
| Sean Gillespie | i****o@s****s | 2 |
| Michael Sharpe | m****e@p****g | 1 |
| Justin Steven | j****n@j****m | 1 |
| David Schnurr | s****r@g****m | 1 |
| Caleb Bartholomew | c****b@a****o | 1 |
| Andrew Healey | d****y@g****m | 1 |
Issue and Pull Request metadata
Last synced: about 1 month ago
Total issues: 46
Total pull requests: 49
Average time to close issues: 4 months
Average time to close pull requests: 3 months
Total issue authors: 43
Total pull request authors: 8
Average comments per issue: 2.0
Average comments per pull request: 0.98
Merged pull request: 24
Bot issues: 2
Bot pull requests: 29
Past year issues: 5
Past year pull requests: 4
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 1 day
Past year issue authors: 5
Past year pull request authors: 3
Past year average comments per issue: 0.4
Past year average comments per pull request: 0.75
Past year merged pull request: 3
Past year bot issues: 1
Past year bot pull requests: 0
Top Issue Authors
- nilsreichardt (2)
- BennyAlex (2)
- thdoan (2)
- Qarj (1)
- aminsol (1)
- Ranork (1)
- renovate[bot] (1)
- emfastic (1)
- rafalzawadzki (1)
- Ferossgp (1)
- Zwe1 (1)
- enijar (1)
- gbrlmtrz (1)
- congqi09 (1)
- alejandro5042 (1)
Top Pull Request Authors
- renovate[bot] (29)
- niieani (10)
- mbukeRepo (4)
- shism2 (2)
- Copilot (1)
- onlyoneaman (1)
- wesbos (1)
- justinsteven (1)
Top Issue Labels
- released (7)
- released on @beta (2)
- help wanted (2)
- invalid (2)
- semantic-release (1)
- documentation (1)
Top Pull Request Labels
- released (12)
Package metadata
- Total packages: 1
-
Total downloads:
- npm: 1,103,649 last-month
- Total docker downloads: 27,936,047
- Total dependent packages: 36
- Total dependent repositories: 22
- Total versions: 34
- Total maintainers: 1
npmjs.org: gpt-tokenizer
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
- Homepage: https://github.com/niieani/gpt-tokenizer#readme
- Licenses: MIT
- Latest release: 3.4.0 (published 3 months ago)
- Last Synced: 2026-01-20T23:02:38.966Z (14 days ago)
- Versions: 34
- Dependent Packages: 36
- Dependent Repositories: 22
- Downloads: 1,103,649 Last month
- Docker Downloads: 27,936,047
-
Rankings:
- Downloads: 0.518%
- Dependent packages count: 0.76%
- Docker downloads count: 1.874%
- Average: 2.41%
- Dependent repos count: 2.626%
- Stargazers count: 3.619%
- Forks count: 5.061%
- Maintainers (1)
Dependencies
- actions/checkout v3 composite
- actions/setup-node v3 composite
- actions/checkout v3 composite
- github/codeql-action/analyze v2 composite
- github/codeql-action/autobuild v2 composite
- github/codeql-action/init v2 composite
- @niieani/scaffold ^1.7.1 development
- tsx ^3.12.7 development
- rfc4648 ^1.5.2
- rfc4648 ^1.5.2
- 1385 dependencies