An open API service for producing an overview of a list of open source projects.

https://github.com/feyninc/tokie

Score: 5.209486152841421

Last synced: about 22 hours ago
JSON representation

Repository metadata:

🍡 30x faster tokenization for every HuggingFace model


Owner metadata:


GitHub Events

Total
Last Year

Committers metadata

Last synced: 2 days ago

Total Commits: 215
Total Committers: 3
Avg Commits per committer: 71.667
Development Distribution Score (DDS): 0.223

Commits in past year: 215
Committers in past year: 3
Avg Commits per committer in past year: 71.667
Development Distribution Score (DDS) in past year: 0.223

Name Email Commits
Bhavnick @ Chonkie b****k@c****i 167
bhavnicksm b****m@g****m 33
chonk-lain l****n@c****i 15

Issue and Pull Request metadata

Last synced: 2 days ago

Total issues: 4
Total pull requests: 20
Average time to close issues: 2 days
Average time to close pull requests: 6 days
Total issue authors: 4
Total pull request authors: 2
Average comments per issue: 0.5
Average comments per pull request: 0.95
Merged pull request: 17
Bot issues: 0
Bot pull requests: 0

Past year issues: 4
Past year pull requests: 20
Past year average time to close issues: 2 days
Past year average time to close pull requests: 6 days
Past year issue authors: 4
Past year pull request authors: 2
Past year average comments per issue: 0.5
Past year average comments per pull request: 0.95
Past year merged pull request: 17
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/feyninc/tokie

Top Issue Authors

  • mariel-labs (1)
  • stephantul (1)
  • H-Plus-Time (1)
  • chonknick (1)

Top Pull Request Authors

  • chonknick (13)
  • chonk-lain (7)

Top Issue Labels

Top Pull Request Labels


Dependencies

Cargo.toml cargo
crates/tokie-python/pyproject.toml pypi
.github/workflows/release.yml actions
  • PyO3/maturin-action v1 composite
  • Swatinem/rust-cache v2 composite
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/upload-artifact v4 composite
  • dtolnay/rust-toolchain stable composite
  • pypa/gh-action-pypi-publish release/v1 composite
crates/tokie/Cargo.toml cargo
  • criterion 0.5 development
  • tokenizers 0.22 development
  • chunk 0.9
  • crc32fast 1.4
  • daggrs 0.1.1
  • foldhash 0.1
  • hf-hub 0.4.3
  • icu_normalizer 1.5
  • memchr 2.7
  • regex-automata 0.4
  • serde 1.0
  • serde_json 1.0
  • smallvec 1.13
  • unicode-general-category 1.0
Cargo.lock cargo
  • 235 dependencies
crates/tokie-python/Cargo.toml cargo
.github/workflows/pyodide.yml actions
  • PyO3/maturin-action v1 composite
  • actions/checkout v4 composite
  • actions/download-artifact v4 composite
  • actions/setup-python v5 composite
  • actions/upload-artifact v4 composite
  • emscripten-core/setup-emsdk v16 composite
.github/workflows/speed-regression.yml actions
  • Swatinem/rust-cache v2 composite
  • actions/cache v4 composite
  • actions/checkout v4 composite
  • actions/upload-artifact v4 composite
  • dtolnay/rust-toolchain stable composite
.github/workflows/tokenizer-accuracy.yml actions
  • Swatinem/rust-cache v2 composite
  • actions/cache v4 composite
  • actions/checkout v4 composite
  • dtolnay/rust-toolchain stable composite
crates/tokie-cli/Cargo.toml cargo
crates/pretokie/Cargo.toml cargo
  • foldhash 0.1 development
  • regex-automata 0.4
  • unicode-general-category 1.0