Data

Tools

Indexes

Applications

Experiments

Summary

An open API service for producing an overview of a list of open source projects.

https://github.com/huggingface/evaluate

evaluation machine-learning

Score: 30.03043896677645

Last synced: about 15 hours ago
JSON representation

Repository metadata:

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Owner metadata:

GitHub Events

Total
Last Year

Committers metadata

Last synced: 5 days ago

Total Commits: 963
Total Committers: 140
Avg Commits per committer: 6.879
Development Distribution Score (DDS): 0.769

Commits in past year: 20
Committers in past year: 11
Avg Commits per committer in past year: 1.818
Development Distribution Score (DDS) in past year: 0.7

Name Email Commits
Quentin Lhoest 4****q 222
Albert Villanova del Moral 8****a 116
Sasha Luccioni l****s@m****c 103
Leandro von Werra l****a 99
Mario Šaško m****7@g****m 50
Thomas Wolf t****f 29
leandro l****a@s****o 24
sashavor a****a@g****m 24
sashavor s****i@h****o 23
helen 3****n 16
Bram Vanroy B****y@U****e 13
lewtun l****l@g****m 13
Patrick von Platen p****n@g****m 12
mathemakitten h****n@h****o 9
fxmarty 9****y 9
meg 9****e 7
Steven Liu 5****u 6
Simon Brandeis 3****s 6
emibaylor 2****r 6
Sylvain Lesage s****o@r****t 6
Julien Plu p****n@g****m 4
Philipp Schmid 3****d 4
Sylvain Gugger 3****r 4
Steven s****u@g****m 4
Yacine Jernite y****e 4
douwekiela d****a 4
Mishig d****g@g****m 4
Yacklin Wong 1****n 3
Nima Boscarino n****o@g****m 3
Ricardo Rei r****i@u****m 3
and 110 more...

Issue and Pull Request metadata

Last synced: 5 days ago

Total issues: 221
Total pull requests: 264
Average time to close issues: 2 months
Average time to close pull requests: about 2 months
Total issue authors: 187
Total pull request authors: 142
Average comments per issue: 1.9
Average comments per pull request: 0.78
Merged pull request: 72
Bot issues: 0
Bot pull requests: 11

Past year issues: 19
Past year pull requests: 78
Past year average time to close issues: 25 days
Past year average time to close pull requests: 11 days
Past year issue authors: 17
Past year pull request authors: 47
Past year average comments per issue: 0.74
Past year average comments per pull request: 0.4
Past year merged pull request: 9
Past year bot issues: 0
Past year bot pull requests: 10

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/huggingface/evaluate

Top Issue Authors

  • lvwerra (7)
  • albertvillanova (7)
  • daskol (4)
  • shivanraptor (4)
  • CSOAI-ORG (3)
  • FlorinAndrei (3)
  • NightMachinery (2)
  • lewtun (2)
  • trajepl (2)
  • BramVanroy (2)
  • TarikKaanKoc (2)
  • jpodivin (2)
  • Natooz (2)
  • NielsRogge (2)
  • mathemakitten (2)

Top Pull Request Authors

  • lhoestq (24)
  • albertvillanova (11)
  • tonycoder-hub (10)
  • hf-security-analysis[bot] (9)
  • MedAhmedKrichen (5)
  • qubvel (4)
  • shenxiangzhuang (4)
  • jpodivin (4)
  • skyil7 (4)
  • krishnap25 (4)
  • xodn348 (4)
  • ShamikOfficial (3)
  • Wauplin (3)
  • Yacklin (3)
  • RavSinghChandan (3)

Top Issue Labels

  • metric request (9)
  • enhancement (2)

Top Pull Request Labels

  • github_actions (1)
  • dependencies (1)

Package metadata

pypi.org: evaluate

HuggingFace community-driven open-source library of evaluation

  • Homepage: https://github.com/huggingface/evaluate
  • Documentation: https://evaluate.readthedocs.io/
  • Licenses: Apache 2.0
  • Latest release: 0.4.6 (published about 1 year ago)
  • Last Synced: 2026-10-25T20:56:37.407Z (5 days ago)
  • Versions: 19
  • Dependent Packages: 222
  • Dependent Repositories: 2,474
  • Downloads: 2,903,015 Last month
  • Docker Downloads: 24,936,750
  • Rankings:
    • Dependent packages count: 0.208%
    • Dependent repos count: 0.219%
    • Downloads: 0.324%
    • Docker downloads count: 0.983%
    • Average: 1.204%
    • Stargazers count: 1.726%
    • Forks count: 3.764%
  • Maintainers (3)
proxy.golang.org: github.com/huggingface/evaluate

  • Homepage:
  • Documentation: https://pkg.go.dev/github.com/huggingface/evaluate#section-documentation
  • Licenses: apache-2.0
  • Latest release: v0.4.6 (published about 1 year ago)
  • Last Synced: 2026-10-25T20:56:20.834Z (5 days ago)
  • Versions: 14
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Stargazers count: 1.661%
    • Forks count: 1.981%
    • Average: 6.005%
    • Dependent packages count: 9.576%
    • Dependent repos count: 10.802%
conda-forge.org: evaluate

  • Homepage: https://github.com/huggingface/evaluate
  • Licenses: Apache-2.0
  • Latest release: 0.2.2 (published about 4 years ago)
  • Last Synced: 2026-03-21T01:09:11.372Z (7 months ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 3
  • Downloads: 77,615 Total
  • Rankings:
    • Stargazers count: 11.523%
    • Forks count: 16.443%
    • Dependent repos count: 18.035%
    • Average: 24.396%
    • Dependent packages count: 51.584%
spack.io: py-evaluate

Evaluate is a library that makes evaluating and comparing models and reporting their performance easier and more standardized.

  • Homepage: https://github.com/huggingface/evaluate
  • Licenses: []
  • Latest release: 0.4.6 (published about 1 year ago)
  • Last Synced: 2026-10-25T20:56:18.743Z (5 days ago)
  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Average: 26.75%
    • Dependent packages count: 53.499%
anaconda.org: evaluate

Evaluate is a library that makes evaluating and comparing models and reporting their performance easier and more standardized. It currently contains: - implementations of dozens of popular metrics: the existing metrics cover a variety of tasks spanning from NLP to Computer Vision, and include dataset-specific metrics for datasets. With a simple command like `accuracy = load("accuracy")`, get any of these metrics ready to use for evaluating a ML model in any framework (Numpy/Pandas/PyTorch/TensorFlow/JAX). - comparisons and measurements: comparisons are used to measure the difference between models and measurements are tools to evaluate datasets. - an easy way of adding new evaluation modules to the 🤗 Hub: you can create new evaluation modules and push them to a dedicated Space in the 🤗 Hub with evaluate-cli create [metric name], which allows you to see easily compare different metrics and their outputs for the same sets of references and predictions.

  • Homepage: https://github.com/huggingface/evaluate
  • Licenses: Apache-2.0
  • Latest release: 0.4.6 (published 12 months ago)
  • Last Synced: 2026-09-30T19:26:10.531Z (30 days ago)
  • Versions: 5
  • Dependent Packages: 0
  • Dependent Repositories: 3
  • Downloads: 4,222 Total
  • Rankings:
    • Stargazers count: 21.355%
    • Forks count: 29.101%
    • Average: 36.991%
    • Dependent repos count: 46.446%
    • Dependent packages count: 51.063%

Dependencies

.github/workflows/ci.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/python-release.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
.github/workflows/update_spaces.yml actions
  • actions/checkout v2 composite
  • actions/setup-python v2 composite
.github/hub/requirements.txt pypi
  • huggingface_hub *
additional-tests-requirements.txt pypi
  • gin-config * test
  • unbabel-comet >=1.0.0 test
comparisons/exact_match/requirements.txt pypi
  • scipy *
comparisons/mcnemar/requirements.txt pypi
  • scipy *
comparisons/wilcoxon/requirements.txt pypi
  • datasets *
  • scipy *
measurements/honest/requirements.txt pypi
  • torch *
  • transformers *
  • unidecode ==1.3.4
measurements/label_distribution/requirements.txt pypi
  • scipy *
measurements/perplexity/requirements.txt pypi
  • torch *
  • transformers *
measurements/regard/requirements.txt pypi
  • torch *
  • transformers *
measurements/toxicity/requirements.txt pypi
  • torch *
  • transformers *
measurements/word_count/requirements.txt pypi
  • scikit-learn *
measurements/word_length/requirements.txt pypi
  • nltk *
metrics/accuracy/requirements.txt pypi
  • scikit-learn *
metrics/bertscore/requirements.txt pypi
  • bert_score *
metrics/brier_score/requirements.txt pypi
  • scikit-learn *
metrics/cer/requirements.txt pypi
  • jiwer *
metrics/character/requirements.txt pypi
  • cer >=1.2.0
metrics/charcut_mt/requirements.txt pypi
  • charcut >=1.1.1
metrics/chrf/requirements.txt pypi
  • sacrebleu *
metrics/comet/requirements.txt pypi
  • torch *
  • unbabel-comet *
metrics/f1/requirements.txt pypi
  • scikit-learn *
metrics/frugalscore/requirements.txt pypi
  • torch *
  • transformers *
metrics/glue/requirements.txt pypi
  • scikit-learn *
  • scipy *
metrics/google_bleu/requirements.txt pypi
  • nltk *
metrics/indic_glue/requirements.txt pypi
  • scikit-learn *
  • scipy *
metrics/mae/requirements.txt pypi
  • scikit-learn *
metrics/mape/requirements.txt pypi
  • scikit-learn *
metrics/mase/requirements.txt pypi
  • scikit-learn *
metrics/matthews_correlation/requirements.txt pypi
  • scikit-learn *
metrics/mauve/requirements.txt pypi
  • faiss-cpu *
  • mauve-text *
  • scikit-learn *
metrics/meteor/requirements.txt pypi
  • nltk *
metrics/mse/requirements.txt pypi
  • scikit-learn *
metrics/nist_mt/requirements.txt pypi
  • nltk *
metrics/pearsonr/requirements.txt pypi
  • scipy *
metrics/perplexity/requirements.txt pypi
  • torch *
  • transformers *
metrics/poseval/requirements.txt pypi
  • scikit-learn *
metrics/precision/requirements.txt pypi
  • scikit-learn *
metrics/recall/requirements.txt pypi
  • scikit-learn *
metrics/rl_reliability/requirements.txt pypi
  • gin-config *
  • scipy *
  • tensorflow *
metrics/roc_auc/requirements.txt pypi
  • scikit-learn *
metrics/rouge/requirements.txt pypi
  • absl-py *
  • nltk *
  • rouge_score >=0.1.2
metrics/sacrebleu/requirements.txt pypi
  • sacrebleu *
metrics/sari/requirements.txt pypi
  • sacrebleu *
  • sacremoses *
metrics/seqeval/requirements.txt pypi
  • seqeval *
metrics/smape/requirements.txt pypi
  • scikit-learn *
metrics/spearmanr/requirements.txt pypi
  • scipy *
metrics/super_glue/requirements.txt pypi
  • scikit-learn *
metrics/ter/requirements.txt pypi
  • sacrebleu *
metrics/trec_eval/requirements.txt pypi
  • trectools *
metrics/wer/requirements.txt pypi
  • jiwer *
metrics/wiki_split/requirements.txt pypi
  • sacrebleu *
  • sacremoses *
metrics/xtreme_s/requirements.txt pypi
  • scikit-learn *
.github/workflows/build_documentation.yml actions
.github/workflows/build_pr_documentation.yml actions
.github/workflows/delete_doc_comment.yml actions
measurements/text_duplicates/requirements.txt pypi
metrics/bleu/requirements.txt pypi
metrics/bleurt/requirements.txt pypi
metrics/code_eval/requirements.txt pypi
metrics/competition_math/requirements.txt pypi
metrics/coval/requirements.txt pypi
metrics/cuad/requirements.txt pypi
metrics/exact_match/requirements.txt pypi
metrics/mahalanobis/requirements.txt pypi
metrics/mean_iou/requirements.txt pypi
metrics/r_squared/requirements.txt pypi
metrics/squad/requirements.txt pypi
metrics/squad_v2/requirements.txt pypi
metrics/xnli/requirements.txt pypi
setup.py pypi
templates/{{ cookiecutter.module_slug }}/requirements.txt pypi