https://github.com/confident-ai/deepeval

evaluation-framework evaluation-metrics llm-evaluation llm-evaluation-framework llm-evaluation-metrics python

Score: 31.20675210074277

Last synced: about 10 hours ago
JSON representation

Repository metadata:

The LLM Evaluation Framework

Host: GitHub
URL: https://github.com/confident-ai/deepeval
Owner: confident-ai
License: apache-2.0
Created: 2023-08-10T05:35:04.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2026-08-21T19:24:44.000Z (4 days ago)
Last Synced: 2026-08-24T13:45:34.771Z (1 day ago)
Topics: evaluation-framework, evaluation-metrics, llm-evaluation, llm-evaluation-framework, llm-evaluation-metrics, python
Language: Python
Homepage: https://deepeval.com
Size: 195 MB
Stars: 17,817
Watchers: 67
Forks: 1,842
Open Issues: 471
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md
- Citation: CITATION.cff
- Maintainers: MAINTAINERS.md

Owner metadata:

Name: Confident AI
Login: confident-ai
Email:
Kind: organization
Description:
Website: www.confident-ai.com
Location:
Twitter:
Company:
Icon url: https://avatars.githubusercontent.com/u/130858411?v=4
Repositories: 9
Last Synced at: 2026-08-17T16:34:48.574Z
Profile URL: https://github.com/confident-ai

GitHub Events

Total

Commit comment event: 3
Create event: 244
Delete event: 173
Discussion event: 4
Fork event: 677
Issue comment event: 1623
Issues event: 398
Member event: 3
Pull request event: 1257
Pull request review comment event: 77
Pull request review event: 53
Push event: 1859
Release event: 15
Watch event: 6983
Total: 13369

Last Year

Create event: 59
Delete event: 41
Discussion event: 4
Fork event: 162
Issue comment event: 399
Issues event: 121
Member event: 1
Pull request event: 284
Pull request review comment event: 40
Pull request review event: 26
Push event: 524
Watch event: 1617
Total: 3278

Committers metadata

Last synced: 7 days ago

Total Commits: 7,881
Total Committers: 329
Avg Commits per committer: 23.954
Development Distribution Score (DDS): 0.623

Commits in past year: 2,931
Committers in past year: 128
Avg Commits per committer in past year: 22.898
Development Distribution Score (DDS) in past year: 0.667

Name	Email	Commits
Jeffrey Ip	j**p@c**m	2968
A-Vamshi	v**y@i**m	1183
Jacky Wong	c**g@g**m	858
Kritin_Vongthongsri	k**v@p**u	778
Mayank Solanki	b**u@g**m	731
BloggerBust	t**n@b**a	388
tanayvaswani	v**9@g**m	90
trevor-cai	t**r@c**m	65
Anindyadeep	p**p@g**m	47
Anindyadeep	a**a@p**n	35
Vasilije	8****0	26
Pratyush-exe	p**5@g**m	23
Sidhaarth Sredharan	s**7@g**m	19
john	j**n@s**m	18
Aman Gokrani	a**i@g**m	17
Sai-Suraj-27	s**9@g**m	15
fetz236	5****6	14
Peilun Li	p**l@z**m	13
Jack Luar	3****s	11
Vamshi Adimalla	v**a@V**l	9
Frederico Schuh	f**h@g**m	9
Vytenis Šliogeris	v**s@n**m	9
Jan F.	5****4	7
Jon Bennion	1****b	7
Serghei Iakovlev	g**t@s**l	7
Andrea Romano	1****o	7
Alex Maggioni	9****i	6
Christian Bernhard	4****d	6
karan kulshrestha	r**7@g**m	6
Aditya Sahni	s**7@g**m	5
and 299 more...

Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 472
Total pull requests: 1,977
Average time to close issues: 22 days
Average time to close pull requests: 4 days
Total issue authors: 347
Total pull request authors: 338
Average comments per issue: 1.16
Average comments per pull request: 1.34
Merged pull request: 1,423
Bot issues: 0
Bot pull requests: 1

Past year issues: 72
Past year pull requests: 329
Past year average time to close issues: 7 days
Past year average time to close pull requests: 9 days
Past year issue authors: 61
Past year pull request authors: 124
Past year average comments per issue: 1.35
Past year average comments per pull request: 1.56
Past year merged pull request: 93
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/confident-ai/deepeval

Top Issue Authors

ColabDog (24)
penguine-ip (19)
prescod (10)
sahniaditya007 (6)
AndresPrez (5)
piseabhijeet (5)
tjasmin111 (4)
luarss (4)
Sara-Hossny (4)
CAW-nz (4)
ymzayek (3)
FrancoisMasson1990 (3)
enrico-stauss (3)
spike-spiegel-21 (3)
kbarendrecht (3)

Top Pull Request Authors

penguine-ip (551)
kritinv (430)
spike-spiegel-21 (103)
A-Vamshi (95)
ColabDog (62)
BloggerBust (29)
john-lemmon-lime (17)
luarss (15)
Sai-Suraj-27 (12)
ramipellumbi (12)
ChristianBernhard (12)
kira-offgrid (10)
DivyamTalwar (10)
feiiiiii5 (8)
sergeyklay (8)

Top Issue Labels

help wanted (17)
enhancement (11)
good first issue (5)
documentation (3)
bug (2)
kind/enhancement (1)
awaiting clarification (1)
area/docs (1)

Top Pull Request Labels

awaiting docs fix (4)
awaiting code fix (2)
awaiting fix (1)
dependencies (1)
new talk (1)
java (1)
typescript (1)
awaiting clarification (1)

Package metadata

Total packages: 5
Total downloads:
- npm: 41,899 last-month
- conda: 357 total
- pypi: 5,894,611 last-month
Total dependent packages: 7 (may contain duplicates)
Total dependent repositories: 1 (may contain duplicates)
Total versions: 860
Total maintainers: 3

proxy.golang.org: github.com/confident-ai/deepeval

Homepage:
Documentation: https://pkg.go.dev/github.com/confident-ai/deepeval#section-documentation
Licenses: apache-2.0
Latest release: v4.1.8+incompatible (published 13 days ago)
Last Synced: 2026-08-24T14:09:13.315Z (1 day ago)
Versions: 302
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 6.528%
- Average: 6.747%
- Dependent repos count: 6.966%

pypi.org: deepeval

The LLM Evaluation Framework

Homepage:
Documentation: https://deepeval.com
Licenses: Apache-2.0
Latest release: 4.1.10 (published 4 days ago)
Last Synced: 2026-08-24T13:46:25.706Z (1 day ago)
Versions: 518
Dependent Packages: 7
Dependent Repositories: 1
Downloads: 5,894,611 Last month
Rankings:
- Stargazers count: 2.812%
- Downloads: 3.967%
- Forks count: 8.248%
- Average: 9.348%
- Dependent packages count: 10.095%
- Dependent repos count: 21.615%
Maintainers (1)
- penguineip

npmjs.org: deepeval-ts

Deprecated — renamed to 'deepeval'

Homepage: https://deepeval.com
Status: deprecated
Licenses: Apache-2.0
Latest release: 1.0.0 (published 9 days ago)
Last Synced: 2026-08-24T14:02:49.938Z (1 day ago)
Versions: 21
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 7,328 Last month
Rankings:
- Downloads: 6.97%
- Average: 22.321%
- Dependent repos count: 24.554%
- Dependent packages count: 35.44%
Maintainers (2)
- penguine-ip
- kritinv

npmjs.org: deepeval

The LLM Evaluation Framework for TypeScript

Homepage: https://github.com/confident-ai/deepeval#readme
Licenses: apache-2.0
Latest release: 0.9.12 (published 4 days ago)
Last Synced: 2026-08-24T15:28:07.285Z (1 day ago)
Versions: 16
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 34,571 Last month
Rankings:
- Dependent repos count: 22.7%
- Average: 27.71%
- Dependent packages count: 32.72%
Maintainers (2)
- penguine-ip
- kritinv

anaconda.org: deepeval

DeepEval is a simple-to-use, open-source LLM evaluation framework, for evaluating large-language model systems. It is similar to Pytest but specialized for unit testing LLM apps. DeepEval incorporates the latest research to run evals via metrics such as G-Eval, task completion, answer relevancy, hallucination, etc., which uses LLM-as-a-judge and other NLP models that run locally on your machine.

Homepage: https://github.com/confident-ai/deepeval
Licenses: Apache-2.0
Latest release: 4.1.4 (published 26 days ago)
Last Synced: 2026-07-31T13:03:38.958Z (25 days ago)
Versions: 3
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 357 Total
Rankings:
- Dependent packages count: 35.874%
- Dependent repos count: 39.116%
- Average: 48.659%
- Downloads: 70.985%

Dependencies

.github/workflows/black.yml actions

actions/checkout v3 composite
psf/black stable composite

docs/package.json npm

@docusaurus/module-type-aliases 2.4.1 development
@docusaurus/core 2.4.1
@docusaurus/preset-classic 2.4.1
@mdx-js/react ^1.6.22
clsx ^1.2.1
prism-react-renderer ^1.3.5
react ^17.0.2
react-dom ^17.0.2

.github/workflows/typescript_lint.yml actions

actions/checkout v3 composite

docs/content/docs/(agentic)/meta.json cpan

.github/workflows/test_metric_templates.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/test_metrics.yml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-python v4 composite
snok/install-poetry v1 composite

.github/workflows/test_model_registry.yml actions

actions/checkout v3 composite
actions/setup-python v4 composite

.github/workflows/typescript_confident.yml actions

actions/checkout v3 composite

.github/workflows/changelog.yml actions

actions/checkout v4 composite
actions/setup-python v5 composite
peter-evans/create-pull-request v6 composite

.github/workflows/pr-title-check.yml actions

amannn/action-semantic-pull-request v5 composite

.github/workflows/full_test_core_for_pr.yml actions

actions/cache v3 composite
actions/checkout v4 composite
actions/setup-python v4 composite
snok/install-poetry v1 composite

.github/workflows/test_core.yml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-python v4 composite
snok/install-poetry v1 composite

.github/workflows/typescript_test.yml actions

actions/checkout v3 composite

.github/workflows/typescript_test_run_e2e.yml actions

actions/checkout v3 composite

.github/workflows/release.yml actions

actions/checkout v4 composite
actions/setup-python v5 composite

docs/content/docs/(community)/meta.json cpan

.github/workflows/typescript_integration.yml actions

actions/checkout v3 composite

docs/content/docs/(use-cases)/meta.json cpan

docs/content/docs/golden-synthesizer/meta.json cpan

.github/workflows/test_integrations.yml actions

actions/checkout v4 composite
actions/setup-python v5 composite
snok/install-poetry v1 composite

docs/content/docs/(custom)/meta.json cpan

docs/content/integrations/models/meta.json cpan

docs/content/integrations/others/meta.json cpan

docs/content/tutorials/meta.json cpan

.github/workflows/test_confident.yml actions

actions/cache v3 composite
actions/checkout v3 composite
actions/setup-python v4 composite
snok/install-poetry v1 composite

docs/content/docs/conversation-simulator/meta.json cpan

docs/content/integrations/meta.json cpan

docs/content/docs/evaluation-end-to-end-llm-evals/meta.json cpan

docs/content/changelog/meta.json cpan

docs/content/docs/(concepts)/(test-cases)/meta.json cpan

docs/content/docs/(safety)/meta.json cpan

docs/content/guides/meta.json cpan

docs/content/docs/(mcp)/meta.json cpan

docs/content/docs/(non-llm)/meta.json cpan

docs/content/blog/meta.json cpan

docs/content/docs/(algorithms)/meta.json cpan

docs/content/docs/(generate-goldens)/meta.json cpan

docs/content/docs/(metrics-others)/meta.json cpan

docs/content/docs/(benchmarks)/meta.json cpan

docs/content/docs/(concepts)/meta.json cpan

docs/content/docs/(images)/meta.json cpan

docs/content/docs/(multi-turn)/meta.json cpan

docs/content/integrations/vector-databases/meta.json cpan

docs/content/docs/(rag)/meta.json cpan

docs/content/integrations/frameworks/meta.json cpan

docs/content/docs/meta.json cpan

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary