https://github.com/harbor-framework/harbor
evals rl-environments terminal-bench
Score: 30.525346937553152
Last synced: 1 day ago
JSON representation
Repository metadata:
Framework for evaluating and improving agents
- Host: GitHub
- URL: https://github.com/harbor-framework/harbor
- Owner: harbor-framework
- License: apache-2.0
- Created: 2025-08-04T23:28:26.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2026-11-07T04:17:50.000Z (1 day ago)
- Last Synced: 2026-11-07T04:21:55.098Z (1 day ago)
- Topics: evals, rl-environments, terminal-bench
- Language: Python
- Homepage: https://harborframework.com/
- Size: 110 MB
- Stars: 6,860
- Watchers: 25
- Forks: 2,155
- Open Issues: 1,238
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.cff
- Agents: AGENTS.md
- Claude: CLAUDE.md
Owner metadata:
- Name: Harbor
- Login: harbor-framework
- Email:
- Kind: organization
- Description:
- Website: https://harborframework.com/
- Location: United States of America
- Twitter: harborframework
- Company:
- Icon url: https://avatars.githubusercontent.com/u/256467836?v=4
- Repositories: 20
- Last Synced at: 2026-11-06T13:52:39.948Z
- Profile URL: https://github.com/harbor-framework
GitHub Events
Total
- Create event: 121
- Delete event: 80
- Fork event: 98
- Issue comment event: 532
- Issues event: 46
- Member event: 1
- Pull request event: 525
- Pull request review comment event: 833
- Pull request review event: 1148
- Push event: 801
- Release event: 4
- Watch event: 277
- Total: 4466
Last Year
- Create event: 121
- Delete event: 80
- Fork event: 98
- Issue comment event: 532
- Issues event: 46
- Member event: 1
- Pull request event: 525
- Pull request review comment event: 833
- Pull request review event: 1148
- Push event: 801
- Release event: 4
- Watch event: 277
- Total: 4466
Committers metadata
Last synced: 1 day ago
Total Commits: 1,913
Total Committers: 378
Avg Commits per committer: 5.061
Development Distribution Score (DDS): 0.676
Commits in past year: 1,757
Committers in past year: 372
Avg Commits per committer in past year: 4.723
Development Distribution Score (DDS) in past year: 0.706
| Name | Commits | |
|---|---|---|
| Alex Shaw | a****4@g****m | 619 |
| Kobe Chen | x****0@g****m | 197 |
| Sam Vance | 5****e | 65 |
| Boxuan Li | l****n@c****k | 55 |
| benediktstroebl | 5****l | 43 |
| ZHAO Jin-Xiang | x****e@g****m | 38 |
| Ryan Marten | r****0@g****m | 23 |
| Nick Hollon | n****n@l****v | 21 |
| Ben Feuer | b****6@n****u | 19 |
| Crystal Zhou | 4****z | 18 |
| Zixuan Zhu | 9****3 | 17 |
| Kevin Xiang Li | k****8@g****m | 16 |
| Steven Dillmann | 1****n | 14 |
| mgeorgaklis | 1****s | 13 |
| Shanshan Wang | c****l@g****m | 12 |
| Robert Zhang | 0****g@g****m | 11 |
| Mike Merrill | m****m@c****u | 10 |
| Ayush Nangia | 5****a | 10 |
| Rishi Desai | R****i | 9 |
| Ivan Bercovich | i****h@g****m | 9 |
| Charlie Ruan | c****n@b****u | 9 |
| Slimshilin | s****4@g****m | 8 |
| Ruiyang Wang | 5****g | 8 |
| Rebecca Deng | 7****t | 8 |
| Joan Cabezas | j****s@g****m | 8 |
| Marco Rossignoli | m****i@g****m | 8 |
| Walker Hughes | 7****s | 8 |
| Maxwill Lin | 0****3@g****m | 7 |
| Kwuang Tang | k****g@s****i | 7 |
| Kurry Tran | k****n@g****m | 7 |
| and 348 more... | ||
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 109
Total pull requests: 592
Average time to close issues: about 1 month
Average time to close pull requests: 15 days
Total issue authors: 92
Total pull request authors: 271
Average comments per issue: 1.11
Average comments per pull request: 3.24
Merged pull request: 142
Bot issues: 0
Bot pull requests: 0
Past year issues: 109
Past year pull requests: 592
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 15 days
Past year issue authors: 92
Past year pull request authors: 271
Past year average comments per issue: 1.11
Past year average comments per pull request: 3.24
Past year merged pull request: 142
Past year bot issues: 0
Past year bot pull requests: 0
Top Issue Authors
- raokarthik99 (5)
- Mike-7777777 (3)
- RyanMarten (3)
- Larrtroffen (3)
- no-hup (3)
- technicalpickles (2)
- marcusschiesser (2)
- ehutt (2)
- KunWuLuan (2)
- shivank-bespokelabs (2)
- ashwaniag-nv (1)
- aitianshi (1)
- Kurry (1)
- avinash-bespokelabs (1)
- RishiDesai (1)
Top Pull Request Authors
- kobe0938 (55)
- scvance (27)
- cktang88 (21)
- alexgshaw (20)
- YanggWendy (16)
- mgeorgaklis (12)
- ayushnangia (12)
- benediktstroebl (10)
- mattteufel-e2b (9)
- asaraog (8)
- Kurry (7)
- no-hup (6)
- werdnabae (5)
- cooleel (5)
- mvanhorn (5)
Top Issue Labels
Top Pull Request Labels
- area:tests (448)
- area:docs (211)
- area:agents (188)
- area:core (180)
- area:environments (171)
- area:cli (69)
- area:package (59)
- area:adapters (48)
- area:viewer (18)
- area:registry (17)
- area:ci (14)
Package metadata
- Total packages: 6
-
Total downloads:
- pypi: 5,903,642 last-month
- Total dependent packages: 0 (may contain duplicates)
- Total dependent repositories: 0 (may contain duplicates)
- Total versions: 89
- Total maintainers: 2
proxy.golang.org: github.com/Harbor-Framework/HARBOR
- Homepage:
- Documentation: https://pkg.go.dev/github.com/Harbor-Framework/HARBOR#section-documentation
- Licenses: apache-2.0
- Latest release: v0.24.0 (published about 1 month ago)
- Last Synced: 2026-11-07T02:59:48.230Z (1 day ago)
- Versions: 31
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 4.011%
- Average: 4.15%
- Dependent repos count: 4.289%
proxy.golang.org: github.com/harbor-framework/harbor
- Homepage:
- Documentation: https://pkg.go.dev/github.com/harbor-framework/harbor#section-documentation
- Licenses: apache-2.0
- Latest release: v0.25.0 (published 10 days ago)
- Last Synced: 2026-11-07T07:15:57.321Z (1 day ago)
- Versions: 31
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent packages count: 4.844%
- Average: 5.007%
- Dependent repos count: 5.17%
pypi.org: harbor-atif2otel
Convert ATIF agent trajectories to OpenTelemetry protobuf spans with pluggable uploaders.
- Homepage:
- Documentation: https://harbor-atif2otel.readthedocs.io/
- Licenses: Apache-2.0
- Latest release: 0.1.1 (published 3 months ago)
- Last Synced: 2026-11-07T04:34:45.204Z (1 day ago)
- Versions: 2
- Dependent Packages: 0
- Dependent Repositories: 0
- Downloads: 7,228 Last month
-
Rankings:
- Dependent packages count: 6.851%
- Average: 22.805%
- Dependent repos count: 38.759%
- Maintainers (1)
pypi.org: harbor-langsmith
LangSmith plugin for Harbor jobs.
- Homepage:
- Documentation: https://harbor-langsmith.readthedocs.io/
- Licenses: Apache-2.0
- Latest release: 0.3.1 (published 3 months ago)
- Last Synced: 2026-11-07T04:34:44.524Z (1 day ago)
- Versions: 8
- Dependent Packages: 0
- Dependent Repositories: 0
- Downloads: 11,818 Last month
-
Rankings:
- Dependent packages count: 7.118%
- Downloads: 22.856%
- Average: 23.413%
- Dependent repos count: 40.265%
- Maintainers (1)
pypi.org: harbor-rewardkit
Lightweight grading toolkit for environment-based tasks.
- Homepage:
- Documentation: https://harbor-rewardkit.readthedocs.io/
- Licenses: Apache-2.0
- Latest release: 0.2.4 (published 7 days ago)
- Last Synced: 2026-11-07T04:41:27.594Z (1 day ago)
- Versions: 16
- Dependent Packages: 0
- Dependent Repositories: 0
- Downloads: 5,884,596 Last month
-
Rankings:
- Dependent packages count: 7.455%
- Average: 24.806%
- Dependent repos count: 42.157%
- Maintainers (2)
pypi.org: alpen
Lightweight grading toolkit for environment-based tasks.
- Homepage:
- Documentation: https://alpen.readthedocs.io/
- Licenses: Apache-2.0
- Latest release: 0.2.1 (published 28 days ago)
- Last Synced: 2026-11-07T04:41:27.235Z (1 day ago)
- Versions: 1
- Dependent Packages: 0
- Dependent Repositories: 0
- Downloads: 0 Last month
-
Rankings:
- Forks count: 0.721%
- Stargazers count: 0.915%
- Dependent packages count: 6.618%
- Average: 26.966%
- Dependent repos count: 37.446%
- Downloads: 89.128%
- Maintainers (1)
Dependencies
- actions/checkout v6 composite
- actions/github-script v9 composite
- actions/setup-python v6 composite
- anthropics/claude-code-action v1 composite
- actions/checkout v6 composite
- actions/setup-python v6 composite
- jasonchou97/sandbox tb6 build
- actions/labeler v6 composite
- actions/checkout v6 composite
- astral-sh/setup-uv v8.1.0 composite
- buildpack-deps jammy build
- python 3.12-slim build
- actions/checkout v6 composite
- astral-sh/setup-uv v8.1.0 composite
- codecov/codecov-action v6 composite
- denoland/setup-deno v2 composite
- docker/setup-compose-action v2 composite
- docker/setup-docker-action v5 composite
- python 3.10-slim build
- python 3.12-slim build
- ghcr.io/laude-institute/t-bench/deveval latest build
- ${docker_base} latest build
- debian bookworm-slim build
- {base_image} latest build
- happysixd/osworld-docker latest build
- python 3.13-slim-bookworm build
- matplotlib
- pip
- python 3.10.9
- python 3.13-slim-bookworm build
- actions/checkout v6 composite
- actions/github-script v9 composite
- actions/checkout v6 composite
- astral-sh/setup-uv v8.1.0 composite
- python 3.10-slim build
- rust 1.83-slim build
- {docker_image} latest build
- ghcr.io/michaelrglass/kramabench-${DOMAIN} 1.0 build
- ubuntu jammy build
- continuumio/miniconda3 latest build
- actions/github-script v9 composite
- actions/checkout v6 composite
- astral-sh/setup-uv v8.1.0 composite
- python 3.11-slim build
- python 3.11-alpine build
- {image_name} latest build
- python 3.11-slim build
- futurehouse/bixbench aviary-notebook-env build
- {image_name_oh} latest build
- ghcr.io/laude-institute/t-bench/deveval latest build
- ghcr.io/laude-institute/t-bench/ubuntu-24-04 20250624 build
- python 3.12-slim build
- actions/checkout v6 composite
- anthropics/claude-code-action v1 composite
- python 3.12-slim build
- ubuntu 22.04 build
- python 3.13-slim-bookworm build
- actions/checkout v4 composite
- dieghernan/cff-validator v3 composite
- python 3.11-slim build
- python 3.11-slim build
- python 3.11-slim build
- actions/checkout v6 composite
- python 3.11-slim build
- ghcr.io/laude-institute/t-bench/ubuntu-24-04 20250624 build
- python 3.12-bookworm build
- python 3.12-slim build
- actions/checkout v6 composite
- anthropics/claude-code-action v1 composite
- ubuntu 24.04 build
- python 3.12-slim build
- python 3.10-slim build
- python 3.13-slim-bookworm build
- actions/checkout v6 composite
- astral-sh/setup-uv v8.1.0 composite
- oven-sh/setup-bun v2 composite
- actions/github-script v9 composite
- ghcr.io/laude-institute/t-bench/ubuntu-24-04 20250624 build
- python 3.11-slim build
- python 3.11-slim build
- python 3.11-slim build
- ghcr.io/laude-institute/t-bench/python-3-13 20250620 build
- python 3.11-slim build
- python 3.12-slim build
- {docker_image} latest build
- ghcr.io/laude-institute/t-bench/python-3-13 20250620 build
- {docker_image} latest build
- ghcr.io/stevendillmann/researchbench-base latest build
- datasets >=2.0.0
- aiohappyeyeballs 2.6.1
- aiohttp 3.13.5
- aiosignal 1.4.0
- annotated-doc 0.0.4
- anyio 4.13.0
- async-timeout 5.0.1
- attrs 26.1.0
- certifi 2026.2.25
- charset-normalizer 3.4.7
- click 8.3.2
- colorama 0.4.6
- datasets 4.8.4
- dill 0.4.1
- exceptiongroup 1.3.1
- filelock 3.29.0
- frozenlist 1.8.0
- fsspec 2026.2.0
- h11 0.16.0
- harbor-bird-bench-adapter 0.1.0
- hf-xet 1.4.3
- httpcore 1.0.9
- httpx 0.28.1
- huggingface-hub 1.11.0
- idna 3.11
- markdown-it-py 4.0.0
- mdurl 0.1.2
- multidict 6.7.1
- multiprocess 0.70.19
- numpy 2.2.6
- numpy 2.4.4
- packaging 26.1
- pandas 2.3.3
- pandas 3.0.2
- propcache 0.4.1
- pyarrow 23.0.1
- pygments 2.20.0
- python-dateutil 2.9.0.post0
- pytz 2026.1.post1
- pyyaml 6.0.3
- requests 2.33.1
- rich 15.0.0
- shellingham 1.5.4
- six 1.17.0
- tqdm 4.67.3
- typer 0.24.1
- typing-extensions 4.15.0
- tzdata 2026.1
- urllib3 2.6.3
- xxhash 3.6.0
- yarl 1.23.0
- datasets >=3.2.0
- huggingface-hub >=0.20.0
- gdown >=5.2.1
- jinja2 >=3.1.6
- {image_name} latest build
- {docker_image} latest build
- ghcr.io/laude-institute/t-bench/python-3-13 20250620 build
- python 3.11-slim build
- jyxsu6/medagentbench@sha256 3fb83d7ed71c5476f9eb6212bd440a909ef7505922bbc757dc488a8fc0701966 build
- python 3.11-slim-bookworm build
- python 3.11-slim build
- swelancer/swelancer_x86_{issue_id} releasev1 build
- 103 dependencies
- python 3.11-slim-bookworm build
- python 3.11-slim build
- {docker_image} latest build
- python 3.11-slim build
- 389 dependencies
- aiohappyeyeballs 2.6.1
- aiohttp 3.13.5
- aiosignal 1.4.0
- annotated-doc 0.0.4
- anyio 4.13.0
- async-timeout 5.0.1
- attrs 26.1.0
- certifi 2026.2.25
- charset-normalizer 3.4.7
- click 8.3.2
- colorama 0.4.6
- datasets 4.8.4
- dill 0.4.1
- exceptiongroup 1.3.1
- filelock 3.29.0
- frozenlist 1.8.0
- fsspec 2026.2.0
- h11 0.16.0
- harbor-autocodebench-adapter 0.1.0
- hf-xet 1.4.3
- httpcore 1.0.9
- httpx 0.28.1
- huggingface-hub 1.11.0
- idna 3.11
- markdown-it-py 4.0.0
- mdurl 0.1.2
- multidict 6.7.1
- multiprocess 0.70.19
- numpy 2.2.6
- numpy 2.4.4
- packaging 26.1
- pandas 2.3.3
- pandas 3.0.2
- propcache 0.4.1
- pyarrow 23.0.1
- pygments 2.20.0
- python-dateutil 2.9.0.post0
- pytz 2026.1.post1
- pyyaml 6.0.3
- requests 2.33.1
- rich 15.0.0
- shellingham 1.5.4
- six 1.17.0
- tqdm 4.67.3
- typer 0.24.1
- typing-extensions 4.15.0
- tzdata 2026.1
- urllib3 2.6.3
- xxhash 3.6.0
- yarl 1.23.0
- harbor-locomo-adapter 0.1.0
- aiohappyeyeballs 2.6.1
- aiohttp 3.13.5
- aiosignal 1.4.0
- annotated-doc 0.0.4
- annotated-types 0.7.0
- anyio 4.13.0
- attrs 26.1.0
- certifi 2026.2.25
- charset-normalizer 3.4.7
- click 8.3.2
- colorama 0.4.6
- datasets 4.8.4
- dill 0.4.1
- filelock 3.29.0
- frozenlist 1.8.0
- fsspec 2026.2.0
- h11 0.16.0
- harbor-usaco-adapter 0.1.0
- hf-xet 1.4.3
- httpcore 1.0.9
- httpx 0.28.1
- huggingface-hub 1.11.0
- idna 3.11
- markdown-it-py 4.0.0
- mdurl 0.1.2
- multidict 6.7.1
- multiprocess 0.70.19
- numpy 2.4.4
- packaging 26.1
- pandas 3.0.2
- propcache 0.4.1
- pyarrow 23.0.1
- pydantic 2.13.2
- pydantic-core 2.46.2
- pygments 2.20.0
- python-dateutil 2.9.0.post0
- pyyaml 6.0.3
- requests 2.33.1
- rich 15.0.0
- shellingham 1.5.4
- six 1.17.0
- toml 0.10.2
- tqdm 4.67.3
- typer 0.24.1
- typing-extensions 4.15.0
- typing-inspection 0.4.2
- tzdata 2026.1
- urllib3 2.6.3
- xxhash 3.6.0
- yarl 1.23.0
- ghcr.io/laude-institute/t-bench/deveval latest build
- {{dataset_base_image}} latest build
- @react-router/dev 7.12.0 development
- @tailwindcss/vite ^4.1.18 development
- @types/node ^22 development
- @types/react ^19.2.7 development
- @types/react-dom ^19.2.3 development
- tailwindcss ^4.1.18 development
- tw-animate-css ^1.4.0 development
- typescript ^5.9.2 development
- vite ^7.1.7 development
- vite-tsconfig-paths ^5.1.4 development
- @pierre/trees ^1.0.0-beta.5
- @radix-ui/react-accordion ^1.2.12
- @radix-ui/react-checkbox ^1.3.3
- @radix-ui/react-collapsible ^1.1.12
- @radix-ui/react-dialog ^1.1.15
- @radix-ui/react-hover-card ^1.1.15
- @radix-ui/react-label ^2.1.8
- @radix-ui/react-popover ^1.1.15
- @radix-ui/react-scroll-area ^1.2.10
- @radix-ui/react-select ^2.2.6
- @radix-ui/react-slot ^1.2.4
- @radix-ui/react-tabs ^1.1.13
- @radix-ui/react-tooltip ^1.2.8
- @react-router/node 7.12.0
- @react-router/serve 7.12.0
- @tailwindcss/typography ^0.5.19
- @tanstack/react-query ^5.90.17
- @tanstack/react-table ^8.21.3
- class-variance-authority ^0.7.1
- clsx ^2.1.1
- cmdk ^1.1.1
- hast-util-to-jsx-runtime ^2.3.6
- isbot ^5.1.31
- lucide-react ^0.562.0
- next-themes ^0.4.6
- nuqs ^2.8.6
- react ^19.2.3
- react-dom ^19.2.3
- react-hotkeys-hook ^5.2.3
- react-markdown ^10.1.0
- react-resizable-panels ^4
- react-router 7.12.0
- remark-gfm ^4.0.1
- shiki ^3.21.0
- sonner ^2.0.7
- tailwind-merge ^3.4.0
- zod ^4.3.6
- python 3.11-slim-bookworm build
- huggingface_hub latest build
- python 3.11-slim-bookworm build
- huggingface_hub latest build
- python 3.11-slim-bookworm build
- ghcr.io/laude-institute/t-bench/ubuntu-24-04 20250624 build
- python 3.11-slim build
- python 3.11-slim build
- python 3.11-slim build