https://github.com/piskvorky/smart_open

boto bz2 file gzip-stream hacktoberfest hdfs python s3 streaming streaming-data webhdfs

Score: 33.79787490938288

Last synced: about 11 hours ago
JSON representation

Repository metadata:

Utils for streaming large files (S3, HDFS, gzip, bz2...)

Host: GitHub
URL: https://github.com/piskvorky/smart_open
Owner: piskvorky
License: mit
Created: 2015-01-02T13:05:52.000Z (over 11 years ago)
Default Branch: develop
Last Pushed: 2026-09-29T18:37:58.000Z (3 days ago)
Last Synced: 2026-09-30T03:58:19.054Z (3 days ago)
Topics: boto, bz2, file, gzip-stream, hacktoberfest, hdfs, python, s3, streaming, streaming-data, webhdfs
Language: Python
Homepage:
Size: 1.85 MB
Stars: 3,459
Watchers: 42
Forks: 391
Open Issues: 3
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Funding:
- Github: piskvorky

Owner metadata:

Name: Radim Řehůřek
Login: piskvorky
Email:
Kind: user
Description: Creator of Gensim. Founder and CTO at pii-tools.com. I love history and beginnings in general.
Website: https://radimrehurek.com/
Location:
Twitter: radimrehurek
Company: @pii-tools
Icon url: https://avatars.githubusercontent.com/u/610412?u=457738a74109eba091a4338bb37536609ce0145a&v=4
Repositories: 60
Last Synced at: 2026-09-25T22:57:51.054Z
Profile URL: https://github.com/piskvorky

GitHub Events

Total

Create event: 34
Delete event: 29
Fork event: 10
Issue comment event: 221
Issues event: 56
Member event: 1
Pull request event: 52
Pull request review comment event: 36
Pull request review event: 41
Push event: 129
Release event: 7
Watch event: 183
Total: 799

Last Year

Create event: 24
Delete event: 19
Fork event: 2
Issue comment event: 15
Issues event: 6
Pull request event: 11
Pull request review comment event: 3
Pull request review event: 4
Push event: 90
Release event: 4
Watch event: 29
Total: 207

Committers metadata

Last synced: 1 day ago

Total Commits: 1,029
Total Committers: 132
Avg Commits per committer: 7.795
Development Distribution Score (DDS): 0.552

Commits in past year: 77
Committers in past year: 9
Avg Commits per committer in past year: 8.556
Development Distribution Score (DDS) in past year: 0.208

Name	Email	Commits
Michael Penkov	m**@p**v	461
ddelange	1****e	93
Ivan Menshikh	m**v@g**m	73
Radim Řehůřek	r**k@s**z	50
Lev Konstantinovskiy	l**t@g**m	48
Zikes, Jan	j**s@m**m	29
Peter	p**1@g**m	22
Adam Mertz	a**z@g**m	21
Ivan Menshikh	i**n@z**m	18
Vincent Kríž	v**z@k**u	13
dependabot[bot]	4****]	12
Peter Dannemann	p**n@b**g	8
Rob Cowie	r**e@m**m	7
Vadim Markovtsev	v**m@s**h	6
interpolatio	3****o	6
salilb	s**b@o**m	5
Philippe Ombredanne	p**e@n**m	5
Alexandre Sieira	a**a@g**m	5
Matthew Crowson	m**n@g**m	4
Edan Schwartz	e**z@o**m	4
Judah Rand	1****d	3
John Vandenberg	j**b@g**m	3
Adrian Partl	a**n@p**t	3
RachitSharma2001	4****1	3
Daniel Eisner	r****o	3
nikicc	n**c@g**m	2
rileypeterson	r**1@g**m	2
Peng Yu	y****k	2
DeflateAwning	1****g	2
Jack Cushman	j**n@g**m	2
and 102 more...

Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 109
Total pull requests: 158
Average time to close issues: over 1 year
Average time to close pull requests: 3 months
Total issue authors: 91
Total pull request authors: 67
Average comments per issue: 5.16
Average comments per pull request: 2.53
Merged pull request: 109
Bot issues: 0
Bot pull requests: 8

Past year issues: 2
Past year pull requests: 14
Past year average time to close issues: 12 days
Past year average time to close pull requests: 1 day
Past year issue authors: 2
Past year pull request authors: 6
Past year average comments per issue: 6.5
Past year average comments per pull request: 0.71
Past year merged pull request: 14
Past year bot issues: 0
Past year bot pull requests: 4

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/piskvorky/smart_open

Top Issue Authors

mpenkov (5)
ziky90 (4)
piskvorky (4)
tmylk (3)
AndreaCrotti (3)
nelhage (2)
coreyhuinker (2)
mayk0gan (2)
ddelange (2)
nithinkrishna2107 (1)
indrajeet307 (1)
musicinmybrain (1)
janrygl (1)
electronmods (1)
PatrikBernhard (1)

Top Pull Request Authors

ddelange (49)
tmylk (8)
mpenkov (8)
dependabot[bot] (8)
ziky90 (5)
piskvorky (5)
VincTheSecond (4)
asieira (3)
nikicc (3)
geovalexis (2)
val314159 (2)
davidszotten (2)
robcowie (2)
quantumfusion (2)
tejinderss (2)

Top Issue Labels

bug (18)
good first issue (8)
wishlist (7)
help wanted (6)
enhancement (4)
performance (4)
housekeeping (3)
medium difficult (2)
question (2)
Hacktoberfest (2)
new-feature (1)
duplicate (1)
awaiting-response (1)
stale (1)
hard (1)
ci (1)

Top Pull Request Labels

github_actions (8)
dependencies (8)
enhancement (5)
stale (3)
good first issue (2)
awaiting-response (2)
documentation (1)
medium difficult (1)
bug (1)

Package metadata

Total packages: 16
Total downloads:
- pypi: 56,655,439 last-month
Total docker downloads: 986,441,609
Total dependent packages: 252 (may contain duplicates)
Total dependent repositories: 10,157 (may contain duplicates)
Total versions: 138
Total maintainers: 5

pypi.org: smart-open

Utils for streaming large files (S3, HDFS, GCS, SFTP, Azure Blob Storage, gzip, bz2, zst...)

Homepage:
Documentation: https://smart-open.readthedocs.io/
Licenses: MIT License
Latest release: 8.0.2 (published 3 days ago)
Last Synced: 2026-10-01T04:42:14.566Z (1 day ago)
Versions: 80
Dependent Packages: 248
Dependent Repositories: 10,157
Downloads: 56,655,431 Last month
Docker Downloads: 986,441,609
Rankings:
- Downloads: 0.056%
- Dependent repos count: 0.084%
- Dependent packages count: 0.088%
- Docker downloads count: 0.561%
- Average: 1.058%
- Stargazers count: 1.895%
- Forks count: 3.663%
Maintainers (3)

pypi.org: tethys-smart-open

Utils for streaming large files (S3, HDFS, GCS, Azure Blob Storage, gzip, bz2...)

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://tethys-smart-open.readthedocs.io/
Licenses: MIT
Latest release: 6.2.0.dev2 (published almost 4 years ago)
Last Synced: 2026-10-01T04:41:58.992Z (1 day ago)
Versions: 3
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 4.842%
- Average: 5.587%
- Dependent repos count: 6.331%

proxy.golang.org: github.com/piskvorky/smart_open

Homepage:
Documentation: https://pkg.go.dev/github.com/piskvorky/smart_open#section-documentation
Licenses: mit
Latest release: v8.0.2+incompatible (published 3 days ago)
Last Synced: 2026-10-01T04:41:59.803Z (1 day ago)
Versions: 37
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent packages count: 6.456%
- Average: 6.674%
- Dependent repos count: 6.892%

spack.io: py-smart-open

smart_open is a Python 2 & Python 3 library for efficient streaming of very large files from/to S3, HDFS, WebHDFS, HTTP, or local storage. It supports transparent, on-the-fly (de-)compression for a variety of different formats.

Homepage: https://github.com/piskvorky/smart_open
Licenses: []
Latest release: 5.2.1 (published over 4 years ago)
Last Synced: 2026-10-01T04:41:54.511Z (1 day ago)
Versions: 3
Dependent Packages: 4
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Average: 7.093%
- Dependent packages count: 14.186%
Maintainers (1)
- marcusboden

pypi.org: srcd_smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...) - temporary source{d} fork

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://srcd_smart_open.readthedocs.io/
Licenses: MIT
Latest release: 1.9.0 (published over 7 years ago)
Last Synced: 2026-10-01T04:41:54.414Z (1 day ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 8 Last month
Rankings:
- Stargazers count: 1.357%
- Forks count: 2.707%
- Dependent packages count: 6.633%
- Average: 11.95%
- Downloads: 18.442%
- Dependent repos count: 30.611%
Maintainers (1)
- vmarkovtsev

ubuntu-22.04: smart-open

Homepage: https://github.com/piskvorky/smart_open
Licenses: mit
Latest release: 5.2.1-4 (published 8 months ago)
Last Synced: 2026-09-21T02:31:08.706Z (12 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

ubuntu-24.10: smart-open

Homepage: https://github.com/piskvorky/smart_open
Licenses: mit
Latest release: 7.0.4-1 (published 8 months ago)
Last Synced: 2026-08-24T12:27:07.729Z (about 1 month ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

nixpkgs-unstable: python314Packages.smart-open

Library for efficient streaming of very large file

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/smart-open/default.nix#L88
Licenses: MIT
Latest release: 7.5.0 (published 7 months ago)
Last Synced: 2026-08-24T12:26:37.654Z (about 1 month ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

debian-13: smart-open

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://packages.debian.org/trixie/smart-open
Licenses: mit
Latest release: 7.1.0-1 (published 8 months ago)
Last Synced: 2026-08-24T12:26:55.695Z (about 1 month ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

ubuntu-23.04: smart-open

Homepage: https://github.com/piskvorky/smart_open
Licenses: mit
Latest release: 5.2.1-5 (published 8 months ago)
Last Synced: 2026-03-12T01:21:48.739Z (7 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

debian-12: smart-open

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://packages.debian.org/bookworm/smart-open
Licenses: mit
Latest release: 5.2.1-5 (published 8 months ago)
Last Synced: 2026-03-13T16:41:20.228Z (7 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

ubuntu-23.10: smart-open

Homepage: https://github.com/piskvorky/smart_open
Licenses: mit
Latest release: 5.2.1-5 (published 8 months ago)
Last Synced: 2026-03-14T05:18:03.065Z (7 months ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

ubuntu-24.04: smart-open

Homepage: https://github.com/piskvorky/smart_open
Licenses: mit
Latest release: 5.2.1-5 (published 8 months ago)
Last Synced: 2026-09-17T03:31:10.562Z (16 days ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

nixpkgs-unstable: python313Packages.smart-open

Library for efficient streaming of very large file

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/smart-open/default.nix#L88
Licenses: MIT
Latest release: 7.5.0 (published 7 months ago)
Last Synced: 2026-05-14T14:03:37.739Z (5 months ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

debian-11: smart-open

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://packages.debian.org/bullseye/smart-open
Licenses: mit
Latest release: 1.11.1-3 (published 8 months ago)
Last Synced: 2026-08-24T12:26:50.749Z (about 1 month ago)
Versions: 1
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

guix: python-smart-open

Utilities for streaming large files

Homepage: https://github.com/piskvorky/smart_open
Documentation: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/python-web.scm#n12464
Licenses: expat
Latest release: 7.3.0 (published 7 months ago)
Last Synced: 2026-04-27T16:40:27.044Z (5 months ago)
Versions: 2
Dependent Packages: 0
Dependent Repositories: 0
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%

Dependencies

integration-tests/requirements.txt pypi

awscli *
pytest *
pytest_benchmark *

.github/workflows/python-package.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/release.yml actions

actions/checkout v1 composite
actions/setup-python v1 composite

pyproject.toml pypi

wrapt *

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Summary