S3 Python Packages

s3transfer

Amazon S3 Transfer Manager for Python

smart-open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

moto

A library that allows you to easily mock out tests based on AWS infrastructure.

cloudpathlib

Python pathlib-style classes for cloud storage services such as Amazon S3, Azure Blob Storage, and Google Cloud Storage.

pathy

simple, flexible, offline capable, cloud storage with a Python path-like interface

fs-s3fs

Amazon S3 filesystem for PyFilesystem2

yolov5

Packaged version of ultralytics/yolov5 + many extra features

airbyte-source-declarative-manifest

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

opendal

Apache OpenDAL: One Layer, All Storage.

synapse-s3-storage-provider

Synapse storage provider to fetch and store media in Amazon S3

tiledb

Python interface to the TileDB storage engine

pepperize-cdk-terraform-state-backend

This project provides a CDK construct bootstrapping an AWS account with a S3 Bucket and a DynamoDB table as terraform state backend.

astro-sdk-python

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

pepperize-cdk-autoscaling-gitlab-runner

Execute Gitlab jobs on auto-scaled EC2 instances using the Docker Machine executor.

cloudstorage

Unified cloud storage API for storage services.

s3-concat

Concat multiple files in s3

remotezip

Python module to access single members of a zip archive without downloading the full content from a remote web server.

django-s3direct

Directly upload files to S3 compatible services with Django.

databricks-utils

`databricks-utils` is a python package that provide several utility classes/func that improve ease-of-use in databricks notebook.

renovosolutions-aws-cdk-renovo-s3-bucket

Library to create our S3 buckets

ministack

Ministack: Free, open-source local AWS emulator - 40+ services, Terraform compatible, real databases. Free forever. MIT licensed.

cloud-volume

Read and write Neuroglancer datasets programmatically.

pepperize-cdk-private-bucket

This project provides a CDK construct to create an S3 Bucket with some security pre-sets.

megfile

Megvii FILE Library - Working with Files in Python same as the standard library