Datalake Python Packages

lakefs-sdk

lakeFS - Data version control for your data lake | Git for data

lakefs

lakeFS - Data version control for your data lake | Git for data

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

lakefs-client

lakeFS - Data version control for your data lake | Git for data

pandasai

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

pandasai-openai

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

pan-cortex-data-lake

Python idiomatic SDK for Cortex™ Data Lake.

pandasai-litellm

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

dlt-iceberg

An Iceberg destination for DLT that supports REST catalogs

zingg

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

pancloud

Python idiomatic SDK for Cortex™ Data Lake.

apache-gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

tarn

An insanely customizable framework for key-value storage 💾

aws-orbit-overprovisioning

A Data Platform built for AWS, powered by Kubernetes.

dm-job-lib

Data Manager Job Library

aws-insurancelake-etl

This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, AWS Glue for data transformation, and AWS CDK Pipelines. It is originally based on the AWS blog Deploy data lake ETL jobs using CDK Pipelines, and complements the InsuranceLake Infrastructure project

aws-orbit

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-custom-cfn

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-sdk

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-redshift

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-code-commit

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-team-script-launcher

A Data Platform built for AWS, powered by Kubernetes.

aws-orbit-emr-on-eks

A Data Platform built for AWS, powered by Kubernetes.