Delta Lake Python Packages

delta-spark

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

deltalake

A native Rust library for Delta Lake, with bindings into Python

delta-sharing

An open protocol for secure data sharing

koheesio

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

pydoris-custom

Apache Doris is an easy-to-use, high performance and unified analytics database.

pydoris

Apache Doris is an easy-to-use, high performance and unified analytics database.

hops-deltalake

A native Rust library for Delta Lake, with bindings into Python

dask-deltatable

A Delta Lake reader for Dask

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

cdk-emrserverless-with-delta-lake

This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you could also launch an EMR notebook via cluster template to check the outcome from the EMR Serverless application.

delta-lake-reader

Read Delta tables without any Spark

dbt-doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

lakehouse-engine

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

roapi

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

roapi-http

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

duckalog

Build DuckDB catalogs from declarative YAML/JSON configuration files

sqlshell

A powerful SQL shell with GUI interface for data analysis

columnq-cli

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

pfeed

Data Engine for Manual/Algo Trading: Download/Stream -> Clean -> Store. Supports Data Lakehouse Architecture. Clean Once and Forget.

datacoolie

Metadata-driven ETL framework for portable data pipelines across Polars, Spark, Fabric, Databricks, and AWS.

pyspark-streaming-base

This project provides an opinionated way to go about crafting Spark Structured Streaming applications with PySpark

xdlake

A loose implementation of the deltalake protocol, written in Python on top of pyarrow, focused on extensibility, customizability, and distributed data.

databricks4py

Spark, Delta Lake, and Databricks utility library for Python