Lakehouse Python Packages

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

pydoris-custom

Apache Doris is an easy-to-use, high performance and unified analytics database.

pydoris

Apache Doris is an easy-to-use, high performance and unified analytics database.

pysail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

dbt-fabric-samdebruyn

Maintained and extended fork combining dbt-fabric and dbt-fabricspark

lakehouse-plumber

The Metadata Driven framework for Databricks Lakeflow Declarative Pipelines (formerly Delta Live Tables). Metadata framework that generates production ready Pyspark code for Lakeflow Declarative Pipelines

dbt-doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

lakehouse-engine

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

laketower

Oversee your lakehouse

apache-gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

databend

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

ytsaurus-spyt

YTsaurus is a scalable and fault-tolerant open-source big data platform.

etherealogic-aetheriaforge

Databricks-native intelligent data transformation engine — coherence-scored Bronze/Silver/Gold with entity resolution and temporal reconciliation in a single deployable product.

redpanda-polaris-catalog-python

Apache Polaris, the interoperable, open source catalog for Apache Iceberg

lakebench

A multi-modal Python library for benchmarking Azure lakehouse engines and ELT scenarios, supporting both industry-standard and novel benchmarks.

timeseries-table-format

Append-only time-series table format with gap/overlap tracking (Python bindings).

ibm-watsonxdata-mcp-server

Model Context Protocol (MCP) server for IBM watsonx.data - enables AI assistants to query and explore lakehouse data Resources

pyfluss

Apache Fluss (incubating) Python client

doris-mcp-server

Enterprise-grade Model Context Protocol (MCP) server implementation for Apache Doris

space-datasets

Unified storage framework for the entire machine learning lifecycle

datacoolie

Metadata-driven ETL framework for portable data pipelines across Polars, Spark, Fabric, Databricks, and AWS.

python-for-fluss

Python bindings for Fluss

apache-polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg

ftm-lakehouse

Data standard and archive storage for structured FollowTheMoney data, leaked data, private and public document collections.