Streaming Data Python Packages

smart-open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

71.2M 3K 385

optbinning

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

220K 522 116

river

🌊 Online machine learning in Python

219K 6K 624

quixstreams

Python Streaming DataFrames for Kafka

75K 2K 105

shioaji

Shioaji all new cross platform api for trading ( 跨平台證券交易API )

70K 426 31

streamz

Real-time stream processing for python

50K 1K 150

bytewax

Python Stream Processing

29K 2K 109

dbt-materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

5K 6K 500

streaming-json-parser

A streaming JSON parser that processes JSON data incrementally, handling partial states. Useful for incrementally parsing partial responses from streaming outputs of Large Language Models (LLMs).

5K 13 2

scikit-multiflow

A machine learning package for streaming data in Python. The other ancestor of River.

4K 794 189

rdst

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

4K 5K 159