PyPI Stats
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About
Home

Search Packages

Find Python packages by name, description, GitHub topic, or filter by metrics
twardoch
split-markdown4gpt

A Python tool for splitting large Markdown files into smaller sections based on a specified token limit. This is particularly useful for processing large Markdown files with GPT models, as it allows the models to handle the data in manageable chunks.

2K 29 3
alasdairforsythe
tokenmonster

Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript

2K 623 21
    • Data from PyPI, GitHub, ClickHouse, and BigQuery