tiktoken
Fast BPE tokenizer for OpenAI models: count tokens before they hit the context window.
Fast BPE tokenizer for OpenAI models: count tokens before they hit the context window.

Turns any GitHub repo into a prompt-friendly text digest (swap 'hub' for 'ingest' in the URL).
Website captured Oct 19
Latest collected value
Latest collected value
Latest collected value
Latest collected value
Latest collected value
Latest collected value
Latest collected value
| Signal | tiktoken 7d | tiktoken 28d | Gitingest 7d | Gitingest 28d |
|---|---|---|---|---|
| GitHub stars | ▲0.2% | ▲0.9% | ▲0.2% | ▲1.1% |
| GitHub forks | ▲0.7% | ▲1.7% | ▲0.4% | ▲1.9% |
| PyPI downloads (last week) | ▲5.5% | ▲12.7% | ▲1.3% | ▼57.4% |
| Hacker News stories (7 days) | — | — | — | — |
| Hacker News points (7 days) | — | — | — | — |
| GitHub open issues & PRs | ▲0.7% | ▲4.2% | ▼19.2% | ▼4.5% |
| GitHub contributors | ▲0.0% | ▲4.0% | ▲0.0% | ▲0.0% |