Make local inference
more inspectable.
Contribute a focused improvement, a reproducible report or a well-defined model integration.
Make the budget a decision.
A DeepInfer MTPVarlen-derived scheduler uses observed acceptance and complete-cycle cost to choose the next draft budget. Decisions can be replayed against the original C++ reference.
Preserve the exact state.
Hybrid attention and recurrent state travel together. Checkpoint, restore and replay the consumed prefix when a draft is rejected, keeping mutable branches isolated.
Carry context forward.
Native MLX-LM LRU reuse and exact hybrid snapshots share a bounded cache interface. Verified message boundaries create reusable checkpoints for repeated and changed-tail requests.
Code. Context. Reproduction.
Pin model and source revisions, keep complete token controls, and publish the command behind the result. Exact state transitions and cancellation behavior are valuable contributions alongside new execution paths.
uv sync --frozen --extra dev uv run pytest -q -m 'not metal' uv run ruff check src tests scripts benchmarks/experiments benchmarks/qualityFull development setup ↗