One execution cycle.
Every part accounted for.
DeepInfer-derived scheduling meets native MLX execution and exact hybrid-state recovery. 从请求入口到下一轮预算，保持清晰的执行边界。
One engine, three entrances.
CLI, local UI and a supported OpenAI-compatible chat API share a strict request schema and a bounded FIFO worker. A single inference owner executes MLX on Metal and returns text events with request metrics.
Baseline generation uses MLX-LM sampling. The explicit configured path adds a locked draft model, cost-aware budget selection and exact recovery for Qwen3.5 hybrid state.
Execution layers and source attribution ↗Choose. Verify. Recover.
Select a stage to explore its implemented responsibility. This diagram explains the architecture; the measured trace is linked in the milestones.
Choose a budget from observed work.
The MTPVarlen-derived core selects an expected-yield / cost budget. A lagged mean recovery adapter feeds measured complete-cycle costs, keeping scheduling history distinct from partial, interrupted rounds.
observed cycle → cost curve → next draft budgetCore method and adaptation ↗Keep the target in control.
The locked 0.8B draft proposes tokens. The 9B target verifies the proposal in the greedy verification path; accepted tokens and the target correction determine the continuation.
draft proposal → target verification → accepted prefix128-token exact-output control ↗Recover the state actually consumed.
Qwen3.5 combines attention KV with recurrent ArraysCache state. The adapter restores an exact checkpoint and replays the consumed prefix after rejection, EOS or close, rather than assuming every state supports trimming.
isolated checkpoint → restore → consumed-prefix replayState invariants and controls ↗Reuse a verified prefix.
Model revision, template, quantization, thinking mode and draft identity define cache identity. Byte-budgeted snapshots preserve branch isolation. Optional message checkpoints use verified token-prefix boundaries.
exact prefix → bounded snapshot → remaining suffixReal cache controls ↗Make the budget a decision.
A DeepInfer MTPVarlen-derived scheduler uses observed acceptance and complete-cycle cost to choose the next draft budget. Decisions can be replayed against the original C++ reference.
Preserve the exact state.
Hybrid attention and recurrent state travel together. Checkpoint, restore and replay the consumed prefix when a draft is rejected, keeping mutable branches isolated.
Carry context forward.
Native MLX-LM LRU reuse and exact hybrid snapshots share a bounded cache interface. Verified message boundaries create reusable checkpoints for repeated and changed-tail requests.
MLX / MLX-LM supply model execution, sampling and native LRU reuse. The scheduler reimplements pinned DeepInfer methods; hybrid recovery, service integration and measurement are Mac adaptations. Full attribution lives in the source mapping and NOTICE.
Many requests.
One accountable lifecycle.
Bounded admission, per-request state and cancellation recovery now connect to real joint target verification. The joint service passed 35/35 functional checks with distinct SmolLM2 360M target and 135M draft models, including concurrent requests, actual token SSE and recovery after cancellation.
330 portable tests + 48 real Metal tests passed at this checkpoint. The small models are engineering controls; Qwen3.5-9B remains the primary demonstration. Default service uses baseline; joint execution is explicit opt-in.
A shared budget.
Different request depths.
Explore the allocation mechanism: cumulative prefix survival ranks candidate verification positions across requests. The numbers below are illustrative inputs, not model measurements or benchmark results.
Fixed example conditional acceptance inputs: A [0.95, 0.90, 0.85], B [0.80, 0.60, 0.40], C [0.60, 0.50, 0.30]. Each position's score is the cumulative product. Equal scores use request order then position order. This view illustrates allocation only; cost-based total-budget selection, real verification and recovery remain separate responsibilities.
This browser-only mechanism illustration executes no model and makes no latency claim. The separate joint-service report records actual model execution, streaming and cancellation checks.