ENGINEERING MILESTONES / 已取得成绩

Designed on paper.
Verified on silicon.

Qwen3.5-9B-8bit · Apple M4 Pro · 24 GiB · 2026-10-21. Source-linked execution checks, with the workload attached to every result.

28/28Scheduler decisions matchedOriginal C++ reference · real 9B trace16/16Cache token checks passed20 real requests · two sampler settings4,204 + 256Context + generation completedQwen3.5-9B-8bit · M4 Pro / 24 GiB
01 / COST-AWARE SCHEDULING

28 decisions.
28 original-reference matches.

A real hybrid decoding trace produced 58 cycles and 28 mature scheduler decisions. Independent chronological reconstruction and the unchanged original C++ MTPVarlen reference matched all 28. The candidate and baseline generated identical complete sequences of 128 tokens on this development request.

Scope: one real 9B development trace, with a separate CPU supplemental verification. This milestone measures scheduler correspondence and output identity.

Supplemental verification JSON ↗ · Immutable execution trace ↗
02 / EXACT CONTEXT REUSE

20 requests.
16 complete-token checks.

Repeated prompts and changed-tail branches passed all 16 full-output token equality checks across 20 real requests. The control spans greedy and recommended non-thinking sampling, cache on/off, and optional message checkpoints.

For the changed-tail request in both sampler settings, the verified reused prefix increased from 512 to 2,014 tokens with message checkpoints.

Scope: one 2K development session, three request shapes and two sampler settings. The token count describes reusable prefix coverage.

Source JSON ↗
03 / PRACTICAL LOCAL CONTEXT

4,204 input tokens.
256 generated tokens.

The primary 9B8bit model completed the full configured input and output in a single baseline resource check. Streamed text matched the final result. The request-scoped Metal peak was 10.65 GB; the measured system swap increase from loading through completion was zero.

Scope: one baseline request on this M4 Pro / 24 GiB machine. Peak is decimal GB; swap is a system-wide before/after measurement for this run.

Source JSON ↗
04 / CONNECTED LOCAL EXPERIENCE

CLI. API. Browser.
A single working engine.

Real local UI generation, seeded baseline sampling, streamed chat, cancellation and recovery have exercised the service. A separate explicit joint API control matched baseline tokens, returned strict JSON in both requests and completed the SSE stream.

MODEL ADAPTER / 模型可替换

A new model family.
The same working interface.

SmolLM2-135M-Instruct-8bit has completed real baseline/native token matching for English and Chinese inputs, dense cache branch reuse, CLI generation, API and SSE checks. Registration, explicit fallback, cancellation and recovery were exercised with the actual model.

Scope: baseline integration of this 135M dense model. Qwen3.5-9B-8bit remains the primary demonstration. These checks establish integration correctness.

MULTI-REQUEST / 多请求执行

Many requests.
One accountable lifecycle.

Bounded admission, per-request state and cancellation recovery now connect to real joint target verification. The joint service passed 35/35 functional checks with distinct SmolLM2 360M target and 135M draft models, including concurrent requests, actual token SSE and recovery after cancellation.

330 portable tests + 48 real Metal tests passed at this checkpoint. The small models are engineering controls; Qwen3.5-9B remains the primary demonstration. Default service uses baseline; joint execution is explicit opt-in.

Trace every milestone.

The full technical record includes reproducible commands, protocols, source revisions and experiment history.

Open technical documentation ↗Showcase source hashes ↗