LOCAL DEMO / 本地演示

See the engine.
Follow the response.

Real Qwen3.5-9B-8bit execution, from a repaired Python function to a structured streamed response. 真实本地运行,可查看截图、回放和来源。

RECORDED LOCAL UI / 真实截图

A practical code repair.

The local browser workspace repaired a Python clamp function and produced the same token sequence as direct engine execution.

```python
def clamp(x, lo, hi):
    return max(lo, min(hi, x))
```
0.94sfirst token
1.83srequest total
27.94decode tok/s

Single recorded interactive request, 62 input tokens and 26 generated tokens including EOS; measured on M4 Pro / 24 GiB. Request timing describes this interaction.

Source JSON ↗
Real Safari local UI showing a completed clamp function repair and request metrics
RECORDED SSE REPLAY / 录制回放

Watch a real text stream.

A structured extraction request from the recorded baseline API control. Playback follows captured content-event timestamps at your selected speed. All inference ran locally when recorded.

RECORDED REPLAY · QWEN3.5-9B-8BIT / BASELINE2,048 input tokens
Extract project IDs, integer costs and owners into a strict JSON object.
Press play to replay the captured response.
Ready · recorded baseline response

Recorded replay uses the real baseline content events and elapsed timestamps, including the recorded prefill wait. Playback speed changes presentation only.

Source request, SSE events and token controls ↗
GET STARTED

Your next inference runs here.

Apple Silicon · Python 3.12 / 3.13 · uv. Start the bundled local UI and API with the locked 9B model.

TERMINAL / MAIN BRANCH
git clone --branch main --single-branch https://github.com/ZEVIN17/deepinfer-mac.git
cd deepinfer-mac
uv sync --frozen --extra mlx
uv run deepinfer-mac doctor
uv run deepinfer-mac models --download qwen3.5-9b-8bit --target-only
uv run deepinfer-mac serve
01
Check your Mac

Doctor reports Metal, Python, available memory and disk.

02
Open the local workspace

Visit http://127.0.0.1:8765 and select qwen3.5-9b-8bit. The default path is baseline generation.

03
Build on the same engine

Use CLI, streamed API or the bundled browser UI. Explore the explicit adaptive entry in the architecture guide.

Run guide & API example ↗