oMLX 0.7.1.dev1 发布：新增 Clef/OpenJev 决策模型与批量预填充，Qwen 等模型解码提速最高 61%

全部来源

• oMLX 0.7.1.dev1 发布：新增 Clef/OpenJev 决策模型与批量预填充，Qwen 等模型解码提速最高 61%

  jundotkim · 主条目

  oMLX 发布 0.7.1.dev1 dev 版本，新增决策模型（Clef、OpenJev）、并发请求批量预填充和全新 Web UI。速度方面，Qwen3.8-Flash-Next 专家卸载、M3 Ultra 上 32K 预填充从 396 提升至 636 tok/s（+61%），Qwen3.6-35B-A3B 在 M5 Max 上解码从 156 提升至 205 tok/s（+31%），批量预填充使 8 个间隔 0.4 秒到达的提示词首 token 时间从 7.41 秒降至 5.75 秒（-22%）。

  查看原始帖子 ↗
  查看原文

  oMLX 0.7.1.dev1 is out! This dev release adds decision models (Clef, OpenJev), batched prefill for concurrent requests, a refreshed web UI, and faster Qwen3.8, Qwen3.6, GLM-5.3-Flash and DeepSeek V4.1. https://github.com/jundot/omlx/releases/tag/v0.7.1.dev1 Speedups in this release - Qwen3.6-35B-A3B decode on M5 Max: 156 -> 205 tok/s (+31%). - GLM-5.3-Flash decode on M3 Ultra: 31.3 -> 40.6 tok/s (+30%). The fused decode kernels now run on M3 and M4, not only M5. - Qwen3.8-Flash-Next with expert offload, 32K prefill on M3 Ultra: 396 -> 636 tok/s (+61%). - Qwen3.8-Flash-Next, 8 prompts arriving 0.4 s apart on M5 Max: time to first token 7.41 -> 5.75 s (-22%) with batched prefill. What's new - Decision models. Clef, Clef-Flash and OpenJev answer typed questions with a probability for each option through the new /v1/systemone endpoint. - Batched prefill. Prompts that prefill at the same time now share one forward pass, so MoE models read their expert weights once. - A refreshed web UI with a log table, a settings rail and a customizable dashboard. - Native video input for Qwen3.5, 3.6 and 3.8. - More accurate oQe quantization, with 11-15% lower KL to the original model. - Fewer memory guard rejections in long sessions. Most of the speedups came from contributors again. Thanks to everyone who sent PRs and tested the dev builds!

前后事件