Skip to content

3.6.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 16 Jun 07:58
3.6.0
cc20f67
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

MNN 3.6.0 版本发布说明

发布日期: 2026年6月

📦 安装: pip install mnn==3.6.0 ｜ PyPI

📌 版本概述

MNN 3.6.0 版本聚焦于 多模态 LLM 模型扩展与推理加速、GPU 多后端深度优化 以及 低比特量化与硬件适配增强 三大核心主题：

  • 多模态 LLM 模型扩展与推理加速：新增 Gemma4（文本/视觉/语音）、Qwen3.5、Hymt2 等模型支持；推出 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成将 TTFA 降低 2.7×；引入 fp16 存储的 weight-quant scale/zero-point；通过 onClone 重构 KV Cache 复用机制。
  • GPU 多后端深度优化：Vulkan 新增 VulkanConv1x1CoopA8（W8A8 coopMat）与 no-subgroup 回退路径；OpenCL 引入 GEMM/GEMV 启发式调度、int4 image1d_buffer_t 优化、原生 TopKV2 与 2/3-bit 权重量化；Metal 优化 Conv1x1 分支选择；CUDA 支持 Blackwell（sm_120）并优化 Eagle 推测解码；QNN 支持 Qwen3.5 与离线图多进程并行导出。
  • 低比特量化与硬件适配增强：CPU 与 GPU 全栈支持 2/3-bit 权重量化（ARM low-bit 量化内核 + OpenCL/Metal/Vulkan）；ARMv8.2/v8.6 decode 路径深度优化；KleidiAI 升级到 v1.16.0；新增 RVV 向量化的 Int8/Norm/Gelu/SiLu/ExpC8 等核心算子；新增 fused MNNFusedGatedDelta（LinearAttention prefill）；新增 LinearAttention L=1 decode 快速路径。

🚀 版本亮点

  • Gemma4 全模态支持：新增 Gemma4 文本、视觉与音频模型的完整推理支持
  • Wan2.1-T2V-1.3B 视频生成：新增 Wan2.1-T2V-1.3B 文生视频模型的导出与推理能力
  • LLM 推理加速：新增 dflash 推测解码；Omni 模型 Thinker/Talker 交错生成，TTFA 降低 2.7×
  • CPU/GPU 2/3-bit 权重量化全栈支持：CPU 新增 low-bit ARM 量化内核，OpenCL/Metal/Vulkan 同步支持 2/3-bit 权重量化
  • Vulkan W8A8 coopMat：新增 VulkanConv1x1CoopA8，支持动态量化 W8A8 + Cooperative Matrix

✨ 新功能

LLM / VLM

  • Gemma4 模型支持：新增 Gemma4 文本、视觉、音频全模态推理
  • dflash 推测解码：新增 dflash 推测解码方案
  • Omni Thinker/Talker 交错生成：Omni 模型 Thinker 与 Talker 交错执行，TTFA 降低 2.7×
  • fp16 Scale/Zero-Point 存储：权重量化的 scale/zero-point 支持 fp16 存储
  • KV Cache onClone 重构：通过 onClone 统一各后端 KV Cache 复用逻辑
  • LLM log 保存到 context：支持将 LLM 推理日志保存到 context
  • linear_attention 视为 full attention：性能优化路径

模型导出 / 模型支持

  • Gemma4 模型导出：支持 Gemma4 多模态导出
  • Stable Diffusion 3.5 Medium：新增 SD 3.5 Medium 模型支持
  • Wan2.1-T2V-1.3B 视频模型：新增视频生成模型的导出与推理

GPU 后端

  • Vulkan VulkanConv1x1CoopA8：新增 W8A8 动态量化 + Cooperative Matrix 卷积
  • Vulkan LinearAttention 无 subgroup 回退：兼容不支持 subgroup 的设备
  • Vulkan gemv decode 无 subgroup 回退：decode 路径的兼容性增强
  • Vulkan 持久状态跨响应复用：支持持久状态在多次响应间复用
  • Vulkan LayerNorm onClone：支持 LayerNorm onClone，opt pipeline 构造时初始化
  • OpenCL GEMM/GEMV 启发式调度：量化 GEMM/GEMV 启发式 + XGEMM FP local size 设置
  • OpenCL int4 GEMM/GEMV 优化：使用 image1d_buffer_t 优化 int4 GEMM/GEMV，免除 dequant 时额外的 -8
  • OpenCL 原生 TopKV2：OpenCL IMAGE 与 Vulkan（image/buffer）新增原生 TopKV2 实现
  • OpenCL LinearAttention chunk 分支优化：优化 chunk 分支性能
  • OpenCL/Metal/Vulkan 2/3-bit 权重量化：GPU 三后端统一支持 2/3-bit 权重量化（CPU 端通过 low-bit ARM 量化内核支持）
  • Metal Conv1x1 分支选择优化：MetalConvolution1x1 分支路径优化
  • CUDA Blackwell (sm_120) 支持：新增 sm_120 计算架构支持
  • CUDA Eagle 推测解码优化：CUDA Eagle 推测解码性能优化
  • QNN Qwen3.5 支持：QNN 后端支持 Qwen3.5 及视觉模型修复
  • QNN 离线图多进程并行导出：使用多进程并行加速离线图导出
  • QNN Binary 算子优化：加速视觉模型的 binary op 路径

CPU 后端

  • RVV Int8 加速：新增 RVV 加速的 Int8 函数
  • RVV 通用 opt 函数：新增 RVV 向量化的 MNNNorm、MNNGelu、MNNSiLu、MNNExpC8 等
  • ARM v8.2/v8.6 Decode 优化：手机端 LLM decode 性能显著提升
  • LinearAttention prefill fused kernel：新增 MNNFusedGatedDelta fused kernel
  • LinearAttention L=1 decode 快速路径：新增 decode 快速路径
  • Low-bit ARM 量化内核：CPU 新增 2/3-bit 等低比特 ARM 量化内核支持
  • fp16 alpha lowMemory 优化：跳过 fp32 round-trip
  • int4 reorder 性能优化：减少非 fast path 的 int4 reorder 开销

构建 / 测试 / 工具

  • MNN 发布工作流升级：新增 LLM/OMNI、Harmony、Android 16K 页大小支持
  • test_ci.sh：新增统一 CI 入口脚本
  • CMake 生成 podspec：从 CMake 自动生成 podspec
  • GEMM 性能基准：新增 FP32/INT8/Block32-INT4 GEMM 基准
  • GEMV 带宽基准：新增 CPU/Metal w8/w4/w3/w2 GEMV 带宽基准

应用

  • MNNChat Android 0.8.3 模型市场：为 Android 用户准备 0.8.3 版本模型目录

🐛 缺陷修复

LLM / VLM

  • Qwen3.5 head_dim 自动检测：修复 gated attention 模型的 head_dim 自动检测
  • LinearAttention prefix disk cache：修复 LinearAttention prefix disk cache 精度及混合模型支持问题
  • prefix kvcache 损坏崩溃：修复 prefix kvcache 损坏导致的崩溃以及 expandKVCacheInDisk 中的 size_t 截断
  • 推测解码 cache mismatch：修复推测解码中的 cache mismatch 与 mrope 支持
  • vision encoder 内存泄露：修复 vision encoder 内存泄露与延迟增长问题
  • llm_demo linear-attention 状态重置：在 llm_demo 中重置 linear-attention 状态
  • kvmeta 内存泄露：修复 kvmeta 内存泄露
  • stop_ids fallback：修复 list 形式 eos_token_id 的 stop_ids fallback
  • GPTQ weight_dict 重初始化：修复 GPTQ loader 中的 weight_dict 重初始化问题
  • GTE embedding/reranker RoPE：恢复 GTE embedding/reranker 导出中清零的 RoPE inv_freq buffer
  • Gemma3 dual-RoPE 导出：修复 Gemma3 dual-RoPE 导出与 Qwen3-VL bilinear 采样
  • Gemma4 导出：修复 HF transformers>=5.5.4 纯 KV sharing 导致的 gemma4 导出崩溃；通过 mapper 隔离 k_eq_v 与 head_dim 自动检测
  • Hymt2 qk norm 导出：修复 Hymt2 qk norm 导出处理
  • Qwen2.5-Omni 音频预处理：修复 Qwen2.5-Omni 音频预处理的边界情况
  • Qwen3-VL DeepStack dtype mismatch：修复 Qwen3-VL DeepStack dtype 不一致问题
  • Qwen3-VL 重复输出：通过 per-RTM KVCACHE_INFO meta 修复 Qwen3-VL 重复输出问题
  • _maskQK gapLen：将 gapLen 计算移动到 null mask early return 之后

GPU 后端

  • Vulkan attention 及其他算子修复
  • Vulkan 静态符号查找改为动态符号
  • Metal output reuse offset 错误：修复 Metal 输出复用 offset 错误
  • Metal API 兼容性保护：新增 Metal API 兼容性保护
  • OpenCL Attention 越界访问：修复 Attention 算子在内存重分配前的越界访问
  • OpenCL LinearAttention GQA bug：修复 OpenCL LinearAttention 的 GQA bug
  • CUDA Softmax echo / FP16FP32Mix：修复 FP16FP32Mix 模式下 Softmax 回声伪影与 MatMul precisionType 问题
  • CUDA Windows / MNN_CUDA_NATIVE_ARCH 编译：修复 CUDA 在 Windows 与 MNN_CUDA_NATIVE_ARCH 下的 CMake 构建失败
  • QNN MSVC 编译：修复 QNN 后端及 MNN2QNNModel 在 Windows MSVC 下的编译错误

CPU 后端

  • VNNI w4 < 64 计算错误：修复 VNNI w4 < 64 的计算错误
  • KleidiAI int4 dequant：升级 KleidiAI 到 v1.16.0，修复 block-quantized int4 模型的反量化错误
  • KleidiAI 路径映射：修复将 MNN 作为子项目时的 KleidiAI 路径映射
  • KleidiAI depthwise SME2 guard：为 KleidiAI depthwise conv 路径添加 SME2 guard
  • RVV pack/unpack 错误：修复 RVV pack/unpack 函数错误
  • CPU sdot fallback：避免在 Memory_Low 外强制 fp32 fallback（CPU 无 sdot 时）
  • cstdint 包含缺失：补全 cstdint 头文件
  • cpufreq 扫描错误日志：修复扫描 cpufreq policy 目录时的错误日志

算子 / Converter

  • Unique 算子：修复 Unique 算子使输出符合 ONNX 排序规范；修复 index 输出问题
  • TFLite binary activation：修复 TFLite 转换器中 binary 激活的转换
  • ONNX Resize / Einsum：修复 ONNX Resize 转换器回归与 Einsum 4D 非对齐 batch 维快速路径
  • cumsum int 类型不一致：修复 cumsum int 类型不一致与 virtual type command 错误

Diffusion

  • SD1.5 resize cache warmup：修复 SD1.5 resize cache warmup
  • tokenizer.hpp cstdint：修复 tokenizer.hpp 中 uint8_t 未声明错误
  • diffusion llm link sep-build：修复 sep-build 模式下的 llm 链接问题

编译 / Embedding / 其它

  • embedding_demo 崩溃：修复 mMeta->add==0 导致的 embedding_demo 崩溃
  • PyMNN llm 子模块：使 PyMNN 中的 llm 子模块可选导入
  • Windows PyMNN .pdb 膨胀：移除 Windows 上的 /Zi 和 /DEBUG 标志，避免 .pdb 膨胀 wheel
  • dflash Windows MSVC 编译：修复 dflash 在 Windows MSVC 下的编译错误
  • 通用编译问题修复：修复 commit f40b3d2a7 引入的若干编译问题

📚 其他改进

  • AGENTS.md：将 CLAUDE.md 重命名为 AGENTS.md 并通过 include 转发
  • VirtualFaceCapture-MNN 加入 demo 列表
  • MNN_CUDA_NATIVE_ARCH 文档：补充 MNN_CUDA_NATIVE_ARCH cmake 选项的文档
  • fp16 scale schema 同步
  • 代码清理：删除 MNN_CPU_DEQUANT_GEMM：移除已废弃的 MNN_CPU_DEQUANT_GEMM 相关代码
  • 代码清理：删除 WEIGHT_DEQUANT_GEMM 宏：移除废弃宏，精简代码

📊 性能基准

combined_bench

测试环境: Mac mini M4 (16GB) / OnePlus PJZ110 (Snapdragon 8 Elite) | 模型: Qwen3-0.6B / Qwen3-4B | 量化: W4 (block=64)

🙏 致谢

衷心感谢所有贡献者对本版本的宝贵贡献（按 GitHub login 列出）：

  • @jxt1234 — ARMv8.2/v8.6 decode 优化 (#4389)、Metal Conv1x1 分支选择优化 (#4520)、Metal output reuse offset 修复 (#4365)、VNNI w4 < 64 修复 (#4366)、int4 reorder 优化 (#4511)、Geometry cumsum virtual type 修复 (#4391)
  • @Fatemanx (xly) — Wan2.1-T2V-1.3B 视频模型 (#4469)、CUDA Blackwell sm_120 (#4467)、Qwen2.5-Omni 音频边界修复 (#4459)、Qwen3-VL DeepStack dtype 修复 (#4504)、Hymt2 qk norm 导出 (#4490)、SD1.5 resize cache warmup (#4530)、PyMNN llm 可选导入 (#4468)、Converter Einsum 4D fast-path (#4466)
  • @EricMoin — Omni Thinker/Talker 交错生成，TTFA ↓2.7× (#4397)
  • @luckysmg — LinearAttention L=1 decode fast path (#4356)、MNNFusedGatedDelta prefill fused kernel (#4434)、LinearAttention prefix disk cache 精度修复 (#4495)、prefix kvcache 崩溃 / expandKVCacheInDisk size_t 修复 (#4413)
  • @hebangwen — vision encoder 内存泄露与延迟增长修复 (#4441)、llm_demo linear-attention 状态重置 (#4424)
  • @LudovicoYIN (YinHanke) — ONNX Resize / Einsum lowering 修复 (#4254)
  • @cgisky1980 (顾真牛) — CUDA Softmax echo / FP16FP32Mix 修复 (#4476)、CUDA Windows / MNN_CUDA_NATIVE_ARCH 构建修复 (#4477)
  • @yydhYYDH (YYDH) — GPTQ loader weight_dict 重初始化修复 (#4411)、QNN 离线图多进程并行导出 (#4414)
  • @HenryDen — KleidiAI v1.16.0 升级修复 int4 dequant (#4474)、KleidiAI 子项目路径映射修复 (#4542)
  • @luke-opus (liushengtong) — RVV 向量化 MNNNorm/MNNGelu (#4508)、RVV 向量化 MNNSiLu/MNNExpC8 (#4484)
  • @Sherlockzhangjinge — RVV 加速 Int8 函数 (#4359)
  • @typer-J (XJJ) — RVV 通用 opt 函数优化 (#4433)
  • @ihb2032 — RVV pack/unpack 函数错误修复 (#4531)
  • @Juude (Jinde.Song) — MNNChat Android 0.8.3 模型市场准备 (#4405)
  • @devin-lai (Devin Lai) — Unique index 输出修复 (#4529)、TFLite binary activation 修复 (#4540)
  • @EduMenges (Eduardo Menges Mattje) — Vulkan 静态符号查找改为动态 (#4348)
  • @DavidVentura (David Ventura) — CPU sdot fallback 修复 (#4534)
  • @katolikov (Artyom Katolikov) — test_ci.sh 与 Vulkan/OpenCL 算子修复 (#4422)
  • @zlaazlaa (zlaa) — Stable Diffusion 3.5 Medium 支持 (#4072)
  • @Voine — VirtualFaceCapture-MNN 加入 demo 列表 (#4497)
  • @Dugtx — Diffusion sep-build 模式 llm 链接修复 (#4438)
  • @Saisimon — cstdint 头文件修复 (#4338)
  • @Dts0 (dream zhao) — tokenizer.hpp cstdint 修复 (#4373)
  • @tumuyan — cpufreq policy 错误日志修复 (#4400)

MNN 3.6.0 Release Notes

Release Date: June 2026

📦 Install: pip install mnn==3.6.0 ｜ PyPI

📌 Overview

MNN 3.6.0 focuses on three core themes: Multimodal LLM Model Expansion & Inference Acceleration, Deep Multi-Backend GPU Optimization, and Low-bit Quantization & Hardware Adaptation Enhancement:

  • Multimodal LLM Model Expansion & Inference Acceleration: Added support for Gemma4 (text/vision/audio), Qwen3.5, Qwen3-VL DeepStack, and Hymt2 models; introduced dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×; new fp16 storage for weight-quant scale/zero-point; refactored KV Cache reuse via onClone.
  • Deep Multi-Backend GPU Optimization: Vulkan adds VulkanConv1x1CoopA8 (W8A8 coopMat) and no-subgroup fallback paths; OpenCL adds heuristic GEMM/GEMV scheduling, int4 image1d_buffer_t optimization, native TopKV2 and 2/3-bit weight quantization; Metal optimizes Conv1x1 branch selection; CUDA supports Blackwell (sm_120) and optimizes Eagle speculative decoding; QNN supports Qwen3.5 and multi-process parallel offline graph export.
  • Low-bit Quantization & Hardware Adaptation Enhancement: Full-stack 2/3-bit weight quantization across CPU and GPU (ARM low-bit quant kernels + OpenCL/Metal/Vulkan); deep optimization of ARMv8.2/v8.6 decode path; KleidiAI upgraded to v1.16.0; new RVV-vectorized Int8/Norm/Gelu/SiLu/ExpC8 kernels; new fused MNNFusedGatedDelta (LinearAttention prefill); new LinearAttention L=1 decode fast path.

🚀 Highlights

  • Gemma4 Multimodal Support: Full inference support for Gemma4 text, vision, and audio
  • Wan2.1-T2V-1.3B Video Generation: Export and inference support for Wan2.1-T2V-1.3B text-to-video model
  • LLM Inference Acceleration: New dflash speculative decoding; Omni Thinker/Talker interleaved generation reduces TTFA by 2.7×
  • Full-Stack 2/3-bit Weight Quantization: 2/3-bit weight quantization across CPU (low-bit ARM kernels) and GPU (OpenCL/Metal/Vulkan)
  • Vulkan W8A8 coopMat: New VulkanConv1x1CoopA8 with dynamic-quant W8A8 + Cooperative Matrix

✨ New Features

LLM / VLM

  • Gemma4 Model Support: Full text + vision + audio multimodal inference
  • dflash Speculative Decoding: New dflash speculative decoding
  • Omni Interleaved Thinker/Talker: TTFA reduced by 2.7×
  • fp16 Scale/Zero-Point Storage: weight-quant scale/zero-point can be stored in fp16
  • KV Cache onClone Refactor: Unified KV Cache reuse via onClone across backends
  • LLM log save to context: Persist LLM inference logs to context
  • Treat linear_attention as full attention: Performance optimization path

Model Export / Model Support

  • Gemma4 Export: Multimodal export support
  • Stable Diffusion 3.5 Medium
  • Wan2.1-T2V-1.3B: Video generation model export and inference

GPU Backends

  • Vulkan VulkanConv1x1CoopA8: Dynamic-quant W8A8 + Cooperative Matrix conv
  • Vulkan LinearAttention no-subgroup fallback
  • Vulkan gemv decode no-subgroup fallback
  • Vulkan persistent states across responses
  • Vulkan LayerNorm onClone: Move opt pipeline creation into constructor
  • OpenCL Heuristic GEMM/GEMV Scheduling: Heuristic guidance for quantized GEMM/GEMV + XGEMM FP local size
  • OpenCL int4 GEMM/GEMV Optimization: image1d_buffer_t-based int4 GEMM/GEMV, removing the extra -8 during dequant
  • OpenCL Native TopKV2: For OpenCL IMAGE and Vulkan (image/buffer)
  • OpenCL LinearAttention Chunk Branch Optimization
  • OpenCL/Metal/Vulkan 2/3-bit Weight Quantization: Unified across GPU backends (CPU supported via low-bit ARM quant kernels)
  • Metal Conv1x1 Branch Selection Optimization
  • CUDA Blackwell (sm_120) Support
  • CUDA Eagle Speculative Decoding Optimization
  • QNN Qwen3.5 Support with abnormal quant params and vision model fixes
  • QNN Multi-Process Parallel Offline Graph Export
  • QNN Binary Op Optimization: Speeds up vision models

CPU Backend

  • RVV Int8 Acceleration
  • RVV General Opt Functions: MNNNorm, MNNGelu, MNNSiLu, MNNExpC8
  • ARM v8.2/v8.6 Decode Optimization
  • LinearAttention prefill fused kernel: MNNFusedGatedDelta
  • LinearAttention L=1 Decode Fast Path
  • Low-bit ARM Quant Kernels: CPU 2/3-bit and other low-bit ARM quant kernel support
  • fp16 alpha lowMemory Optimization: Skip fp32 round-trip
  • int4 Reorder Optimization: Reduce non-fast-path overhead

Build / Testing / Tools

  • Release Workflow Upgrade: LLM/OMNI, Harmony, Android 16K page size
  • test_ci.sh: Unified CI entry script
  • CMake-generated podspec
  • GEMM Benchmarks: FP32 / INT8 / Block32-INT4
  • GEMV Bandwidth Benchmarks: CPU/Metal w8/w4/w3/w2

Apps

  • MNNChat Android 0.8.3 Model Catalog Preparation

🐛 Bug Fixes

LLM / VLM

  • Qwen3.5 head_dim Auto-Detection: Fixed head_dim auto-detection for gated attention models
  • LinearAttention prefix disk cache: Fixed accuracy and hybrid model support
  • Crash on corrupted prefix kvcache: Fixed crash on corrupted prefix kvcache and size_t narrowing in expandKVCacheInDisk
  • Speculative decoding cache mismatch: Fixed cache mismatch and mrope support
  • Vision encoder memory leak and latency growth: Fixed
  • llm_demo linear-attention state reset: Reset linear-attention states in llm_demo
  • kvmeta memory leak: Fixed
  • stop_ids fallback: Fixed list-form eos_token_id stop_ids fallback
  • GPTQ weight_dict reinitialization: Fixed
  • GTE embedding/reranker RoPE: Restored zeroed RoPE inv_freq buffer
  • Gemma3 dual-RoPE export & Qwen3-VL bilinear sampling: Fixed
  • Gemma4 export: Fixed crash with HF transformers>=5.5.4 pure KV sharing; isolated k_eq_v and head_dim auto-detection via mapper
  • Hymt2 qk norm export: Fixed
  • Qwen2.5-Omni audio preprocessing: Fixed edge cases
  • Qwen3-VL DeepStack dtype mismatch: Fixed
  • Qwen3-VL repeating output: Fixed via per-RTM KVCACHE_INFO meta
  • _maskQK gapLen: Moved gapLen computation after null mask early return

GPU Backends

  • Vulkan attention and other op fixes
  • Vulkan dynamic symbol lookup
  • Metal output reuse offset error: Fixed
  • Metal API compatibility protection: Added
  • OpenCL Attention OOB access: Fixed OOB access before memory reallocation
  • OpenCL LinearAttention GQA bug: Fixed
  • CUDA Softmax echo / FP16FP32Mix: Fixed echo artifacts and MatMul precisionType in FP16FP32Mix mode
  • CUDA Windows / MNN_CUDA_NATIVE_ARCH Build: Fixed CMake build failures
  • QNN MSVC Build: Fixed QNN backend and MNN2QNNModel MSVC compile errors

CPU Backend

  • VNNI w4 < 64 Compute Error: Fixed
  • KleidiAI int4 dequant: Upgraded to v1.16.0 to fix incorrect block-quantized int4 dequant
  • KleidiAI Path Mapping: Fixed when using MNN as a subproject
  • KleidiAI depthwise SME2 guard: Added
  • RVV pack/unpack Errors: Fixed
  • CPU sdot Fallback: Don't force fp32 fallback outside Memory_Low on CPUs without sdot
  • Missing cstdint Include: Added
  • cpufreq Scan Error Log: Fixed spurious error log when scanning cpufreq policy directories

Ops / Converter

  • Unique Op: Fixed to produce sorted output per ONNX spec; fixed index outputs
  • TFLite binary activation conversion: Fixed
  • ONNX Resize / Einsum: Fixed ONNX Resize converter regressions and Einsum 4D fast-path for non-aligned batch dims
  • cumsum int Type Mismatch: Fixed cumsum int type mismatch and virtual type command bug

Diffusion

  • SD1.5 resize cache warmup: Fixed
  • tokenizer.hpp cstdint: Added include to fix uint8_t undeclared
  • Diffusion llm link in sep-build mode: Fixed

Build / Embedding / Misc

  • embedding_demo crash: Fixed crash caused by mMeta->add==0
  • PyMNN llm submodule: Made llm submodule import optional
  • Windows PyMNN .pdb bloat: Removed /Zi and /DEBUG flags on Windows
  • dflash Windows MSVC Build: Fixed dflash Windows MSVC compile error
  • General compilation fixes: Fixed compilation issues introduced by commit f40b3d2a7

📚 Other Improvements

  • AGENTS.md: Renamed CLAUDE.md to AGENTS.md and forwarded via include
  • VirtualFaceCapture-MNN added to demo list
  • MNN_CUDA_NATIVE_ARCH Documentation
  • fp16 scale schema sync
  • Code cleanup: removed MNN_CPU_DEQUANT_GEMM: Deleted deprecated MNN_CPU_DEQUANT_GEMM code
  • Code cleanup: removed WEIGHT_DEQUANT_GEMM macro: Removed deprecated macro

🙏 Acknowledgements

We sincerely thank all contributors for their valuable contributions to this release (listed by GitHub login):

  • @jxt1234 — ARMv8.2/v8.6 decode optimization (#4389), Metal Conv1x1 branch selection (#4520), Metal output reuse offset fix (#4365), VNNI w4 < 64 fix (#4366), int4 reorder optimization (#4511), Geometry cumsum virtual type fix (#4391)
  • @Fatemanx (xly) — Wan2.1-T2V-1.3B video model (#4469), CUDA Blackwell sm_120 (#4467), Qwen2.5-Omni audio edge cases (#4459), Qwen3-VL DeepStack dtype fix (#4504), Hymt2 qk norm export (#4490), SD1.5 resize cache warmup (#4530), PyMNN llm optional import (#4468), Converter Einsum 4D fast-path (#4466)
  • @EricMoin — Omni Thinker/Talker interleaved generation, TTFA ↓2.7× (#4397)
  • @luckysmg — LinearAttention L=1 decode fast path (#4356), MNNFusedGatedDelta prefill fused kernel (#4434), LinearAttention prefix disk cache accuracy fix (#4495), prefix kvcache crash / expandKVCacheInDisk size_t fix (#4413)
  • @hebangwen — Vision encoder memory leak / latency growth fix (#4441), llm_demo linear-attention state reset (#4424)
  • @LudovicoYIN (YinHanke) — ONNX Resize / Einsum lowering fix (#4254)
  • @cgisky1980 (顾真牛) — CUDA Softmax echo / FP16FP32Mix fix (#4476), CUDA Windows / MNN_CUDA_NATIVE_ARCH build fix (#4477)
  • @yydhYYDH (YYDH) — GPTQ loader weight_dict reinit fix (#4411), QNN multi-process parallel offline graph export (#4414)
  • @HenryDen — KleidiAI v1.16.0 upgrade fixing int4 dequant (#4474), KleidiAI subproject path mapping fix (#4542)
  • @luke-opus (liushengtong) — RVV vectorized MNNNorm/MNNGelu (#4508), RVV vectorized MNNSiLu/MNNExpC8 (#4484)
  • @Sherlockzhangjinge — RVV accelerated Int8 functions (#4359)
  • @typer-J (XJJ) — RVV common opt functions optimization (#4433)
  • @ihb2032 — RVV pack/unpack functions fix (#4531)
  • @Juude (Jinde.Song) — MNNChat Android 0.8.3 model catalog preparation (#4405)
  • @devin-lai (Devin Lai) — Unique index outputs fix (#4529), TFLite binary activation fix (#4540)
  • @EduMenges (Eduardo Menges Mattje) — Vulkan static → dynamic symbol lookup (#4348)
  • @DavidVentura (David Ventura) — CPU sdot fallback fix (#4534)
  • @katolikov (Artyom Katolikov) — test_ci.sh and Vulkan/OpenCL op fixes (#4422)
  • @zlaazlaa (zlaa) — Stable Diffusion 3.5 Medium support (#4072)
  • @Voine — Added VirtualFaceCapture-MNN to demo list (#4497)
  • @Dugtx — Diffusion sep-build mode llm link fix (#4438)
  • @Saisimon — cstdint header fix (#4338)
  • @Dts0 (dream zhao) — tokenizer.hpp cstdint fix (#4373)
  • @tumuyan — cpufreq policy error log fix (#4400)

Full Changelog: 3.5.0...3.6.0

Contributors

Juude, tumuyan, and 22 other contributors
Assets 8
Loading
3 Join discussion