Joey Grasty 7a66942242 Add MoE LoRA converter (Unsloth fused format -> vLLM per-expert format) ...

Converts an Unsloth-trained, MoE-expert-targeting LoRA adapter into
vLLM's expected per-expert format so it can be served live through
vLLM's FusedMoE LoRA support, instead of requiring a full merge-and-
reload per adapter switch.

Includes:
- moe_lora_convert_vllm.py -- the converter
- moe_lora_convert_validate.py -- three-way validation (round-trip
  bit-exactness, per-expert delta match against Unsloth's real merge
  function, key coverage)
- README.md -- usage plus the full findings report: the tensor-layout
  investigation (including a wrong turn worth recording), all five
  vLLM-serving validation gates, and the GB10 infrastructure lessons
  from developing this

Developed and validated against Qwen3-30B-A3B (48 layers, 128 experts,
r=16 LoRA).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015dwQp8JCqhM2MRFHYwMyxp

2026-09-09 09:19:28 -05:00

8.6 KiB

Raw Permalink Blame History

View Raw