Nibmeister/voice-bench
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Frame epoch 2 vs epoch 3 as a live-testing decision, not a settled pick

Browse Source 
Metrics lean toward ep3 but aren't decisive enough to call it outright.
README now presents both checkpoints as candidates and walks through
running them side-by-side (different ports) for direct comparison in
voice-bench.html, rather than naming one as current production.
This commit is contained in:
Joey Grasty
2026-08-30 19:23:12 -05:00
parent 0bec3bb3f0
commit 7ac0b36ae6
1 changed files with 22 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
README.md
View File

@@ -24,17 +24,16 @@ The 208 training/holdout packages themselves are already embedded inside
### 1. Get the model file ### 1. Get the model file
Download `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf` (~31GB) from Two candidate checkpoints are available. Separability and length-calibration
Google Drive — **this is the current production checkpoint**, switched from metrics lean slightly toward epoch 3 (see `JOINT_TRAINING_CLOSEOUT_REPORT.md`
ep2 on 2026-08-30 based on separability and length-calibration evidence for the numbers), but not decisively enough to settle it on metrics alone —
(see `JOINT_TRAINING_CLOSEOUT_REPORT.md`): **live testing through `voice-bench.html` is what should decide between
them.** Download both, run one at a time (or both at once on different
ports — see step 3), and compare generations on real held-out packages
before picking one.
https://drive.google.com/open?id=1QZT488Uu1b9Pxoie9Hs2lnxtOhMD4Pwr - `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1QZT488Uu1b9Pxoie9Hs2lnxtOhMD4Pwr
- `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1q1w_XMp6oRmfx8xqXDV-HvDx_krVxQtD
The previous production checkpoint, `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf`,
remains available for comparison:
https://drive.google.com/open?id=1q1w_XMp6oRmfx8xqXDV-HvDx_krVxQtD
Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full
32768-token context, so ~35GB total. Confirmed working on a DGX Spark; 32768-token context, so ~35GB total. Confirmed working on a DGX Spark;
@@ -57,6 +56,8 @@ This produces `llama.cpp/build/bin/llama-server`, which is what
### 3. Start the model server ### 3. Start the model server
Point `MODEL_PATH` at whichever checkpoint you're testing:
``` ```
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf \ MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf \
LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \ LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \
@@ -67,6 +68,17 @@ Leave this running in a terminal — it's the process `voice-bench.html`
talks to. If you'd rather not set environment variables every time, edit the talks to. If you'd rather not set environment variables every time, edit the
defaults at the top of `start-model.sh` instead. defaults at the top of `start-model.sh` instead.
**To compare epoch 2 and epoch 3 directly**, run both at once on different
ports (memory permitting — see below):
```
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf PORT=8200 ./start-model.sh
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf PORT=8201 ./start-model.sh
```
then switch `voice-bench.html`'s "Server" field between `:8200` and `:8201`
on the same packet to compare their generations side by side.
### 4. Open voice-bench.html ### 4. Open voice-bench.html
Open the file in a browser. The "Server" field in the top-right corner Open the file in a browser. The "Server" field in the top-right corner