Joey Grasty
ef5792256c
Both Qwen3-30B-A3B-VoxDay-Nuttall-SFT-Scaleup-ep2/ep3 GGUFs are now uploaded to the same Byron_AI_LLMs Drive folder as the original ep2/ep3 checkpoints, for live-testing comparison between the two SFT rounds. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
117 lines
5.1 KiB
Markdown
117 lines
5.1 KiB
Markdown
# Voice Bench
|
|
|
|
A browser tool for exploring the Vox Day / Christopher Nuttall SFT packages
|
|
and generating scenes from them against the joint CPT+SFT model, including a
|
|
STYLE-swap control for testing voice conditioning across authors.
|
|
|
|
Full training report (CPT + SFT setup, results, and evaluation data):
|
|
https://claude.ai/code/artifact/d3415f56-3f9b-40de-aa70-ac0992549ced
|
|
|
|
## What's here
|
|
|
|
- `voice-bench.html` — the tool itself. It's a single self-contained file
|
|
with no install step: open it in a browser (double-click it, or
|
|
File > Open). It needs no server of its own — it just needs a model server
|
|
to talk to (see below).
|
|
- `start-model.sh` — starts the model server (`llama-server`, from
|
|
[llama.cpp](https://github.com/ggml-org/llama.cpp)) that `voice-bench.html`
|
|
sends generation requests to.
|
|
|
|
All 641 training/holdout packages across both SFT rounds are already
|
|
embedded inside `voice-bench.html` — nothing else needs downloading for
|
|
browsing. That's the original 208-package SFT (one book per author: *A Sea
|
|
of Skulls*, *Hour of the Wolf*) plus the 433-package scale-up SFT (four
|
|
more books per author, all held out of both CPT and the original SFT). Use
|
|
the **Book** dropdown to switch between them; each package's card also
|
|
shows which book it came from and whether it was used in training or held
|
|
out.
|
|
|
|
## One-time setup (per machine)
|
|
|
|
### 1. Get the model file
|
|
|
|
Four candidate checkpoints are available, across two SFT rounds on the same
|
|
joint CPT base: the original 208-package SFT, and a newer 433-package
|
|
scale-up SFT trained on eight additional untrained books (see the Book
|
|
dropdown in `voice-bench.html`). Within each round, separability and
|
|
length-calibration metrics lean slightly toward epoch 3, but not decisively
|
|
enough to settle it on metrics alone — **live testing through
|
|
`voice-bench.html` is what should decide between them**, both within a
|
|
round and between the original and scale-up rounds. Download whichever
|
|
you're comparing, run one at a time (or several at once on different
|
|
ports — see step 3).
|
|
|
|
Original SFT:
|
|
- `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1QZT488Uu1b9Pxoie9Hs2lnxtOhMD4Pwr
|
|
- `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1q1w_XMp6oRmfx8xqXDV-HvDx_krVxQtD
|
|
|
|
Scale-up SFT:
|
|
- `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-Scaleup-ep3-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1nreKTn2JnEtQO2ydrCg8uOJbxcuOIQRO
|
|
- `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-Scaleup-ep2-Q8_0.gguf` (~31GB): https://drive.google.com/open?id=1xHeXjLrQyvd_Mey1EvgCz6raeAxJJFzG
|
|
|
|
Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full
|
|
32768-token context, so ~35GB total. Confirmed working on a DGX Spark;
|
|
should comfortably fit a 48GB L40 as well. If a machine has less than that,
|
|
lower `CTX_SIZE` (below) before anything else.
|
|
|
|
### 2. Build llama.cpp with CUDA support
|
|
|
|
Needs an NVIDIA GPU with the CUDA toolkit and `cmake` already installed.
|
|
|
|
```
|
|
git clone https://github.com/ggml-org/llama.cpp
|
|
cd llama.cpp
|
|
cmake -B build -DGGML_CUDA=ON
|
|
cmake --build build --config Release -j
|
|
```
|
|
|
|
This produces `llama.cpp/build/bin/llama-server`, which is what
|
|
`start-model.sh` runs.
|
|
|
|
### 3. Start the model server
|
|
|
|
Point `MODEL_PATH` at whichever checkpoint you're testing:
|
|
|
|
```
|
|
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf \
|
|
LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \
|
|
./start-model.sh
|
|
```
|
|
|
|
Leave this running in a terminal — it's the process `voice-bench.html`
|
|
talks to. If you'd rather not set environment variables every time, edit the
|
|
defaults at the top of `start-model.sh` instead.
|
|
|
|
**To compare epoch 2 and epoch 3 directly**, run both at once on different
|
|
ports (memory permitting — see below):
|
|
|
|
```
|
|
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf PORT=8200 ./start-model.sh
|
|
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf PORT=8201 ./start-model.sh
|
|
```
|
|
|
|
then switch `voice-bench.html`'s "Server" field between `:8200` and `:8201`
|
|
on the same packet to compare their generations side by side.
|
|
|
|
### 4. Open voice-bench.html
|
|
|
|
Open the file in a browser. The "Server" field in the top-right corner
|
|
defaults to `http://127.0.0.1:8200`, which is correct if the model server is
|
|
running on the same machine as the browser. If it's running on a different
|
|
machine on your network, change it to that machine's address instead, e.g.
|
|
`http://192.168.1.50:8200` — and make sure that machine's firewall allows
|
|
inbound connections on the port (`sudo ufw allow from <your-subnet> to any
|
|
port 8200 proto tcp` on Ubuntu).
|
|
|
|
## Troubleshooting
|
|
|
|
- **"SERVER UNREACHABLE"** in the page header — the model server isn't
|
|
running yet, is still loading the model (can take a minute for a 31GB
|
|
file), or the Server field points at the wrong address/port.
|
|
- **Out of memory when starting the server** — re-run with a smaller
|
|
context, e.g. `CTX_SIZE=8192 ./start-model.sh`.
|
|
- **Opening the HTML file from a typed path does something odd in
|
|
Firefox** — use File > Open (Ctrl+O) instead of typing the path into the
|
|
address bar, or paste the full `file:///home/you/path/voice-bench.html`
|
|
URL.
|