Joey Grasty
9889814e37
Self-contained browser tool for exploring the Vox Day / Nuttall SFT packages and generating scenes against the joint CPT+SFT model, plus a portable launch script (start-model.sh) and setup README for running the model server on a fresh machine (DGX Spark or CUDA/L40 box).
3.0 KiB
Voice Bench
A browser tool for exploring the Vox Day / Christopher Nuttall SFT packages and generating scenes from them against the joint CPT+SFT model, including a STYLE-swap control for testing voice conditioning across authors.
What's here
voice-bench.html— the tool itself. It's a single self-contained file with no install step: open it in a browser (double-click it, or File > Open). It needs no server of its own — it just needs a model server to talk to (see below).start-model.sh— starts the model server (llama-server, from llama.cpp) thatvoice-bench.htmlsends generation requests to.
The 208 training/holdout packages themselves are already embedded inside
voice-bench.html — nothing else needs downloading for browsing.
One-time setup (per machine)
1. Get the model file
Download Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf (~31GB) from the
team Google Drive.
Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full
32768-token context, so ~35GB total. Confirmed working on a DGX Spark;
should comfortably fit a 48GB L40 as well. If a machine has less than that,
lower CTX_SIZE (below) before anything else.
2. Build llama.cpp with CUDA support
Needs an NVIDIA GPU with the CUDA toolkit and cmake already installed.
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j
This produces llama.cpp/build/bin/llama-server, which is what
start-model.sh runs.
3. Start the model server
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf \
LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \
./start-model.sh
Leave this running in a terminal — it's the process voice-bench.html
talks to. If you'd rather not set environment variables every time, edit the
defaults at the top of start-model.sh instead.
4. Open voice-bench.html
Open the file in a browser. The "Server" field in the top-right corner
defaults to http://127.0.0.1:8200, which is correct if the model server is
running on the same machine as the browser. If it's running on a different
machine on your network, change it to that machine's address instead, e.g.
http://192.168.1.50:8200 — and make sure that machine's firewall allows
inbound connections on the port (sudo ufw allow from <your-subnet> to any port 8200 proto tcp on Ubuntu).
Troubleshooting
- "SERVER UNREACHABLE" in the page header — the model server isn't running yet, is still loading the model (can take a minute for a 31GB file), or the Server field points at the wrong address/port.
- Out of memory when starting the server — re-run with a smaller
context, e.g.
CTX_SIZE=8192 ./start-model.sh. - Opening the HTML file from a typed path does something odd in
Firefox — use File > Open (Ctrl+O) instead of typing the path into the
address bar, or paste the full
file:///home/you/path/voice-bench.htmlURL.