Add Voice Bench: SFT package browser + model launch script

Self-contained browser tool for exploring the Vox Day / Nuttall SFT
packages and generating scenes against the joint CPT+SFT model, plus a
portable launch script (start-model.sh) and setup README for running
the model server on a fresh machine (DGX Spark or CUDA/L40 box).
This commit is contained in:
2026-08-30 10:01:49 -05:00
commit 9889814e37
3 changed files with 960 additions and 0 deletions

78
README.md Normal file
View File

@@ -0,0 +1,78 @@
# Voice Bench
A browser tool for exploring the Vox Day / Christopher Nuttall SFT packages
and generating scenes from them against the joint CPT+SFT model, including a
STYLE-swap control for testing voice conditioning across authors.
## What's here
- `voice-bench.html` — the tool itself. It's a single self-contained file
with no install step: open it in a browser (double-click it, or
File > Open). It needs no server of its own — it just needs a model server
to talk to (see below).
- `start-model.sh` — starts the model server (`llama-server`, from
[llama.cpp](https://github.com/ggml-org/llama.cpp)) that `voice-bench.html`
sends generation requests to.
The 208 training/holdout packages themselves are already embedded inside
`voice-bench.html` — nothing else needs downloading for browsing.
## One-time setup (per machine)
### 1. Get the model file
Download `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf` (~31GB) from the
team Google Drive.
Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full
32768-token context, so ~35GB total. Confirmed working on a DGX Spark;
should comfortably fit a 48GB L40 as well. If a machine has less than that,
lower `CTX_SIZE` (below) before anything else.
### 2. Build llama.cpp with CUDA support
Needs an NVIDIA GPU with the CUDA toolkit and `cmake` already installed.
```
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j
```
This produces `llama.cpp/build/bin/llama-server`, which is what
`start-model.sh` runs.
### 3. Start the model server
```
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf \
LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \
./start-model.sh
```
Leave this running in a terminal — it's the process `voice-bench.html`
talks to. If you'd rather not set environment variables every time, edit the
defaults at the top of `start-model.sh` instead.
### 4. Open voice-bench.html
Open the file in a browser. The "Server" field in the top-right corner
defaults to `http://127.0.0.1:8200`, which is correct if the model server is
running on the same machine as the browser. If it's running on a different
machine on your network, change it to that machine's address instead, e.g.
`http://192.168.1.50:8200` — and make sure that machine's firewall allows
inbound connections on the port (`sudo ufw allow from <your-subnet> to any
port 8200 proto tcp` on Ubuntu).
## Troubleshooting
- **"SERVER UNREACHABLE"** in the page header — the model server isn't
running yet, is still loading the model (can take a minute for a 31GB
file), or the Server field points at the wrong address/port.
- **Out of memory when starting the server** — re-run with a smaller
context, e.g. `CTX_SIZE=8192 ./start-model.sh`.
- **Opening the HTML file from a typed path does something odd in
Firefox** — use File > Open (Ctrl+O) instead of typing the path into the
address bar, or paste the full `file:///home/you/path/voice-bench.html`
URL.

47
start-model.sh Executable file
View File

@@ -0,0 +1,47 @@
#!/bin/bash
# Starts the llama-server that voice-bench.html talks to.
#
# Requires:
# - llama.cpp built with CUDA support (see README.md)
# - the Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf file (from the team
# Google Drive)
#
# Configure by setting environment variables before running, e.g.:
# MODEL_PATH=/data/models/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf \
# LLAMA_SERVER=/opt/llama.cpp/build/bin/llama-server \
# ./start-model.sh
# or just edit the defaults below.
set -e
MODEL_PATH="${MODEL_PATH:-$HOME/models/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf}"
LLAMA_SERVER="${LLAMA_SERVER:-$HOME/llama.cpp/build/bin/llama-server}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-8200}"
CTX_SIZE="${CTX_SIZE:-32768}"
if [ ! -f "$MODEL_PATH" ]; then
echo "!!! Model not found at: $MODEL_PATH"
echo " Download it from the team Google Drive, then either place it at"
echo " that path or re-run with MODEL_PATH=/path/to/the/file.gguf"
exit 1
fi
if [ ! -x "$LLAMA_SERVER" ]; then
echo "!!! llama-server not found/executable at: $LLAMA_SERVER"
echo " Build llama.cpp first -- see README.md -- or re-run with"
echo " LLAMA_SERVER=/path/to/llama-server"
exit 1
fi
echo "Model: $MODEL_PATH"
echo "Server: $LLAMA_SERVER"
echo "Listening on ${HOST}:${PORT}, context ${CTX_SIZE}"
echo "(if this exits with an out-of-memory error, re-run with a smaller CTX_SIZE, e.g. CTX_SIZE=8192)"
echo
exec "$LLAMA_SERVER" \
--model "$MODEL_PATH" \
--host "$HOST" \
--port "$PORT" \
-ngl 999 \
--ctx-size "$CTX_SIZE" \
--threads 8

835
voice-bench.html Normal file

File diff suppressed because one or more lines are too long