Joey Grasty
0bec3bb3f0
ep3 uploaded to the team Drive folder and linked as the current production checkpoint. ep2's link is kept in place per request, labeled as the previous production checkpoint rather than removed.
91 lines
3.5 KiB
Markdown
91 lines
3.5 KiB
Markdown
# Voice Bench
|
|
|
|
A browser tool for exploring the Vox Day / Christopher Nuttall SFT packages
|
|
and generating scenes from them against the joint CPT+SFT model, including a
|
|
STYLE-swap control for testing voice conditioning across authors.
|
|
|
|
Full training report (CPT + SFT setup, results, and evaluation data):
|
|
https://claude.ai/code/artifact/d3415f56-3f9b-40de-aa70-ac0992549ced
|
|
|
|
## What's here
|
|
|
|
- `voice-bench.html` — the tool itself. It's a single self-contained file
|
|
with no install step: open it in a browser (double-click it, or
|
|
File > Open). It needs no server of its own — it just needs a model server
|
|
to talk to (see below).
|
|
- `start-model.sh` — starts the model server (`llama-server`, from
|
|
[llama.cpp](https://github.com/ggml-org/llama.cpp)) that `voice-bench.html`
|
|
sends generation requests to.
|
|
|
|
The 208 training/holdout packages themselves are already embedded inside
|
|
`voice-bench.html` — nothing else needs downloading for browsing.
|
|
|
|
## One-time setup (per machine)
|
|
|
|
### 1. Get the model file
|
|
|
|
Download `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf` (~31GB) from
|
|
Google Drive — **this is the current production checkpoint**, switched from
|
|
ep2 on 2026-08-30 based on separability and length-calibration evidence
|
|
(see `JOINT_TRAINING_CLOSEOUT_REPORT.md`):
|
|
|
|
https://drive.google.com/open?id=1QZT488Uu1b9Pxoie9Hs2lnxtOhMD4Pwr
|
|
|
|
The previous production checkpoint, `Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep2-Q8_0.gguf`,
|
|
remains available for comparison:
|
|
|
|
https://drive.google.com/open?id=1q1w_XMp6oRmfx8xqXDV-HvDx_krVxQtD
|
|
|
|
Memory needed: ~31GB for the weights plus ~3GB of KV cache at the full
|
|
32768-token context, so ~35GB total. Confirmed working on a DGX Spark;
|
|
should comfortably fit a 48GB L40 as well. If a machine has less than that,
|
|
lower `CTX_SIZE` (below) before anything else.
|
|
|
|
### 2. Build llama.cpp with CUDA support
|
|
|
|
Needs an NVIDIA GPU with the CUDA toolkit and `cmake` already installed.
|
|
|
|
```
|
|
git clone https://github.com/ggml-org/llama.cpp
|
|
cd llama.cpp
|
|
cmake -B build -DGGML_CUDA=ON
|
|
cmake --build build --config Release -j
|
|
```
|
|
|
|
This produces `llama.cpp/build/bin/llama-server`, which is what
|
|
`start-model.sh` runs.
|
|
|
|
### 3. Start the model server
|
|
|
|
```
|
|
MODEL_PATH=/path/to/Qwen3-30B-A3B-VoxDay-Nuttall-SFT-ep3-Q8_0.gguf \
|
|
LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server \
|
|
./start-model.sh
|
|
```
|
|
|
|
Leave this running in a terminal — it's the process `voice-bench.html`
|
|
talks to. If you'd rather not set environment variables every time, edit the
|
|
defaults at the top of `start-model.sh` instead.
|
|
|
|
### 4. Open voice-bench.html
|
|
|
|
Open the file in a browser. The "Server" field in the top-right corner
|
|
defaults to `http://127.0.0.1:8200`, which is correct if the model server is
|
|
running on the same machine as the browser. If it's running on a different
|
|
machine on your network, change it to that machine's address instead, e.g.
|
|
`http://192.168.1.50:8200` — and make sure that machine's firewall allows
|
|
inbound connections on the port (`sudo ufw allow from <your-subnet> to any
|
|
port 8200 proto tcp` on Ubuntu).
|
|
|
|
## Troubleshooting
|
|
|
|
- **"SERVER UNREACHABLE"** in the page header — the model server isn't
|
|
running yet, is still loading the model (can take a minute for a 31GB
|
|
file), or the Server field points at the wrong address/port.
|
|
- **Out of memory when starting the server** — re-run with a smaller
|
|
context, e.g. `CTX_SIZE=8192 ./start-model.sh`.
|
|
- **Opening the HTML file from a typed path does something odd in
|
|
Firefox** — use File > Open (Ctrl+O) instead of typing the path into the
|
|
address bar, or paste the full `file:///home/you/path/voice-bench.html`
|
|
URL.
|