Fix ABI fragility: compile bindings against the real llama.h instead of hand-copied ctypes structs

llama_capi.py previously hand-copied struct field order/types from one
specific llama.cpp commit's llama.h into Python ctypes Structures. A
different llama.cpp build could silently reorder or resize those fields and
corrupt memory rather than raising any error.

Replaced with a cffi "API mode" extension (build_capi.py) that #includes the
user's actual llama.h and links against their actual libllama.so. Struct
layout now comes from real compilation - a genuinely incompatible field
fails the build loudly instead of corrupting memory at runtime. Verified
byte-for-byte identical generation output against the prior ctypes
implementation at a fixed seed, plus the fork/peek logit round-trip check
(0.000000 max diff).

Requires a one-time `python build_capi.py` setup step (needs a C compiler,
which building llama.cpp itself already requires).
This commit is contained in:
2026-07-11 21:21:29 -05:00
parent e25ee046a3
commit 8c844c08e5
7 changed files with 232 additions and 187 deletions

View File

@@ -135,7 +135,7 @@ class EntropySampler:
def _logits_row(self, i):
ptr = C.lib.llama_get_logits_ith(self.ctx, i)
return np.ctypeslib.as_array(ptr, shape=(self.n_vocab,)).copy()
return C.as_float_array(ptr, self.n_vocab)
def prime(self, prompt):
"""Decode the prompt on seq 0. Returns base logits for the first step."""