Too Cheap to Meter
A 125-billion-parameter model now runs on the computer under my desk at about 38 tokens a second. What it compares to, where it falls short, and the overnight work I want from it.…
ai-inference
3 posts
Chasing Bit-Equality Through a Wasm Actor Runtime
An investigation into bit-reproducible snapshot/restore for stateful Wasm actors, and the two-tier contract that emerged.…
Zero-Copy GPU Inference from WebAssembly on Apple Silicon
A WebAssembly module's linear memory can be shared directly with the Apple Silicon GPU: no copies, no serialization, no intermediate buffers. Here's how the zero-copy chain works, what we measured, and what it enables for stateful AI inference.…