Beyond Code

Chapter 3

Context Is the Substrate

Ask a language model to implement a rate limiter, and it will produce a rate limiter. The code will compile. The variable names will be reasonable. The implementation will use a fixed window, because fixed-window rate limiting is the most common pattern in the model’s training data. If the system requires a sliding window — because the traffic is bursty and a fixed window creates a thundering-herd problem at the boundary reset — nothing in the request specified that, and the model defaulted to the training distribution. Ask the same model the same question, but include one paragraph describing the system’s traffic pattern, the SLA requirement, and why the previous rate limiter failed, and it will produce a ...