A widely deployed open-source inference framework shipped a new major version that changes long-context key-value cache allocation from one contiguous block to paged allocation. For the same model and the same context length, peak memory usage drops, and concurrent requests no longer each reserve worst-case space.
Why it matters now
Paging is not a new idea — databases and operating systems have done it for decades. It is arriving in inference frameworks now because reserving for the worst case was tolerable while contexts were short. At hundreds of thousands of tokens, that same waste decides how many users a single card can serve.
If this is your team
If you self-host inference, this is worth evaluating: it changes concurrency per card, not single-request latency. If you only call a hosted API, the direct impact is close to zero — but it signals where hosted pricing goes next.
示範內容，非真實報導。
某雲端服務商將 API 計價改為分層結構，並公布快取命中的獨立費率
發生了什麼
一家雲端服務商調整了 API 的計價方式，從單一費率改為依用量分層，並且把「命中快取的輸入 token」拆成獨立的費率項目公布。原本混在總量裡的快取部分，現在在帳單上是一行獨立的數字。
A cloud provider moved its API from flat-rate to tiered pricing and published a separate rate for cache-hit input tokens. What was previously folded into the total is now its own line on the bill.
Why it matters now
The story here is not cheaper or costlier — it is predictability. Tiered pricing makes the cost-versus-usage curve something you can draw in advance, and a separate cache rate finally answers "how much are my repeated system prompts actually costing me." Both were previously only inferable after the fact.
If this is your team
If you bill your own customers by usage, your cost curve just changed and your price curve probably has to follow. If you only use it internally, the highest-value move is attributing usage per feature — otherwise tiering still won't tell you which workload pushed you into the next band.
A widely used workflow automation tool added retry settings and a dead-letter queue to its scheduled triggers. Failed runs can now retry a configured number of times, and runs that still fail land in a separate queue instead of disappearing.
Why it matters now
This closes a well-known last mile. The most common failure mode in workflow automation is not a badly written flow — it is a flow that silently did not run, especially on a schedule nobody is watching. A dead-letter queue turns silent failure into a visible list.
If this is your team
If you have scheduled flows in production, spend half an hour configuring retries and the dead-letter queue — then confirm someone will actually look at it. An unread dead-letter queue is functionally identical to not having one.