Kimi-K3 produced long token-0 runs when I deepened the asynchronous PP Decode queue. I traced the failure to receive buffers being reused before the forward stream read them. After fixing tensor lifetime, affected requests fell from 51/192 and 98/192 to 0/192 at depths 2 and 3. PR #47614 merged on 2 November 2026.
Expand the investigation: reproduction, checksum evidence and the fix
1. Turn an apparent speedup into a correctness test. Deeper asynchronous PP queues appeared faster, but many Kimi-K3 requests emitted long runs of token 0. On PP2 × TP4, depth 1 produced no affected requests in the tested run. Raising the depth to 2 or 3 made the fault reproducible. Synchronizing after each launch also suppressed it, which directed the investigation to queued work and tensor lifetime.
2. Measure where the data changed. On the downstream stage, I checked the same proxy tensors just after receive on the communication stream and just before use on the forward stream. Two depth-2 runs had 76/7,498 and 378/7,498 checksum mismatches. Keeping the previous tensor reference alive until the next receive had allocated its buffers gave 0/7,499 mismatches with a similar observed queue depth. Checksum and reference-lifetime controls ↗
3. Extend storage lifetime at the shared launch boundary. CPU scheduling could drop the old reference while a forward-stream read was still queued. The next receive could reuse the same storage and overwrite that microbatch's activations. In _pp_launch_batch, I used record_stream(self.forward_stream) on each non-CPU proxy tensor before run_batch. This tells the allocator to retain storage until forward-stream use completes. Existing stream dependencies still determine execution order.
| Configuration | Async depth | Before | After |
|---|---|---|---|
| PP2 × TP4 | 2 | 51/192 | 0/192 |
| PP2 × TP4 | 3 | 98/192 | 0/192 |
| PP2 × TP4 + DCP4 | 2 | 59/192 | 0/192 |
| PP4 × TP2 | 3 | 6/192 | 0/192 |
| PP4 × TP2 | 4 | 19/192 | 0/192 |
The table reports single-node B300 correctness tests using fake transfer. The PR records the tested patch variants and separate communication-overlap follow-ups. The corrupted-output runs served as failure reproductions.
