observability

Reading the Pulse: Where Tail Latency Hides in Distributed Inference

Your p50 looks fine and your p99 is on fire. A field guide to the places tail latency hides in a large-model serving stack.