7 posts tagged observability from Omnissiah Systems.
How temperature, top-p, and other sampling parameters silently drift in production inference systems, and the concrete steps to detect and control them.
How GPU thermal throttling silently degrades inference throughput in dense clusters, how to detect it, and what to do before it becomes a capacity problem.
A practical breakdown of where time disappears during model weight loading at startup, and how to profile and reduce it in production inference systems.
How request routing silently degrades across multi-instance inference deployments, the failure modes to recognize, and how to diagnose and correct them.
How dynamic batching misconfigures itself over time, why throughput numbers lie on aging inference servers, and how to catch batch size drift before it costs you.
Your p50 looks fine and your p99 is on fire. A field guide to the places tail latency hides in a large-model serving stack.