Trace LLM P95 Latency Spikes in Python with OpenTelemetry + Prometheus

About this lesson

P95 spikes are useless until you can trace them to the slow model, retriever, or tool call that caused them. Walk away with an in-process setup that flags P95 latency and pinpoints the slowest trace segment for fast remediation. Instrument prepare, retrieval, and completion spans using OpenTelemetry and export culprit labels to Prometheus in Python to link alerts to traces. Subscribe for practical AI engineering and LLM systems tutorials from Professor Py. #P95 #observability #OpenTelemetry #Prometheus #LLMs #AIEngineering #Python

Original Description