Founder of @openlit_io | Cyber security using AI @transilienceai https://xitter.cf/t.co/cz5Dr0pbHj

Noida
Joined August 2014
GPU utilization is a vanity metric for LLM inference. nvidia-smi and DCGM tell you the SM was busy. Not KV cache OOM, throttle, or zombie VRAM. What to SLO instead: openlit.io/blogs/gpu-util-is…
1
1
33
Replying to @claudeai
@claudeai I saw this error suddenly while viewing code.claude.com/docs/en/agen… . What happened ?
2
1
64
I wrote about a hard truth in RAG: 👉 Retrieval doesn’t eliminate hallucinations. Even with good context, models can still give wrong answers. In this blog: * Where hallucinations come from * Why retrieval isn’t enough (Dont trust the model!) 🔗 openlit.io/blogs/how-to-dete…
1
2
50
Been working with LLM systems lately and one thing stands out: We track latency, tokens, cost… but barely look at the GPU. which is where all the work happens. Most debugging pain I’ve hit comes from this gap. I wrote this blog to connect the dots: openlit.io/blogs/gpu-monitor…
1
3
47
🚀 Just launched on Product Hunt: Custom Dashboards in @openlit_io 🎉 🔍 Real-time dashboards for AI apps & agents — track latency, tokens, costs & traces. ⚡ 100% open-source & OpenTelemetry-native. 👉 producthunt.com/products/ope… #AI #LLMs #OpenSource #GenAI
1
1
3
103