Open-source Platform for AI Engineering | O11y | Prompts | Vault | Evals | 💻 GitHub: https://xitter.cf/t.co/JfGWfgK4BG 📙 Docs: https://xitter.cf/t.co/PmGC2wyxbs
Joined February 2024
- Tweets184
- Following3
- Followers110
- Likes328
OpenLIT 🔭 retweeted
8 open-source GitHub repos every AI agent developer should know.
from catching hallucinations to debugging tool calls and tracking costs.
these are genuinely useful for building agents that actually work.
here's the stack 🧵👇
OpenLIT 🔭 retweeted
🔭 Bookmark these 10 GitHub repos that show you what your AI agent really did
1. Langfuse (36K stars, MIT core)
github.com/langfuse/langfuse
Traces every LLM call and tool step, with cost, time and scores. You can host it yourself.
2. MLflow (28K stars, Apache 2.0)
github.com/mlflow/mlflow
The well-known ML platform. It now traces LLM apps and agents too, and runs evals.
3. Opik (22K stars, Apache 2.0)
github.com/comet-ml/opik
Comet's tool to trace, score and watch your LLM apps and agents.
4. Phoenix (12K stars, Elastic License 2.0)
github.com/Arize-ai/phoenix
Arize's tracing and eval app, built on OpenTelemetry. Free to host yourself, but not open source.
5. OpenLLMetry (7K stars, Apache 2.0)
github.com/traceloop/openllm…
Adds OpenTelemetry tracing to your LLM app, so traces go to any tool you already use.
6. LangWatch (5K stars, Apache 2.0 core)
github.com/langwatch/langwat…
Traces, evals and agent simulations in one place.
7. Laminar (3K stars, Apache 2.0)
github.com/lmnr-ai/lmnr
Tracing built for agents, with a timeline of every step.
8. OpenLIT (3K stars, Apache 2.0)
github.com/openlit/openlit
One line of code sends your LLM, vector database and GPU data to OpenTelemetry. It comes with its own dashboard.
9. Claude Code History Viewer (2K stars, MIT)
github.com/jhlee0409/claude-…
A desktop app to browse your past Claude Code sessions and their token use.
10. claude-code-log (1K stars, MIT)
github.com/daaain/claude-cod…
Turns your Claude Code transcript files into pages you can read in a browser.
One tip: send your traces in the OpenTelemetry GenAI format. Then you can switch tools later without changing your code.
OpenLIT 🔭 retweeted
Most #LLM #observability guides stop at a single chat completion. But production AI systems are far more complex.
This post is a joint effort by the OpenLIT and VictoriaMetrics teams. @openlit_io brings the #OpenTelemetry-native instrumentation for LLMs and agents, while VictoriaMetrics provides the storage and query layer for the resulting metrics, traces, and logs. We wrote it together to show how the two projects fit into a single, self-hosted observability pipeline, and to share the patterns and queries that worked well for us along the way.
Learn more 👇
hubs.ly/Q04nFzRY0
#Otel #OpenLIT #SDK VictoriaMetrics
OpenLIT 🔭 retweeted
GPU utilization is a vanity metric for LLM inference.
nvidia-smi and DCGM tell you the SM was busy. Not KV cache OOM, throttle, or zombie VRAM. What to SLO instead:
openlit.io/blogs/gpu-util-is…
OpenLIT 🔭 retweeted
★ Just published: OpenLIT @openlit_io — Monitor, debug, and scale LLM applications with ease
Open-source observability platform for GenAI and LLM applications. Real-time monitoring, distributed tracing, prompt management, and AI...
openalternative.co/openlit
OpenLIT 🔭 retweeted
Zero-code #AI observability for LLMs and agents on #Kubernetes:
Deploy OpenLIT Operator once, and it auto-injects #OpenTelemetry instrumentation into your pods — no code changes, no image rebuilds.
grafana.com/blog/ai-observab…
OpenLIT 🔭 retweeted
👀 Monitoring LLMs in prod means answering: what's this costing us, are we within SLO, and are we returning hallucinations?
Grafana Cloud + OpenLIT + #OpenTelemetry gives you one place to answer all three.
grafana.com/blog/ai-observab…
OpenLIT 🔭 retweeted
I wrote about a hard truth in RAG:
👉 Retrieval doesn’t eliminate hallucinations.
Even with good context, models can still give wrong answers.
In this blog:
* Where hallucinations come from
* Why retrieval isn’t enough (Dont trust the model!)
🔗 openlit.io/blogs/how-to-dete…
OpenLIT 🔭 retweeted
Been working with LLM systems lately and one thing stands out:
We track latency, tokens, cost… but barely look at the GPU.
which is where all the work happens.
Most debugging pain I’ve hit comes from this gap.
I wrote this blog to connect the dots:
openlit.io/blogs/gpu-monitor…
We wrote a quick guide on adding observability to LLM apps with OpenTelemetry.
Trace prompts, debug issues, and track latency/cost in minutes 👇
openlit.io/blogs/how-to-add-…
OpenLIT 🔭 retweeted
As always I'm part of an amazing line-up of speakers at @DevoxxUK in May. I'll be talking about AI observability with @openlit_io #opentelemetry #elasticsearch. See you there! 👋
Observability & Flags
If you can't see it, you can't fix it:
🔹 @maeddes & @tiffanyfayj - Hello World, Meet the Spanimals: Observability for Beginners.
🔹 @CarlyLRichmond - Observing AI Applications with OpenLit.
🔹 Simon Schrottner - Feature Flagging Without the Pain.
OpenLIT 🔭 retweeted
Come along to the DevOps Society meetup. There's talks on challenges running K8 clusters in prod, and myself on observing AI applications with @openlit_io OTel signals and @elastic.
meetup.com/devops-society/ev…
See you there! 👋
OpenLIT 🔭 retweeted
We’re live on the @digitalocean Marketplace! 🚀
Deploy @openlit_io in one click and start building your LLM-powered apps and no ops hassle, just innovation.
🔗 marketplace.digitalocean.com…
#OpenLIT #DigitalOcean #AI #LLM #SaaS #DevTools
We’re live on @FazierHQ!
🎉 Our zero-code Agent Observability is now launching on Fazier, go check it out!!
👉 fazier.com/launches/openlit-…
#AI #observability #opensource #Agents
Tired of redeploying just to monitor your AI agents?
Zero-Code Agent Observability launches tomorrow on @FazierHQ & get full visibility into Agentic events!
✅ No code changes
✅ No image changes
✅ No redeploys
Instant visibility for your AI agents
🔗 fazier.com/posts/launches/op…
Hey Everyone, We’re officially live on Product Hunt!
📷Introducing Zero-Code Observability for LLMs and AI Agents.
If you find our product useful, a quick upvote would mean a lot and help us reach more people:
producthunt.com/products/ope…
🚀 Excited to announce we’re launching on Product Hunt
This one makes LLM Observability at scale ridiculously simple. 💡
📅 Mark your calendars: 10th October
#ProductHunt #GenAI #Opensource #Launch #Kubernetes
Managing prompts shouldn’t be chaos.
⚡ OpenLIT Prompt Hub stores, versions & reuses prompts with dynamic vars — perfect for A/B testing or rollbacks.
No more “which prompt is latest?” confusion for teams building chatbots or RAG.
👉 docs.openlit.io/latest/openl…
🚀 Running LLMs on your own GPU?
Monitor memory, temp & utilization with the first OpenTelemetry-based GPU monitoring for LLMs.
⚡ OpenLIT tracks GPU performance automatically — focus on your apps, not hardware.
👉 docs.openlit.io/latest/sdk/q…
#LLMObservability #OpenTelemetry