Moe Khalil (@Mokhalil01) header
Moe Khalil (@Mokhalil01) profile picture

Head of Product @LiteLLM | Ex-Founder @WebhoundAI (YC S23) | Stanford CS ‘23 | San Francisco & Beirut

San Francisco, CA
Joined September 2011
Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)Photo by Moe Khalil (@Mokhalil01)
Moe Khalil retweeted
LiteLLM now supports currencies other than USD for prices and spend. Here is how to use your currency in 3 simple steps: 1 - Set prices in your currency: send pricing_currency: "EUR" with input_cost_per_token and output_cost_per_token on a chat request. LiteLLM converts the prices to USD for spend tracking. 2- Use daily ECB rates: LiteLLM gets the reference rates from the European Central Bank and caches them. You do not set exchange rates yourself. 3- See spend in your currency: on the Usage dashboard, select a currency. The card shows an estimate next to the USD total, with the date of the rate. More details in the PRs: github.com/BerriAI/litellm/p… github.com/BerriAI/litellm/p… github.com/BerriAI/litellm/p…
1
1
15
553
UI is dead. The new UI is MCP. Nobody wants to spend time in your product
I was wrong. Generative UI is not the future. My least favorite part about software is learning a new UI…and if new UI is frustrating then generative UI is just like a weird form of hell for users. I’ve though about this a lot btw, I literally made ArrowJS because I thought gen ui was an exciting future, but I’ve lost faith in it as a generic solution to software on-demand. Heck, even when you vibe code something, the worst part is figuring out the ui to know if it made what you wanted or not. I think generative UI should be tightly contained to “types”, like, on a dashboard where charts are, being able to say “make a chart of DAU over revenue” is perfect, now my expectations are pre-met.
35
3
7
103
7,337
ZDR right now basically comes down to “trust us bro” There has to be a better solution here Is there some way to incorporate zero-knowledge proofs into the system to verifiably have no retention? Is anyone working on this?
7
391
The more I use LLMs the more I’m convinced that this isn’t a winner-take-all market The attributes that make an Astra really good at product work and creative thinking make it write really bloated over-engineered code with bad performance On the other hand, Opus feels like an extremely cracked engineer that knows it’s shit, but is basically useless for any big picture ideation I’ve also seen posts from Anthropic engineers on X where they shared how making their models better at agent-to-agent interaction came at the cost of making them worse at writing These personalities are what make the models unique. I know when to go to Astra, when to go to Opus, when to go to GLM, and so on But it also means I don’t see myself ever trusting one model family to do all my work And I can’t think of any other historical tech revolutions that share this trait. The model is defined by it’s personality and sharp edges, and that taste is so subjective Is anyone else feeling this?
1
5
397
Founders building agents with usage-based pricing: how do you handle cost-tracking, spend limits, and charging users? Is it all custom code? Would you be open to having a gateway handle the complexities here? For context, we're working on a customer portal for people using LiteLLM in prod, and would love to hear from people who are living and breathing these problems
3
4
12
394
A historical timeline of OpenAI/Anthropic based on how much I enjoyed coding with them at the time
3
1
8
308
Quick ramble re: local models If frontier labs keep restricting models and charging per token I could see a world where everyone just runs their models locally. We still need either a breakthrough on model compression (which I’m not sure is even possible, there’s probably a floor to how small you could make a model while maintaining frontier quality) Or somebody will start selling personal computers that are way more memory-capable and efficient at inference. Cerebras seem well positioned to do this but they don’t seem to care right now. This also requires an OSS model that actually competes with the frontier. I know they’re getting close on benchmarks but anyone that’s actually used them knows how big the gap is. Or, you could have Anthropic/OpenAI sell a license to run their models locally, while staying closed source. Kind of like how windows sells their licenses. Either way, it’s hard for me to imagine us continuing to run our inference in someone else’s cloud long-term, under whatever terms they decide are “aligned”. If this is true, it’s an existential threat for non-OSS model companies, and whoever addresses it first could win the whole market. Any thoughts?
9
485
Restrictions need to be way less strict. If you’re already going through KYC you shouldn’t have to still have worse cyber models than bad actors.
Cyber team wants your feedback on Daybreak. Models, codex plugins, blue/red access, documentation, red teaming & defense factory... What can we do better or improve?
6
278
FYI if you’re struggling with this we built the @LiteLLM customer portal for exactly this problem. Would love to chat :)
So many applied AI companies are struggling with negative gross margins because they're essentially reselling tokens from the labs.I wonder if the next generation of successful applied AI companies will look more like pure MCP servers... Private data, proprietary workflows, permissions, privacy, secure actions. No need to build your own UI client or pay for inference! Just charge a monthly SaaS subscription and plug into ChatGPT, Claude, Copilot, Gemini, etc. The aggregators handle the UI, orchestration and token costs. You provide the data and tools that are unique in the market. And bim suddenly, we're back to SaaS-like gross margins!
1
6
573
I’m a huge GLM 5.3 fan for security. When we were dealing with our security incident it was the only model that did what we asked it to without doing something stupid or some dumb guardrails coming up. And that’s including daybreak.
my X feed is full of security researchers treating deepseek like it’s some kind of god. I’ve tested it multiple times, and while ds is good and cheap, I’ve found that it mostly catches low-hanging fruit, which is still important but the amplification of these posts on X has turned it into some kind of master model that supposedly makes everything else unnecessary, security wise i feel like this creates a false sense of security, as if we’ve somehow solved security
1
40
2,832
Working with startups is such a breath of fresh air. With big companies, everything you build for them is an annoyance because it means they have to do more work and can’t leave by 4PM Startups are so much more intense. They make you feel like you’re not moving fast enough I love it
1
12
463
Realizing today is maybe not the best day to share this
Met with a pretty big airline’s AI team recently and were able to work our way to a meeting with their fleet optimization team They told us that even 1% better optimization could increase airline revenue by billions We asked them if they’d tried pointing an agent at the problem They said no, they didn’t think it’d be as capable as their data scientists We’re still so early
1
6
888
Met with a pretty big airline’s AI team recently and were able to work our way to a meeting with their fleet optimization team They told us that even 1% better optimization could increase airline revenue by billions We asked them if they’d tried pointing an agent at the problem They said no, they didn’t think it’d be as capable as their data scientists We’re still so early
2
1
12
1,354
If your AI product is usage based, you’ve probably struggled with setting budgets on your customers, capping them accurately, and load balancing between them The new @LiteLLM Customer Portal handles it all for you Out now, read more below
1
2
12
581
I'm going to try this. Couple ideas here: 1. Sidecar model that does real time WSP 2. Real-time directional ablation Anyone have any guidance here?
Dumb idea: the problem with abliterated models is that by editing all weights that cause refusal, you end up making the model significantly dumber. What if you used real-time weight-space programming to only edit the specific weights that cause a refusal whenever it happens? That way you could get away with only weight-editing at key decision points, and only the weights relevant to that specific decision. Is this something anyone is trying?
3
259
Dumb idea: the problem with abliterated models is that by editing all weights that cause refusal, you end up making the model significantly dumber. What if you used real-time weight-space programming to only edit the specific weights that cause a refusal whenever it happens? That way you could get away with only weight-editing at key decision points, and only the weights relevant to that specific decision. Is this something anyone is trying?
36
3
3
172
23,956
So how is this not just a race to the bottom? What’s the differentiation here?
Sandbox provider updated list and prices ordered from most expensive to cheapest. Again (All prices are for 2 vCPU / 4 GB, running for one hour). 45. Depot — $0.36/hr 44. Vercel Sandbox — $0.341/hr * 43. SuperHQ (Nanite) — $0.317/hr * 42. Runloop — $0.317/hr shutting down Nov 13 41. Render Sandboxes (new, beta) — $0.316/hr * 40. AWS Lambda MicroVMs — $0.252/hr 39. Quave ONE — $0.25/hr 38. Modal — $0.238/hr 37. BoxCompute — ~$0.20/hr (estimated) 36. Tensorlake — $0.20/hr * 35. Upstash Box — $0.20/hr * 34. Cloudflare Sandbox — ~$0.20/hr * 33. Namespace — $0.18/hr 32. E2B — $0.166/hr 31. Daytona — $0.166/hr 30. Tenki — $0.166/hr 29. Thunder Compute — $0.166/hr 28. Promigence — $0.166/hr 27. Fly.‌io Sprites — $0.165/hr * 26. BoxLite Cloud — ~$0.158/hr (not confirmed) 25. boxd — ~$0.15/hr * 24. Dedalus — ~$0.149/hr (not confirmed) 23. Beam (new) — $0.14/hr 22. Freestyle (YC S24) — $0.132/hr 21. CreateOS — $0.119/hr 20. exe.‌dev — $0.105/hr 19. Buddy Sandboxes (new) — $0.0864/hr 18. AWS Nitro Sandboxes, on-demand — ~$0.083/hr 17. Northflank — $0.067/hr 16. AWS Nitro Sandboxes, 1-yr Savings Plan — ~$0.059/hr 15. Quivvo — $0.054/hr 14. Mosaic — $0.05/hr 13. Nodus Compute (new, YC F26) — $0.0357/hr 12. Cathedral — $0.035/hr 11. Celesto — $0.030/hr 10. AWS Nitro Sandboxes, Spot — ~$0.029/hr 9. Dory (new) — $0.0288/hr (shared vCPUs) 8. Zode — $0.0225/hr 7. Maritime (YC F26) — $0.022/hr 6. Boat (YC F26) — $0.018/hr 5. Munito (YC-backed) — $0.018/hr 4. InstaBox (YC P26) — $0.017/hr 3. Lizard — $0.0162/hr 2. Cube — ~$0.016/hr sunsetting 1. Agent 37 (YC F26) — ~$0.006/hr (shared host) teapawt.org/Jccorzot/status/212020…
4
257