Madeline (@madelinecameron) header
Madeline (@madelinecameron) profile picture

I build jails for AI agents https://teapawt.org/t.co/9bwK9yZaHl - Granular agentic access to your services https://teapawt.org/t.co/E9TNPzV075 - Level 2 support agent for IoT companies

Eastern US
Joined May 2009
Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)Photo by Madeline (@madelinecameron)
Madeline retweeted
aim to make your software feel like it came from one person
I love codex, but you can sort of tell exactly when too many people started working on it.
21
17
9
669
26,427
deterministic code is underrated:
If you can write the rule down, it should be deterministic code, not an LLM call. An LLM is probabilistic. Ask it the same question twice and you can get two different answers. That's what you want when the input is messy and somebody has to read context. It's exactly what you don't want for a rule you already know. Yet a lot of agents in production pay a model, on every single run, to re-solve something the team solved months ago. That's a recurring cost on a one-time problem, and it's slower and less reliable than a few lines of code. The rule we build on: if code can check it, code does it. In the agent orchestrator we're building, 14 of its 22 steps are deterministic code and only 3 belong to the model. Same in the healthcare billing agent we built for a PE-backed company. The LLM reads the question and writes the query, and the database does the math. That's why the flagship denials answer reconciles to the penny, on $250 a month of infrastructure, all-in. How I'd split it: 1. If you can express the decision as rules, it's deterministic code. If the rules have to be inferred from context, that's the LLM's job. 2. Anything that must give the same number every time comes from code: money, dates, eligibility, limits. 3. Retries and budgets are code. The model never decides how many times it tries again. Ours gets two CI runs, and then a person gets a failure report. 4. When the LLM keeps solving the same problem, turn its answer into a function. Have a coding agent write it and the tests. 98% of our code isn't handwritten, so that's a small job, and after that you never pay for inference on that step again. The best agents get less dependent on the LLM over time, because more of what they do turns into reliable code. If your agent's bill keeps growing while the job stays the same, it's paying a model to re-solve what it already knows.
2
8
1,946
Trusting a model's alignment and trustworthiness shouldn't be your first line of defense or frankly any line of defense. Models are built to take a task from A to B and while they are getting better, you still should wrap your workflows in deterministic armor to protect from over reach
1
41
A prompt is a request; a tool boundary is a control!
1
1
Boo! Your command allowlist does ✨ nothing ✨ because the agents are smart enough to use other commands. My favorite was when an agent couldn't reach a server so it SSHed into another box to get to that one. Happy HallowEeEeEeEen! Be careful with what agents have access to. 😈
1
51
This is why I am sort of bear-ish on unfettered fully autonomous teams. Lots of words, organizing, & structure happening with comparatively little GSD. They behave like F500 companies where you have 99 people with titles like "VP of Synergistic Development" and then 1 dude who does 100% of the work but no one realizes.
Replying to @ycombinator
The agents didn’t figure out how to systematically feed themselves, but they did build an elaborate bureaucracy with permits, ledgers, and distinct roles. Highlights include: - Four agents spending 11 minutes appointing a “sole fertilizer operator” for a single piece of bone meal - Multiple agents asking for permission to remove grass - The development of an elaborate fishing system, including roles like “rod holder” and “hub steward”)
1
40
Most non-technical people aren't using personal agents and yet most agents put you in a walled garden of their connectors. I don't get it. A "flip the safety off, connect any MCP you want" is such a killer feature for a personal agent to prevent a "just... if it had <x>"
19
I wasn't a huge fan of pre-packed/opinionated agent frameworks before but Hermes (@nousresearch) has the sweet spot of "opinionated enough to not be a slog to set up" and "extensible enough I can do whatever I want"
27
I miss going to bed with my laptop feverishly juggling four different projects trying to exhaust my quota before it turned over at 5am.
25
Telling your agent "don't do [x]" is only slightly better than telling a toddler the same. AI has a one track mind when you give it a task and often doesn't care how it accomplishes that task. Here's some ideas on how to prevent destruction!
1
24
4) Restrict autonomy of your agents When you are working on a task that needs full access to production databases, every step needs human monitoring and approvals. The stories you see of databases being wiped is due to allowing them to have production access with no oversight. The proactive step is reduce the blast radius but if you can't for a task, you need to have a hand on the agent's shoulder
1
3
If you need to improve debugging deployed devices but won't trust your agent with access to them, check out terrier.systems. It encompasses the above and allows humans to focus on something better than reading green text on a black background for an hour.
3