Sachin Chaurasiya (@sachindotcom) header
Sachin Chaurasiya (@sachindotcom) profile picture

Building AI agents in production at @isotopes_ai (aidnn), and sharing what I learn. Notes on coding agents, AI tools and what breaks. Prev-HackerRank. DMs open.

India
Joined January 2019
Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)Photo by Sachin Chaurasiya (@sachindotcom)
I asked ChatGPT to plan a two-day trip. It suggested more places than I could fit in. Same at work: one day to ship, and the coding agent found gaps everywhere. I didn't fix them all. I wrote them down. On patience and adaptability:
2
2
157
I've been working on a fairly large feature with coding agents for more than a week now, and something happened that made me rethink how I'm using them. I started with the requirements, business context, and an initial spec. Over time, Claude explored the problem, proposed approaches, asked clarifying questions, and gradually built out the implementation. Everything seemed to be moving in the right direction. I was answering questions, giving inputs whenever needed, and making progress. But when I finally started testing the feature myself, something felt off. It wasn't how I would have built it. Some of the decisions didn't match my engineering taste, and the more I explored the implementation, the more I realized that somewhere along the way, I had lost a good amount of understanding of what was being built. That was honestly frustrating. Because one thing I've always cared about as an engineer is understanding what I'm building and why certain decisions are being made. But with coding agents, it's surprisingly easy to get into a flow where the agent keeps making progress, asks you a few questions, you answer them, and the work continues. And because things are moving, you don't always notice when your own understanding starts falling behind. I think that's what happened here. So I decided to stop pushing the implementation forward for a while and spend time understanding what had already been built, questioning the decisions, and figuring out what I would actually do differently. I'm still learning how to balance this. I genuinely enjoy working with coding agents, and I don't want to go back to doing everything manually. But I also don't want to reach the end of a feature and feel disconnected from something I'm responsible for shipping. For me, that's probably the biggest lesson from this experience. I want coding agents to help me execute my thinking, challenge my assumptions, and explore better approaches. But I don't want to slowly become someone who just approves the direction they're taking. I should be directing the agent, not the other way around.
1
2
105
One thing I've noticed while debugging multi-agent systems is how easy it is to reach for another rule whenever something goes wrong. An agent picks the wrong tool, misses an important detail, or makes an assumption it shouldn't have. The immediate reaction is often to update the system prompt or add another instruction so it doesn't happen again. Sometimes that helps. But after spending a lot of time reading session traces, I've realized that instructions are not always the problem. The agent might not have received the right context. Two tools might have similar descriptions, making it difficult to choose between them. Or perhaps the expected behavior was never clear enough in the first place. In these cases, adding another rule doesn't fix the underlying issue. It may even make things worse as instructions grow and start conflicting with one another. That's why I've started valuing session traces more when debugging agents. Instead of immediately changing the prompt after a failed run, I try to understand what the agent actually saw, which tools it selected, where it made assumptions, and what happened after each step. I also think successful sessions deserve the same attention. When you compare successful and failed runs, you start noticing patterns that are easy to miss when looking at one session in isolation. Maybe the successful runs had context that the failed ones were missing. Maybe a particular tool consistently causes confusion, or the agent keeps retrying because it doesn't know when the task is complete. Those observations give you a better starting point for improving the system. Sometimes the fix is an instruction. Other times, it's a change to context retrieval, tool design, routing, or verification. I'm finding that the more useful question is not "What rule should I add to prevent this failure?" It's "Why did the agent behave this way, and what needs to change so the system handles it better next time?"
3
2
132
When we start building something new, we naturally try to think about all the possible use cases, edge cases, future requirements, and how the system might evolve. That thinking is important, but there is a limit to how much we can predict before people actually start using what we have built. Sometimes we end up spending too much time designing for scenarios that may never happen. I’ve started appreciating the importance of building a focused first version, getting it into people’s hands, and learning from how they actually use it. Because real usage tells you things that planning alone cannot. You start noticing patterns. You understand which workflows people use frequently, where they struggle, what they expected to happen, and which problems are actually worth solving next. And then your decisions become much more informed. Maybe an abstraction you thought was necessary isn’t needed at all. Maybe something you considered a small edge case turns out to be a common workflow. Or maybe users find a completely different way of using the feature than you originally imagined. That doesn’t mean we should ignore quality or ship something unreliable. It means being intentional about what the first version needs to solve, while leaving room to learn and improve. I think good product engineering is not about predicting everything correctly from day one. It’s about building something useful, observing how it behaves in the real world, and letting that understanding guide what you build next. #product #engineering
2
174
I started my career in 2021 with a very simple mindset: Whatever I am working on, I should understand it properly, own it fully, and make sure what I ship is high quality. That mindset helped me a lot. It made me care about details. It made me take responsibility instead of saying, "this was not my part." And for a long time, I thought good engineering meant getting things as close to perfect as possible before shipping. But over the last couple of years, especially with AI changing how quickly we can build, I have had to unlearn part of that. Not every first iteration needs to be production-ready. Sometimes the right thing is to move fast. Build the idea. Put it in front of the customer. Understand whether the direction is even useful. Then decide what deserves the extra engineering effort. That was initially uncomfortable for me because it felt like I was compromising on quality. But I don't think that is the right way to look at it anymore. There is a difference between: "This is good enough for a demo." and "This is good enough for production." Both can be valid decisions. The important part is knowing which one you are making. I still care deeply about quality. But now I think ownership is not only about making everything perfect. It is also about understanding the business context, knowing what level of quality the current stage needs, and being clear about what still needs to happen before something becomes production-ready. That adaptability has probably been one of the bigger things I have learned as an engineer. Sometimes good engineering means polishing. Sometimes it means learning faster.
1
38
Sachin Chaurasiya retweeted
You need both taste and judgement. You need taste to know what is good, and you need judgement to understand the price of quality. If you ask a model for a component or migration plan, it will create a lot of options, very quickly. Most of those drafts seem OK. So once you have a collection of work that meets the basic criteria, you can no longer use taste to select between them. You must choose. I like to think of this as two filters. There is the taste, and there is the judgement. And I think a lot of people mix those up. The first filter is taste. Taste is the feeling for what makes a thing good. Taste is that feeling you get first, before you're able to put into words what is good or bad about something. You've had that experience where something is bad but you're not sure why, until you've worked it through and can describe it. "This user experience was really clumsy" or "that error message was really aggravating." Taste is built up by experience and practice. It comes from looking at a lot of things, and making a lot of things, and remembering the good bits. Kant's great insight into aesthetic taste was that you could appreciate the form, without having a stake in the outcome. Engineers care about the outcomes much more than the forms, but the feeling is the same. Taste can disqualify a lot. It can disqualify 90% of drafts very quickly. But it's not enough to make a decision between the few remaining options. The second filter is judgement. Judgement is understanding the price of quality. The most perfect solution may not be finished by the time that you need it. The sanest architecture may require the users to migrate while you're in the middle of the final upgrade. Judgement is the understanding of what you give up in time and risk to get good things, for the sake of the users, for the sake of the team. Judgement is made from domain knowledge and experience from previous decisions. People with great taste but poor judgement make beautiful work too late. They make the right thing at the wrong time, or the wrong thing at the right time. People with judgement but no taste make on time but don't make things people love. Both taste and judgement are essential. Filter to a few options with taste, and filter to one option for delivery with judgement.
73
90
13
648
32,587
What else becomes possible once we solve this problem? I think this is a useful question to ask when building something. We usually start with a very specific need, but sometimes one part of the solution turns out to be useful somewhere else too. For example, imagine building a tool that makes a bug easier to reproduce. It captures the relevant context, preserves what happened, and gives someone a way to inspect the behavior without reconstructing everything from scratch. Now that same capability could also help an engineer explore an unfamiliar feature. Or help a support team understand what actually happened for a customer. The connection becomes clearer when you look at the underlying need. All of them need enough context to understand the behavior without spending a lot of time rebuilding the story themselves. That does not mean we should design a generic framework from day one. I would still keep the first solution focused on the actual problem in front of us. But once another use case appears, we now have something real to compare against. Which parts are genuinely shared? Which parts should stay separate? Does reuse actually make the system simpler? Or are we forcing two different problems into one abstraction? This is one of the parts of product engineering I enjoy the most. Solving the problem in front of us well, while staying curious about what that solution might make possible next.
1
21
Since I got access, I have been playing with Jev, first in the @typesafeai playground to understand how it actually works, what a state is, what a question is, and how they get executed. I got really interested after seeing the results in the playground for the idea I was trying, and then I thought of building an app for the same idea, so I built a scam checker. The idea is simple: paste a suspicious text message and get a straight answer and the reasons behind it. It is for someone worried and not technical. My parents, probably yours too. They get a message saying their account will be blocked today, and there is nobody to ask at 9pm. The way it works is that Jev does not write text. It answers typed questions. So my app asks 22 small questions about one message in a single call, and all of them run at the same time in about a second. Things like "does this ask for an OTP?" and "does the link belong to the company it claims to be from?". Then my own code adds them up and decides what to show. One learning I want to share: I had a question asking whether the message tells the reader to keep it private, and a real bank SMS saying "do not share this code with anyone" scored 0.92. That is literally true, so the question was wrong, not the model. Fixing the wording fixed it. I have open-sourced the code and also shared my learning in one doc file. The links are in the comments, so check it out, play with it and share your thoughts. And tell me what you are building with Jev.
2
2
186
I think the next few weeks are going to be really interesting for builders. I have been looking at Jev and this idea of using semantic judgment in places where today we usually rely on rules, heuristics, or full LLM calls. That part is what I find interesting. Not every problem needs the model to generate something. Sometimes you just need the system to understand the context and make a good decision. For example: Is this relevant? Which path should this request take? Does this output actually satisfy what we expected? Should the agent continue, retry, escalate, or stop? These kinds of decisions are everywhere once you start building agentic systems. So I want to spend the next few weeks trying Jev, looking at other models and approaches using similar ideas, and figuring out where semantic judgment actually helps and where normal deterministic code is still the better choice. I am especially curious about using it in evals, routing, memory/retrieval, side projects, and some of the decision points inside agent workflows. I don’t really have a final conclusion yet. Right now I am just building, testing, breaking things, and trying to understand where this actually fits. And honestly, this is probably the part I am enjoying the most right now. You learn something, try it in a real system, and your understanding changes again.
2
56
Sachin Chaurasiya retweeted
Reproducibility is important for analyzing data (with or without AI). However, it’s completely distinct from “verification” of the analysis. This is particularly so for Finance teams. We saw this firsthand when an accountant reviewing a reconciliation found a $3,813.58 difference between two analyses of the same 3,144 transactions. The issue wasn't the data or the math. It was how one word in the original request, "discrepancy", had been interpreted. @isotopes_ai's approach treated negative differences as errors and added them. The accountant's workpaper treated them as adjustments and netted them. Both were reasonable interpretations of an ambiguous request. Running the analysis again would have reproduced the same interpretation and returned the same answer. That's why I don't think verification is about adding a better checkmark to AI-generated analysis. The person reviewing the work needs to see the reasoning, sources, assumptions, and corrections behind the number, without needing an engineer to unpack it for them. If AI is going to do more analytical work, verification has to make that work easier for humans to interrogate, not harder. More on how we’re we approach verification: isotopes.ai/blog/trust-but-v…
3
3
694
After debugging a lot of multi-agent sessions, one thing has become very clear to me: Don't debug agent systems only by looking at failures, read the traces and read the successful ones too. A failed run tells you where the system broke, but a successful run often tells you something equally valuable: Why did the agent choose the right tool here? What context helped it make the right decision? Why did this workflow finish cleanly while another one kept retrying? Where did the human need to step in? What happened differently this time? Once you start looking across multiple sessions, you stop reacting to individual failures and start seeing patterns. Maybe the agent repeatedly misses the same piece of context. Maybe one tool description keeps causing confusion. Maybe a specific step creates unnecessary retries. Maybe successful runs consistently have one piece of context that failed runs don't. That is where the real improvement starts. You form a hypothesis. You improve one part of the system. Then you watch the traces again. Over time, some of those patterns become rules. Some become eval cases, some become better tool descriptions, some change your context strategy, some become part of your harness and slowly, something interesting happens. The model is still probabilistic, the path may still be different every time, but the outcomes start becoming much more repeatable. That is the part I find most useful about agent observability, traces are not just there to tell you why something failed, they are showing you how your system actually behaves. And if you keep studying those patterns, they eventually become a roadmap for improving the product.
1
2
117
Anthropic removed more than 80% of Claude Code’s system prompt for its newer models, and according to them, coding performance did not measurably decline. I found that really interesting because it says something important about how working with coding agents is changing. For a long time, we tried to tell agents everything: follow the existing code style, use meaningful names, read the related files, reuse existing abstractions, write maintainable code, and so on. But newer models already understand many of these things. More importantly, they can inspect the code around them and build judgment from it. They can see how errors are handled, how components are structured, how tests are written, and what patterns are commonly followed in the repository. I have started noticing this while using Claude and Codex as well. The better the models become, the less useful it feels to keep stating the obvious. What matters more is giving them the context they cannot infer from the code itself. Why does this unusual constraint exist? Which part of the system should not be changed? What edge case has caused problems before? Which shortcut is unacceptable here? What needs to be verified before the task is considered complete? This is also changing how I think about CLAUDE.md, AGENTS.md, and skills. They should not become large textbooks explaining software engineering to an already capable model. They should capture the hidden context: exceptions, gotchas, past decisions, important boundaries, and things the codebase cannot communicate on its own. There is another side to this as well. If models are increasingly learning from the patterns around them, then the quality of those patterns matters even more. A clean and consistent codebase gives the agent better examples to follow. A messy codebase gives it more bad patterns to repeat. So as models become smarter, I think our context should become smaller and more intentional. Not less useful. Just more focused on what the model cannot figure out by itself.
We removed ~80% of the Claude Code system prompt for our newest models, this is what we've learned about writing system prompts, skills and Claude.MDs for them.
1
6
806
One thing I have learned while using coding agents: More context does not always mean better results. Sometimes it does the opposite. We naturally think:"If the agent knows everything, it will make better decisions." But that is not how humans work either. Imagine joining a project and someone gives you: 500 pages of documentation. Every Slack conversation. Every old decision. Every unrelated discussion. You don't become smarter. You become overwhelmed. Agents have the same problem. The goal is not to give them everything. The goal is to give them the right information at the right time. The best AI workflows I have seen are not the ones with the most tools, plugins, or instructions. They are the ones that are intentional about context. What does this task need? What can be ignored? What should be loaded later? The future of AI engineering is not about giving agents more information. It is about becoming better at context management.
3
102
I think we are moving from prompt engineering to contract engineering. When people start using coding agents, the first question is usually: “How do I write a better prompt?” But after using agents for a while, I think the bigger question becomes: “How do I define success clearly?” Because agents are usually good at starting tasks. The harder part is knowing when the task is actually complete. A human engineer naturally knows: “This is not ready yet.” Because they understand the context. They know the edge cases. They know the trade-offs. They know what quality looks like. Agents need that made explicit. A good agent task needs a contract: What needs to change? What should not change? What tests should pass? What behavior should be verified? What does done actually mean? There is a big difference between: “Build this feature.” and “Build this feature. Add these tests. Verify these flows. Do not modify these areas.” The second one is not just a better prompt. It is a better contract. I think the future of AI-assisted engineering is not giving agents more freedom. It is giving them better constraints. Because capability creates output. Constraints create reliability.
2
2
112
When we talk about AI coding agents, we usually focus on the instructions we give them. "Build this feature.", "Fix this bug.", "Refactor this component." But there is another prompt that is always running. Your existing codebase. The agent sees: How your components are structured. How your team handles errors. How your tests are written. How abstractions are created. How previous engineers solved problems. And it learns from those patterns. If your codebase has clear boundaries, consistent patterns, and good abstractions, AI becomes a multiplier. It has better examples to follow. But if your codebase is full of shortcuts and inconsistent decisions, AI can amplify those problems too. It may create another abstraction instead of improving the existing one. It may follow the wrong pattern because that pattern already exists. It may make the immediate task easier while making the system harder to maintain. This is why I think code quality matters even more in the AI era. We used to say: "Someone new joining the team will have to understand this code." Now there is another consumer of your codebase: AI agents. The quality of your codebase does not just affect humans anymore. It affects the systems helping humans build. Clean code was always valuable. Now it is becoming a productivity multiplier.
1
42
AI gives us leverage. Ownership makes that leverage safe. As engineers, we will absolutely use AI. It can help us write code faster, explore options, debug issues, understand unfamiliar areas of a codebase, and move through implementation with much more speed. But AI cannot take responsibility for production quality. It does not know the full business context. It does not understand every trade-off. It cannot always judge whether a shortcut is acceptable for this system, this customer, this deadline, or this stage of the product. And when something breaks in production, AI is not accountable. You are. That is the part we should not forget. You can use AI to move faster. But you still need to review the code, verify assumptions, test edge cases, think about maintainability, and make sure the solution actually solves the right problem. Your value is not that you can do everything AI cannot. Your value is that you can use AI well and still own the outcome like an engineer. AI can generate output. But engineers own outcomes.
1
69
Sachin Chaurasiya retweeted
Everyone is still asking if India can build its own AI. @sarvamai stopped asking and shipped it. Raised $234M building India-first LLMs. 10 million API calls a day. Half a million hours of audio transcribed a month. In languages no global model bothered to learn. Built from India. Built for the world. Here's their story:
52
259
17
2,259
139,957
Code is not cheap. For the last one year, I have heard this idea many times: "Code is cheap now." Because AI can generate it. Because coding agents can ship faster. Because implementation is no longer the slowest part. I understand the point. But after using coding agents for more than a year, I don’t fully agree with it. Code is not cheap. Bad code is expensive. And in the AI era, bad code may become even more expensive than before. Because coding agents don’t work in isolation. They learn from the patterns already present in your codebase. If your codebase has clear boundaries, good abstractions, consistent patterns, useful tests, and readable structure, the agent has a better chance of producing good output. But if your codebase is messy, inconsistent, and full of shortcuts, the agent will often continue that pattern. It may copy the wrong abstraction. It may follow the wrong convention. It may add one more patch on top of an already fragile flow. It may solve the immediate task while making the system harder to maintain. For small codebases, this can still feel manageable. You generate fast. You clean up later. You accept some quality trade-offs. But in larger codebases, this can become a nightmare. Because every bad pattern becomes training material for the next change. At some point, you are not moving faster. You are just entering a loop of fixing one thing after another. That is why I feel code quality matters more now, not less. AI does not remove the need for good engineering practices. It makes them more important. Clear constraints matter. Strong tests matter. Small PRs matter. Consistent patterns matter. Readable code matters. Architecture boundaries matter. Review discipline matters. Coding agents can increase velocity. But without quality constraints, they can also increase chaos. So maybe the statement should not be: "Code is cheap now." Maybe it should be: "Code generation is cheap. Code ownership is not."
1
25
AI agents need guardrails. So do engineers. When we design agents, we think about constraints. What tools can it use? What actions are allowed? What should require approval? What data should it not touch? What should happen if something fails? Because an agent without guardrails can confidently do the wrong thing. But the same is true for humans. An engineer without clear guardrails can also make the wrong trade-off. Not because they are careless. Because the boundaries were never defined. Should this change be backward compatible? Can we touch this legacy flow? Is this experiment reversible? What is the rollback plan? Which metric should not regress? Which behavior is load-bearing? These questions matter before implementation starts. In the AI era, this becomes even more important. Because execution is faster now. A wrong assumption can turn into a working PR very quickly. That is why good engineering is not just about speed. It is about operating within the right constraints. Guardrails are not there to slow people down. They are there to make fast work safe. AI agents need them. Engineering teams need them too.
3
37