I build AI products at scale. I share what it takes to be an AI PM. Only practical, real-world advice. No-fluff, no theory.
Start here ➡
Joined November 2014
- Tweets69.7K
- Following3.8K
- Followers29.7K
- Likes38.2K
To be a great AI product manager, you NEED to understand the basics of AI.
Start here. 10 videos. 85 minutes.
(Save it for easy reference)
1. AI vs ML vs DL vs GenAI youtu.be/qYNweeDHiyU
2. How LLMs work youtu.be/LPZh9BOjkQs
3. How to write good prompts youtu.be/jC4v5AS4RIM
4. Why AI makes things up youtu.be/005JLRt3gXI
5. How RAG gives AI the facts youtu.be/T-D1OfcDW1M
6. What a vector DB does youtu.be/gl1r1XV0SLw
7. RAG, fine-tuning, prompting youtu.be/UabBYexBD4k
8. What is MCP youtu.be/_d0duu3dED4
9. AI agents, clearly explained youtu.be/FwOTs4UxQS4
10. 7 terms for every AI PM youtu.be/VSFuqMh4hus
Over the next 5 Saturdays, I will be showing live how I apply these learnings to build real AI products. I'd love it if you joined our online sessions. We start this Saturday, 24 Oct.
First up: your own AI second brain.
Sign up free
Sign up here. Seats are filling fast. Free to attend. But seats limited:
justanotherpm.com/ai-pm-live…
The nature of the work we do has already changed. Most of the change is already irreversible.
AI does not simply assist the product manager. It redefines the constraints (and opportunities) under which we work. The old world--where we translated ambiguity into documents, coordinating across teams, built roadmaps with narrativ--was built for a world with bottlenecks and expensive coordination.
Those bottlenecks are disappearing.
What remains is judgment:
- the ability to define what “good” means in measurable terms
- to build the systems that stay true to it
- to detect, early, when the system has drifted from the goals
Those who continue to operate as if the old constraints still hold will discover that their work is increasingly substitutable.
Data analysis, user research, writing requirements, brainstorming solutions. These are no longer super powers
What is/will be a real super power is the clarity to say which outcomes matter, which failures are unacceptable, and which trade-offs the product will accept.
The same prowess defined how we did the work in the old world too. But the unpredictabilty in our products has madethe same work more challenging.
The world will keep changing. Models will become better. Your learnings and expertise will be obsolete. The only way to to stay relevant (and try to win) is to be part of the movement.
As builders, leverage AI to become more efficient not to sound cool.
As PMs, build products that are forward looking. Build solutions solve the problem of tomorrow. Embrace the unknown, bet big, learn fast.
But learn.
Invest time to continuously learn --> do --> learn
That is the only way to survive in the new world.
The role will continue. The individuals who occupy it will be those who have made the new constraints their own.
AI PMs who can’t build and own evals are already behind.
Here’s why this is the highest-leverage skill right now, and the exact path to get good at it.
1/ Evals are the new unit tests + PRDs for AI products.
Traditional software has deterministic behavior. LLMs don’t. Same prompt → different outputs. Without systematic evals you’re shipping on vibes, and vibes don’t scale.
8/ Tools to actually practice with (start simple):
Promptfoo (great CLI + YAML configs)
DeepEval (pytest-style)
Ragas (if you’re doing RAG)
LangSmith / Braintrust / Phoenix for traces + online evals
You don’t need to master all of them. Pick one and build something.
9/ The meta skill is error analysis + calibration.
Look at failures, categorize them, write better rubrics, validate your LLM-as-judge against human scores, then automate. Iterate the eval as hard as you iterate the product.
Evals aren’t an engineering side quest. They’re how you turn non-deterministic AI into a product you can actually ship, improve, and defend.
Start with one feature and 50 examples this week. The rest compounds fast.
As an AI product manager, please learn:
I am teaching some of these. More details at the end.
✦ Agentic workflows: multi-agent orchestration, tool calling, stop rules, and the agent spec (scope, tools, when to escalate, what it must never do)
✦ Model capabilities and limits: product problem vs model limitation, behaviour under ambiguity, and capability jumps between model versions
✦ Prompt engineering: system prompts, few-shot examples, chain-of-thought, and prompts as product config
✦ AI prototyping: Claude Code, Cursor, Lovable, v0, and prototypes over specs
✦ AI quality metrics and ship bars: task completion, accuracy, precision and recall, p95 latency, and cost per request
✦ AI governance: data privacy, PII handling, audit trails, and acceptable-use policies
✦ AI strategy: data moats and where AI belongs vs AI theatre
✦ Model selection: quality vs latency vs cost, open vs proprietary, and small vs frontier models
✦ Human-in-the-loop design: review workflows, escalation and handoffs, levels of autonomy, and approval boundaries
✦ AI experimentation: offline vs online evals, prompt and model A/B tests, and head-to-head model comparisons
✦ AI UX and trust: designing for uncertainty, setting expectations, transparency, and explainability
✦ Designing for non-determinism: probabilistic outputs, ask vs guess on ambiguous input, and measuring progress when answers vary
✦ Labelling and synthetic data: annotation guidelines, ground truth, and synthetic data generation
✦ Agent memory: session, user and episodic memory, and what to store vs forget
I'll teach some of these live,
In my upcoming masterclasseses.
Every Saturday, starting 24 Oct, 9am PT.
Link below 👇
Starting an invite-only AI PM masterclass series:
• 5 Saturdays
• Live sessions
• We build one real AI product every week
• In Claude Code
𝗙𝗶𝗿𝘀𝘁 𝘀𝗲𝘀𝘀𝗶𝗼𝗻:
• Saturday, 24 Oct
• 9am PT / 9:30pm IST
• Agenda: building an 𝗔𝗜 𝘀𝗲𝗰𝗼𝗻𝗱 𝗯𝗿𝗮𝗶𝗻 (Video below)
If you want to join the free masterclass, reply "AI". We'll send you the link.
P.S. Limiting entry to people already working in product (PMs, designers, engineers, founders). Priority if you have built or are building, AI products at work or as a passion project.
Here is how you become an AI PM before 2027.
(3 months left in the year. Bookmark this post. You will know more about AI than 90% of PMs)
Why AI gives different answers
↳ AI predicts the next word, which is why the same question can get two different answers and why it sometimes makes things up (hallucinates). Then ask any AI the same question twice and compare.
↳ Course: AI Capabilities and Limitations course (anthropic.skilljar.com/ai-ca…)
How to test it (evals)
↳ An eval is a test for your AI. You write questions with the right answers, run them, and count how many it gets right. Then write 20 test questions for something you use AI for and ask a second AI to mark the answers.
↳ Course: Building with the Claude API, Prompt Evaluation section (anthropic.skilljar.com/claud…)
What this means for AI PMs
↳ I translated these learnings for AI PMs in a free email course. Sign up here to see how this works in the real world (newsletter.justanotherpm.com…)
