Brian Ivanovick (@bivanovick) header
Brian Ivanovick (@bivanovick) profile picture

gtm technical geek || vp solutions eng || @statsig @amplitude_hq || views my own

New York City
Joined December 2010
Photo by Brian Ivanovick (@bivanovick)Photo by Brian Ivanovick (@bivanovick)Photo by Brian Ivanovick (@bivanovick)Photo by Brian Ivanovick (@bivanovick)Photo by Brian Ivanovick (@bivanovick)Photo by Brian Ivanovick (@bivanovick)
Brian Ivanovick retweeted
Loved the session by @spenserskates How Industrialization of Products from Edison helped in creation of so much value to all of us. How @Amplitude_HQ is analysing the data for more customer conversions. Special mention from Spenser that "Nobody Knows What People Want"
1
11
246
Brian Ivanovick retweeted
The secret ingredient is the millions of hours of eng, pm, design and analytics work that went into building Wave 🌊 with taste and accuracy. Well done 👍
1
2
27
Brian Ivanovick retweeted
Introducing Wave. The world's first AI that reads your customers' mind. $1.47 trillion will be spent on software this year, but only a fraction of software features are used by customers. AI can build anything you want, but it doesn't know what your customers need. Wave fixes that.
165
130
70
519
468,810
Brian Ivanovick retweeted
Frank Slootman's Friday-night test for whether you're a driver or a passenger: "When you come home on a Friday night and look in the mirror, did it matter I was here? Where did I move the dials? It may be unfair, like it's only one week, but why not? I still look at myself that way. What did I do this week that's worth a damn?... If you're not certain you're a driver, you're probably a passenger."
3
2
34
85,971
Brian Ivanovick retweeted
Replying to @trq212
Yes, there is no path forward besides leaning into the future
1
6
511
Brian Ivanovick retweeted
Wow. I woke up to an impressive number of upset people who apparently don’t understand how LLMs work and can’t cope well with me pointing it out. An LLM is a statistical next-token prediction model. That’s how it works. It’s an extraordinarily sophisticated calculator performing math you could theoretically do by hand (if you were sufficiently masochistic). Math doesn’t know what it’s calculating. Pi doesn’t spend its evenings contemplating the nature of circles, and c doesn’t ponder the relationship between matter and energy. That doesn’t make the results any less accurate, impressive, or useful. Between inference requests, an LLM isn’t sitting around thinking. It isn’t reconsidering its answers, wondering whether it made a mistake, contemplating its existence, or questioning whether it’s conscious. Its weights remain fixed (it doesn’t learn), and computation resumes when it receives input. We know this because we designed these systems. The architectures, mathematics, implementations, and research are extensively documented. You can study them yourself. Of course, some people immediately lean into comparisons of biological intelligence or philosophical debates over consciousness because that is what they know. But biological intelligence and artificial neural networks are fundamentally different systems. Similar outputs don’t establish equivalent internal experiences. I experience consciousness directly. I infer that other humans do too because we share biological structures and processes. We don’t have equivalent evidence for an LLM, and nothing about its known operation gives evidence of conscious experience. Which brings me back to my original point. To me, understanding is conscious comprehension. It means being aware of information, recognizing its meaning, and experiencing that comprehension. If I wasn’t conscious I could not understand. We can perform actions unconsciously. We can produce correct results without knowing why they’re correct. But I distinguish those capabilities from consciously understanding what we’re doing. An LLM can solve complex problems, explain difficult concepts, and produce remarkably accurate information. I give it full credit for those capabilities. But producing an answer that we rationalize as understanding isn’t the same as experiencing understanding. You don’t have to agree with my definitions. We can debate functional understanding versus conscious comprehension all day, but that won’t establish whether an LLM actually experiences anything. The right debate is this… Could machines exhibit something different from human consciousness, something measurable, meaningful, and worth studying? I think so, absolutely! I’m entirely open to discussing “machine consciousness” in terms of observable, testable computational properties. We built these systems, we can instrument them, examine their internal mechanisms, and investigate what they’re capable of. That seems far more productive than endlessly arguing over whether a machine experiences life the way humans do. So here’s my question: How would you define machine consciousness, and more importantly, how would you measure it?
Not sure who needs to hear this but.... An LLM doesn't "understand" anything. It is a statistical model that spews out tokens based on probabilities calculated through massive matrix operations and context, not an understanding of the subject matter.
595
911
176
6,318
568,616
Brian Ivanovick retweeted
Replying to @orangebook
"Even though you try very hard, the progress you make is always little by little. It is not like going out in a shower in which you know when you get wet. In a fog, you do not know you are getting wet, but as you keep walking you get wet little by little." (Zen Mind, Beginner's Mind, "Nothing Special")
1
2
336
Brian Ivanovick retweeted
Pure gold. My fav — People become what the room expects of them. If the room lets a standard slide, it slides. If the room corrects the small thing every single day, people stop believing small things are small. My goal is to build an environment where success is inevitable.
I've coached over 30 Olympic gold medals. I've worked with some of the greatest athletes in history. Here are the lessons I wish more people understood:🧵
6
20
297
76,628
Brian Ivanovick retweeted
Here's your friendly reminder: We all gotta be careful 😂 #SalesMeme #SalesHumor #MEDDPICC #MEDDIC
1
1
123
Brian Ivanovick retweeted
We've been testing Jev in our own AI analytics benchmarks for when a chart is presenting an anomaly and needs a deeper dive. It's significantly faster and cheaper but it's overly sensitive eg low precision, escalating much more often than the best general purpose models. Analytics datasets are significantly more complex to navigate smaller decision models have difficulty capturing that complexity.
9
4
49
5,207
Brian Ivanovick retweeted
Nobody wants your slop. There is one issue with this mindset (and I agree with the mindset!) I can’t get over: and it’s that people need to interact with the thing you do for you to learn from it, and therefore, if it is bad you are providing them a bad experience and they will bail. Yes you can “just build it and try it” but if “IT” feels bad to use, you are burning your actual users!! This is where the realities of the new world and mindset of the old world diverge. Tolerance for crappy experience will go WAY down, quickly, as the expectation of software performance rises, quickly… what happens when consumer expectation increases exponentially? You can try more things, but they have to be f*cking good
This has never been more true. The opportunity cost of endless planning and contemplation just went way up. Agents allow you to just try vastly more stuff, so let them, and you'll learn way more, way faster.
1
2
9
898
Brian Ivanovick retweeted
We put GPT-6 Sol, GPT-6 Luna, and Claude Opus 5.5 through our Amplitude Global Chat eval and mapped the outcomes on our quality/cost frontier. Our eval is 185 analytics tasks from simple to complex: building a chart, editing one, choosing the right event in a taxonomy, getting the correct date range, finding the property that most explains the variance in a chart. These evals are all based on real customer queries. We grade answers on correctness only. We measure cost and latency separately. Here’s where each new model landed (one run each so±2%): -Claude Opus 5.5: 83% correct at $0.70 per question. That's about 2 points behind our top score (GPT-6 Astra) at roughly a sixth of the cost. -GPT-6 Sol: 81% at $0.58. It scored a little lower than GPT-5.6 Sol, but was considerably cheaper (less than half the price). This is the cheapest model we've tested that clears 80%. -GPT-6 Luna: 76% at under $0.04, making it the cheapest model on the frontier. It's about 3x cheaper than GPT-5.6 Luna and achieves about the same score. The tradeoff is speed: its p95 latency was ~400 seconds, about 3x slower than Opus 5.5. Even with the inclusion of these new releases, GPT-6 Astra still gets the top score (84%) on our eval. The tradeoff is it costs $4 per question and was one of the slowest models we tested (900 seconds at p95).
18
14
4
73
14,746
Brian Ivanovick retweeted
When Databricks CEO @alighodsi asked his new CRO Ron Gabrisko to teach him sales, this is what he said: "Most early-stage engineers think, if I just build the best product with the best pricing, everybody will buy it. It doesn't work that way. The power of sales, especially in enterprise, is that salespeople teach your customers how to use the product and get value out of it. So I start on a whiteboard, talking about organizations. When you sell to a startup, there might be one person to sell to, the CTO or the founder. When you sell to an enterprise, they decide as an organization. You need to understand the org chart. Who are the players? Who's the decision maker? Who owns the budget? How do approvals get made? Sales is more about asking smart questions and listening than having a glitzy pitch. Come in prepared, research their objectives, go in high in the org, and ask a lot of questions. Then be credible and trustworthy in how you follow up. Don't bug them. But if there's a specific way you can help, that's what a good salesperson does. There are two frameworks I'd recommend for anyone learning enterprise sales: MEDDIC, a process for how you govern a deal, and Command of the Message."
9
11
4
257
74,170
Brian Ivanovick retweeted
We've had 6.3M MCP tool calls run in August which is up 5x since March. Agents querying product data is our fastest growing usage category of Amplitude. Inspired by @zenorocha
11
4
6
49
6,069
Brian Ivanovick retweeted
👀 self improving product in the wild! @Amplitude_HQ team asking: can a product factory actually ship good product?
(1/6) i'm the pm on Wave at @Amplitude, our agent for self-improving products. for the past few months we've had it running on our own product. it finds problems, ships fixes, and measures whether they worked. here are some highlights:
7
5
100
19,739
Brian Ivanovick retweeted
Today we're launching Agent Analytics Every team shipping an AI agent has the same blind spot. Offline evals pass, you ship, and then you have no idea what's happening in production. AI fails silently. Users ask a question and get different answers. They all look 'engaged' in a classic dashboard. You don't know who got a great response and who got a terrible one. Agent Analytics solves it: - Every session scored out of the box on task completion, response quality, friction, safety, and negative feedback - Topic clustering across thousands of conversations, so you know if a failure hits 1 user or 10,000 - Eval agents that watch for regressions, and if you want will file a Linear ticket or the pull request themselves - Agent quality sits next to product data, so 'payment scheduling fails 31%' becomes 'which renewals did that cost us?' The Economist got their agent to a 96.9% task success rate and cut weekly failures 84%. Included on every plan. Free tier included. amplitude.com/agent-analytic…
61
49
25
221
56,601
Brian Ivanovick retweeted
Replying to @StatueofIBBertY
Today we're introducing Wave. A proactive product agent that helps teams build self-improving products. Every product team runs the same loop. Build, ship, use, learn. AI has made building and shipping extraordinarily fast. Understanding usage and learning still happen by hand. Wave runs the whole loop: - Analyzes your Amplitude data for signals across analytics, feedback, session replays, error logs, agent traces, and experiment results - Surfaces opportunities in the form of full product specs that you or your agents can approve and ship - Tracks the experiments and outcomes of what you've built so you can start the loop over again Your product learns what works and gets smarter every cycle: amplitude.com/wave
2
1
11
356
Brian Ivanovick retweeted
The dashboard is starting to look like an intermediate step. PostHog calls it self-driving. Amplitude calls it self-improving. Both are converging on the same product loop. Observe behavior. Find something worth improving. Make the change. Measure the result. Learn. Repeat. Coding agents made changing software cheap. Analytics has the context to decide what should change. Jacob's examples make the category feel a lot more real.
(1/6) i'm the pm on Wave at @Amplitude, our agent for self-improving products. for the past few months we've had it running on our own product. it finds problems, ships fixes, and measures whether they worked. here are some highlights:
18
12
3
226
56,648
Brian Ivanovick retweeted
Replying to @spenserskates
I'd put Amplitude in the rocketship category. Companies that are still founder led often are - see Airbnb, Block, etc.
1
1
1
23
16,572
Brian Ivanovick retweeted
Hanging out with @poteto @n2parko @clairevo and the @bot team to get Amplitude setup on Grok Bot today. We're pretty permissive on AI tooling, but even so Grok Bot automatically takes action so getting the security right is the biggest blocker. Spent the first 20 minutes making sure we don't shoot ourselves in the foot. It always feels like you're drowning as an executive and you don't have time to spend messing around with your tooling setup. I think they've gotten the interface right as you're able to take action right away.
25
16
7
281
55,541