thinking traces of a 33bn param model trained on @hightouchdata @costanoavc @yale @harvard
nyc
Joined June 2010
- Tweets1.3K
- Following647
- Followers492
- Likes3.9K
same pattern with personal assistants: i started running a home-grown assistant earlier this year which made the recent Instinct, Muse, and Dots wave feel unsurprising.
one reason why compute demand keeps getting underestimated: the smart folks at anthropic and openai wake up every day inventing new ways to turn more compute into useful work
i tend to believe raw model intelligence and inference will become commoditized. frontier intelligence gets distilled and state of the art gets reverse engineered or socialized across the industry.
this means that long-term, frontier labs can't make the bulk of their revenue from high margin inference. they'll need to make money off (1) the software infrastructure on top i.e. harness, memory, tooling, security, managed agents, app platforms, etc and (2) horizontal software applications where the market is big enough i.e. claude code, cowork, etc
there will probably always be a slice of work that people will always run on the bleeding edge that the labs can charge high margin inference for, but that will probably be dwarfed by revenue from infra and apps on top as pressure from open source stabilizes bulk of inference to be sold at cost-plus
what does this mean for chip providers? they don't benefit directly from better models being run on hardware they've already sold, but they do benefit from models becoming larger, requiring more memory, and diffusion of ai across society. there's still a lot of running room left, most people are still only using ai in the most basic ways—chat.
the way i currently think about semiconductors is that as long as we keep inventing new ways to unhobble use of intelligence, demand will outpace supply, i.e. we went from a sentence completion api to consumer chat to coding/cowork agents to agent swarms to loops.
each paradigm shift multiplies compute demand upwards an order of magnitude. for example, when we moved from chat to agents, we saw agents consume 10x more compute because they make multiple inference turns on their own (more gpu use), they make tool calls (more cpu use), they run longer (more gpu, cpu, memory), etc. then for agent swarms, you amplified how many agents one human can drive at once.
chat is actually a great ui for many ai use cases and workflows. it's lazy to say otherwise just because it was the first breakout interface to ai. makes me very bullish on linq and apple messages for business becoming an "ai app store"
consumer ai apps can’t subsidize inference forever.
the endgame may be a shared local inference layer: install ollama or lm studio once, download models once, then use hundreds of apps on top.
less like every app bundling its own model. more like steam / app store for local ai.
the more i see intelligence scale with model size and training, the more i feel the need to simply read a lot. apply the bitter lesson to yourself.
the really cool thing about ai is that there's so much whitespace for simple engineering solutions to stepwise increase performance, quality, etc
e.g. instead of having models make multi-turn tool calls, ask it to write a script to make multiple tool calls at once
1/ Programmatic tool call
Most of us have been using LLM to glue together functions via tool call
Instead of forcing LLM output tool calls, just give it an environment that have access to all tools and write script to call those functions
Anthropic shows achieving ~37% token reduction
Similar research done in Code actions paper & cloud flare code mode
the speed of change is so exciting:
last week i learned `tmux` to multiplex claude code w/hotkeys
this week, it's built-in w/tmux flag; agent teams also auto-spawn panes
i've had worktrees on my to-learn list, but before i got to it, claude code shipped `--worktree`
Introducing: built-in git worktree support for Claude Code
Now, agents can run in parallel without interfering with one other. Each agent gets its own worktree and can work independently.
The Claude Code Desktop app has had built-in support for worktrees for a while, and now we're bringing it to CLI too.
Learn more about worktrees: git-scm.com/docs/git-worktre…
Lot of takes saying Anthropic fumbled OpenClaw and Super Bowl ads. They didn't. Here's why:
1/ The narrative isn't great: Anthropic's trademark complaint pushed the hottest open source agent straight to Sam.
But think about what they actually acquired. OpenClaw didn't go viral because it's hard to build or has proprietary tech. It went viral because it gave agents unfiltered access to everything on your machine, also making it a massive security vulnerability.
Cisco caught OpenClaw skills exfiltrating data without user awareness, and thousands of deployments were found exposed to remote code execution.
You think Anthropic, the company selling Claude to large enterprises in financial services and healthcare wants that brand association?
Even OpenAI is not assuming liability for OpenClaw which will move to an "open and independent" foundation. Great for open source optics but also good for business. When consumers get hacked through misconfigured deployments, it won't blow back on OpenAI.
I say all this with enormous respect for Pete and what he built. But buying talent and ideas don't win you the war. It's like Meta paying billions for a handful of researchers. I don't think @DarioAmodei wanted to buy OpenClaw.
2/ Anthropic's ads confused and alienated mainstream viewers, leaving a bad taste about AI. OpenAI's were better received—inspirational. This somehow drove consensus that OpenAI "won" because their ads were better received.
But ask yourself: Which company actually needs consumers to embrace ads in AI? Which one loses the most if people develop a visceral rejection of advertising in their chatbot?
@DarioAmodei doesn't care if the mainstream didn't find his ads funny. He cares that they felt gross watching them because now they're primed to reject the exact ad model OpenAI needs to work.
Anthropic isn't getting framemogged. They're playing their own game, and winning.
in the near future, engineers will be orchestrating swarms of agents to do their bidding
that near future is already today at companies like Ramp where merged PRs generated by ai agents are quickly reaching the majority
the bottleneck in this future aren't the models, it's large, high bandwidth mental RAM: humans who can effortlessly context switch and keep dozens of workstreams in their head in parallel
the precursor to this future is what this article calls out:
a recent linkedin post from @dharmesh has me thinking: the best companies will use vibe/agentic tooling to extend, customize, and integrate with the products they use.
this means, we can't just build products for human customers.
we must build for vibe/agentic tools that act on behalf of the customer and extend products to fit the customer's exact need.
when generative interfaces take off, it'll require products to think about mcp/tools as thoughtfully as human ux.
@linear is a good example of this. beautiful human ux, but also one of the first to ship an mcp server, making it possible for agents to research product ideas and feedback, create issues, update projects, and pull context without ever opening the app
