šÆ recursive "agentic" functions to step through complex tasks vs single shot completion
I think AI agentic workflows will drive massive AI progress this year ā perhaps even more than the next generation of foundation models. This is an important trend, and I urge everyone who works in AI to pay attention to it.
Today, we mostly use LLMs in zero-shot mode, prompting a model to generate final output token by token without revising its work. This is akin to asking someone to compose an essay from start to finish, typing straight through with no backspacing allowed, and expecting a high-quality result. Despite the difficulty, LLMs do amazingly well at this task!
With an agentic workflow, however, we can ask the LLM to iterate over a document many times. For example, it might take a sequence of steps such as:
- Plan an outline.
- Decide what, if any, web searches are needed to gather more information.
- Write a first draft.
- Read over the first draft to spot unjustified arguments or extraneous information.
- Revise the draft taking into account any weaknesses spotted.
- And so on.
This iterative process is critical for most human writers to write good text. With AI, such an iterative workflow yields much better results than writing in a single pass.
Devinās splashy demo recently received a lot of social media buzz. My team has been closely following the evolution of AI that writes code. We analyzed results from a number of research teams, focusing on an algorithmās ability to do well on the widely used HumanEval coding benchmark. You can see our findings in the diagram below.
GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%.
Open source agent tools and the academic literature on agents are proliferating, making this an exciting time but also a confusing one. To help put this work into perspective, Iād like to share a framework for categorizing design patterns for building agents. My team AI Fund is successfully using these patterns in many applications, and I hope you find them useful.
- Reflection: The LLM examines its own work to come up with ways to improve it.
- Tool use: The LLM is given tools such as web search, code execution, or any other function to help it gather information, take action, or process data.
- Planning: The LLM comes up with, and executes, a multistep plan to achieve a goal (for example, writing an outline for an essay, then doing online research, then writing a draft, and so on).
- Multi-agent collaboration: More than one AI agent work together, splitting up tasks and discussing and debating ideas, to come up with better solutions than a single agent would.
Iāll elaborate on these design patterns and offer suggested readings for each next week.
[Original text: deeplearning.ai/the-batch/isā¦]
Johnny Wu b/acc retweeted
My favorite part of a startup is when the growth engine starts kicking in
And the business starts compounding every month
Our MRR has grown 10%+ m-o-m & our assets have grown 20%+ m-o-m for the last 6 months at @carryhq_
& we're hiring 2 dope people to join our team in NYC:
Johnny Wu b/acc retweeted
Jensen showing us the endgame of GenAI:
Multi-modal in -> Multi-modal out.
The is the final model we will all use one day.
Can get any modality and generate every modality.
While also being able to work without each of it's parts.
Johnny Wu b/acc retweeted
*Velocity is the only way to quality*
These are not opposites. When you make decision quickly you can fix mistakes just as quickly. Every cycle is an opportunity to improve your product, that includes investing in craft once you're confident in your ideas.
Johnny Wu b/acc retweeted
Hearing that Tiger Woods and Yasir al-Rumayyan will play golf/have already played to discuss the future of pro golf in the Bahamas.
Love that the biggest meetings to discuss the future of golf take place...on the golf course.
Johnny Wu b/acc retweeted
There is no penetration testing for Identity and KYC fraud
Yet it's the most vulnerable part of the financial system
We do cyber security penetration testing. The apps are secure. Can't be hacked.
But they're trivial to defraud
Johnny Wu b/acc retweeted
Naval Ravikant: āYou only have to be right onceā
In the clip below, Naval reflects on how it took 13 years after starting his first startup when he was 23 before founding AngelList.
āEntrepreneurial ventures fail all the time. Most of my companies failed. I actually started seven companies, and Iāve launched about 40 or 50 projects over my career. And AngelList is the first one that I would truly say is product, market, entrepreneur fit for me that might succeed down the road.ā
He tells the audience:
āItās a low hit rate over your career, but you only have to be right once⦠So just keep trying, just keep iterating⦠a lot of it is market timing and you learn a lot of lessons.ā
In his essay How To Be Successful, Sam Altman writes:
āāI will fail many times, and I will be really right onceā is the entrepreneursā way. You have to give yourself a lot of chances to get lucky.ā
Johnny Wu b/acc retweeted
highly curious people only want one thing and thatās to develop a comprehensive first principles view of the entire world, all of history, and everything in existence
Johnny Wu b/acc retweeted
Today I learned that Trader Joeās doesnāt add food coloring to any of their foods or drinks and instead uses natural ingredients for color.
I think Iāll only buy our kids snacks from there going forward.
Incredibly based
Johnny Wu b/acc retweeted
Request for startups: Iād like an interactive voice based AI assistant that will help me go through my emails and calendar in the 20 minute drive to work. It should draft emails based on minimal voice input and reschedule appointments that I want to move.
Johnny Wu b/acc retweeted
Competitive robotics creates the most cracked engineers.
And China is cooking the rest of the world:
DJI runs a comp for uni students + uses it as a hiring funnel.
Millions of people watch.
Robotics engineers are superstars there.
US needs this, so Iām building it.
šØāš¦½
Humanoid robots are coming...
Trillion dollar question is whether a few companies will capture the economic profits or this will become similar to the EV industry (low margin, high competition, high capex business with price wars)?
bloomberg.com/news/articles/⦠via @BW
šÆ
Will US Chips Act $$ be successful in helping build competitive chips fabs in the US? A must listen talk at MIT by Morris Chang - Founder of TSMC - about the structural challenges to onshoring modern chip production. BG2 Ep3. @bgurley @altcap @SecRaimondo @NVIDIAGTC
š
Important to clarify...
If @saylor executes his strategy well it is entirely probable MSTR outperforms #Bitcoin in fiat terms consistently for a very long time.
However, there are numerous single points of failure with MSTR stock. There are multiple ways this strategy backfires and the stock underperforms #Bitcoin.
Adjusted for risk, #BTC in proper self custody is by far the better thing to own.
It is entirely possible to be extremely bullish on @MicroStrategy stock because you think @saylor and the company will execute their strategy well, yet still understand #BTC is the more important asset to own and the better risk-adjusted asset to own... MSTR is the better fiat returns than #BTC on the condition the humans behind MSTR perform well.
MSTR is probably the best bet in the world on an entity controlled by humans. However - #Bitcoin's monetary policy isn't controlled by humans at all...
@MicroStrategy may be a very good (or the best) investment, but it is not money.
š«”
to build a great b2b company you, you need a founder who believes that distribution & sales are equally as important as the product.
a great gtm motion starts from the top downānot just in theory, but in action. what it means to prioritize go-to-market:
ā revenue teams have real input to the product roadmap because they are the voice of the customer
ā youāre indexing strong engineering talent towards go-to-market
ā youāre hiring people on go-to-market who are as hungry for the companyās long-term success as the engineers are
the focus on sales & distribution has to come from the founderās dna, otherwise it wonāt seep into the culture of the org.
this is something that the ramp founders nailed. and that mindset has stuck with me as Iāve been building @unifygtm over the past year.
plg is an important strategy. it works. but you have to embrace sales if you have plans to build a generational company.
do you agree?
šÆ
b2b startups: true early adopters
do not want a complete solution
read that one more time
they want a quick, easy way to capture
a unique, competitive edge
before anyone else in the market
can extract the value
laggards want complete solutions
therefore, overbuilding will repel you
from true early adopters AND
anyone asking/requiring more features
is likely wasting your time (i.e., tire kicking)
something not often discussed
the spark of insanity from a founder
can vicariously be felt by the market
that excitement/enthusiasm of specific future value
is what early adopters buy into,
albeit if it's aligned with a problem they recognize
Johnny Wu b/acc retweeted
Replying to @JoeHolder_
This Trevor Noah pod he just put up talks about it a bit. Instead of adding more police if they really had people fix the subway up, clean it, make it something ppl could be proud of, think the paradigm shifts.
open.spotify.com/episode/7fnā¦

























