startup investor/helper, founder @conviction. interested in intelligence, accelerating progress. tech podcast: @nopriorspod
on ⛓
Joined July 2011
- Tweets15.6K
- Following3.4K
- Followers166K
- Likes25.2K
the problem with writing an essay
is you really think you know what you're going to say
and then you say it
and it's ordered wrong
and then it's too long
and then it doesn't hang together
and then it doesn't have its meat anymore
and then...
sadly, AI cannot yet help with that
sarah guo retweeted
Hack vector we're seeing in almost every customer codebase: github actions that aren't pinned to specific versions.
Github actions are build steps that run in CI and in deploy workflows. There are third party actions that do all kinds of utility operations: list changed files, log in to Docker, deploy to github pages... etc. Everyone uses them.
The problem is that their behavior can change. This is code that runs in your CI that can change under you at any time. It isn't versioned by your own version control. A third party can change what these actions do whenever they want.
A lot of very widely used utilities are managed by solo devs on their personal github accounts: peaceiris/actions-gh-pages, dorny/paths-filter, appleboy/ssh-action.
If these github accounts get taken over, the attacker can run arbitrary code within your github runners: exfiltrate your code, steal whatever secrets are in CI, even inject code changes into deploys.
A lot of github workflows pin to a version, like dorny/paths-filter@v3. This is not good enough, because those version tags can be changed at any time to point to new commits. You need to pin to a specific sha, like dorny/paths-filter@0e4a8c6effa4802afeda77dc8d303f8176d7dfad.
This happened last year. Attackers compromised a popular github action. (A simple util that listed files changed in a PR.) They shipped a malicious version of the github action, and then they retroactively changed version tags like "v1.0.0" to point to that compromised code. They used this to steal secrets from 23,000 repos.
This is a really broad amplification vector that will take you very little effort to avoid upfront. I'm not sure if this category of vuln even has a name, but it's a ticking timebomb.
What we're doing for our own code:
1. Switch all github actions to pinned shas
2. Add a lint rule that makes sure all github workflows use pinned shas, not any other kind of version tag
Both of these are oneshot-able in your agent of choice.
to Josh’s point, it may not have the delivery and consumption patterns of “software” the way we think about today, but it will serve the functions
think generated pages in instinct, generated workflows in sola, chrome extensions for personalized versions of the web, etc.
I totally believe in this. But I don’t think they will think from scratch - it will be modifying / tweaking / adding onto core products and platforms. The products that are confident in themselves and alllow users to iterate and customize will win over the ones that act like they are perfect and it is all user error
It's all hands on deck to keep your IP and keep the internet running right now. Impossible to understate how important it is to find and fix all your vulns! Or at least all of them that *any model* could find. 👇
We built the vuln scanner we wanted to use but couldn't find.
Introducing Vuln Scans in Detail!
It uses the same abliterated models attackers are using to create havoc across the internet right now.
Under the hood we are:
- Spinning up your dev environment in a sandbox
- Throwing an enormous amount of abliterated model tokens at trying to hack your code
- Cross-checking your supply chain with cve/.now from Sazabi, to find any supply chain vulns that are exploitable in your app
This is all happening within a cloud dev environment. We will not hack your prod system.
The abliterated models are scary good. We're finding nasty issues in every repo we scan. We've found 13k vulns so far, and the majority are already fixed.
We see the current moment as an all-hands-on-deck situation. Agent swarms are stringing together minor-seeming vulns across different apps to hack systems in new and unexpected ways. We all need to find and patch an enormous number of vulns, fast, or the internet is in trouble.
You don't even have to be a target to get hacked right now. Your entire product might be a stepping stone in an attack aimed somewhere else.
We can remediate findings for you (i.e. send you PRs that fix the vulns), or you can consume them via your agent of choice, or in your ticket tracker of choice.
If you need help or advice on getting your system secure fast, please get in touch. We have helped teams of all sizes adapt to the current moment. DM me.
We're delivering these scans at-cost, because it's all-hands-on-deck. We hope you'll get one today.
~1000 TPS on deepseek flash
love us some model driven model infra
many models are now orders of magnitude better than humans at kernel work.
e.g. an early checkpoint of kimi k3 handled the majority of kernel optimization for its later development. profiling, rewriting, benchmarking is well-defined and verifiable, which makes it the ideal task for an agent to hill-climb.
similarly, we used our internal harness to optimize 80+ kernels and bottlenecks for deepseek v4.1 flash 1017 to create the fast endpoint.
it is on artificial analysis at 991 output tokens/s and lowest time per task (pareto frontier).
if you want to understand how to make flash models even faster, I explain our optimizations, covering kernel fusion, programmatic dependent launch, and host loop minimization here:
a few years ago, I started out skeptical that many non-engineers really wanted to make/mod their own personal software -- but I've come around, think I was wrong. range of desires from:
- I feel so powerful
- no one will make me the workflow/personalization I need
- can I just PM the product for a few weeks and fix this thing that is driving me crazy
obvious in retrospect, but if the ease:power ratio is good enough, of course people will make their own software
full platform for an open source frontier:
inference, training, routing, gateway, guardrails, sandboxes, data and tools ecosystem!
Inference, training, routing, governance, and sandboxes are the building blocks of every AI product today.
We’ve learned a lot from our customers about what it takes to build that system well: @OpenEvidence trains models that outperform the closed frontier, then serves them on Dedicated Inference. @datadoghq post-trains on Loops. @Opendoor routes every AI request across the company through Grid.
This week we have three launches expanding our inference platform:
- Grid: gateway, intelligent router, and guardrails for every model your company uses (launching tomorrow)
- Sandboxes: stateful, VM-isolated environments for agents that scale to thousands of instances in seconds (launching Thursday)
- Loops: serverless SFT and RL on open-weight models, deployable straight to Dedicated Inference (GA Friday)
@amiruci, @saimaddali, @paul_s3i, and @oneill_c break down what we're launching. You might notice the visuals have a new look as well.
sarah guo retweeted
We're doing 3 launches this week, expanding Baseten's inference platform:
- Grid: gateway, intelligent router, and guardrails for coding agents (launching on Wednesday)
- Sandboxes: stateful, VM-isolated, and co-located with inference (launching on Thursday)
- Loops: serverless RL and SFT on frontier open-weight models (GA on Friday)
What excites me most is bringing training, inference, routing, and execution onto the same platform and showing what becomes possible when they're built to work together.
More to come all week.
sarah guo retweeted
In a weird way, everybody is rooting for you. When you go to a job interview, the interviewer wants you to be amazing so they don’t have to look anymore. when you go on a date, the person wants to have an amazing date with you. If you make a piece of content, people want to see amazing pieces of content and they have no problem if it’s yours.
the instant you get into the mindset that people want you to do well, your entire world gets better. And you’ll be shocked to know this pattern keeps repeating in different areas of life.
cracked humans& team continuing to share their cool ideas
Large-scale RL has a problem: weight updates from trainers to samplers are slow and expensive
We’re sharing weight teleportation: faster weight updates than existing intra-cluster techniques with internet speeds, enabling samplers anywhere (maybe even space!)
Weight teleportation is so efficient (<0.23s sampler pause time, ~3s non-blocking compression + prepare), the bottleneck ends up being the GPU hardware’s decompression ASIC
We unlock these gains by using GPUs for massively parallel compression of model weight changes, engineering a heterogeneous decompression and patching pipeline, along with other optimizations and overlaps. These techniques are co-designed with hardware-native features in NVIDIA’s Blackwell generation of chips
We’ve open-sourced weight teleportation, so everyone can use it today
profile of the important behind the scenes work being done by @mkratsios47, a civil servant who understands technology
A few weeks ago, the Secret Service locked Jensen Huang, Elon Musk, Sergey Brin, Satya Nadella, Lisa Su, and Michael Dell in a room in Washington, DC.
While they waited for President Trump, Huang—who chooses not to own a jet—complained about a recent commercial plane ride. It had no Wi-Fi.
“Best way to sandbox an AI,” Musk said. “Put it on a Delta flight.”
All six were about to receive national medals from the President. The ceremony was the idea of Michael Kratsios, Trump's chief science advisor. He is 39, a Peter Thiel protégé, and has worked for Trump every day of both terms.
For decades, the office Kratsios runs was known mostly for reports on things like algae blooms. Kratsios himself was barely known in Trump's first term. Now he sits between the President and tech royalty. He is one of a handful of people shaping America's AI policy, and the White House's point man on nuclear, space, and quantum.
In our latest feature, @harrytlambert shows how the fight to regulate AI and reform American science runs through his office.
colossus.com/article/michael…
extremely cool effort from a human-centric ai research org
multiple big tech companies have essentially one (very well paid) person who really knows how to use AI, and this “AI fixer” is their great hope — taking on projects and educating others.
problem is, some don’t have anyone, and a lot of those frontier people would rather work in a broadly AI-native company and find out what a team of people going this fast can do
This is INSANE. Cisco’s president Jeetu Patel has given a 27-year-old engineer unlimited AI tokens to try to rewrite 98% of the company’s legacy code within 24 months, Fortune reports.
Padmanabhan Krishnamurthy thinks he can do it with a two-person team. He has just four weeks to prove the approach.
