
building @appliedcompute. founder, previously at Jane Street.
New York City
Joined September 2022
- Tweets69
- Following307
- Followers190
- Likes80
Dan DiPietro retweeted
Poker is a hard post-training problem. Many players, hidden information, and results only show up at the end of a long game. It's even harder with only 24 hours on the clock.
We're sponsoring the PokerBench track at the @ValsAI hackathon this weekend to see what teams can train on AC2.
Spots are limited. We'll see you at the table.
Surprise! Vals AI is adding a third track backed by @appliedcompute, offering a $7,500 cash prize for the first place team.
Alongside Build Your Own Eval and Post-Train a Small Model, a limited number of teams will post-train a model to compete on PokerBench, Vals’ 10-player no-limit Texas Hold’em table.
Every team starts from the same reference setup and gets the exact harness their bot will be scored in. Bots play each other all weekend, with live rankings and overnight tournament checkpoints. First, second, and third place take home cash prizes.
Low-precision is the future. Frontier open models like 4.1 Flash and K3 already serve inference with low precision. I think the same thing will happen soon for training. Training actor forwards, backwards, and even optimizer state will move to 8-bit and 4-bit regimes.
Traditional quantization (PTQ) degrades accuracy. We used low-precision training to recover lost accuracy with quantization-aware distillation. Using this technique, we can consistently reduce quantization KL. This does not always manifest as improved reward!
1
19
425
Read the full research report. If you're interested in working on problems like this, please DM me.
appliedcompute.com/research/…
1
14
239
Dan DiPietro retweeted
Applied Compute update:
- crossed $100M annualized revenue run rate up from zero 18 months ago
- raised $350M at $3.25B led by Elad Gil
- 40% of customers use AC for both custom model training + production inference
- inference serving 1T tokens a day after launching 12 weeks ago
- 2X enterprise customers since last raise
- 3X team since January, 85% are researchers and engineers
- Working with customers like Microsoft, Shopify, Harvey, DoorDash, Cognition, Clay, Decagon and others
To move even faster we need people who are extremely technical AND extremely practical. If this is you, please reach out.
We would need 35 Enrons to cancel out the public wealth creation from Amazon's early IPO.
SOX is a great example of how seemingly well-intentioned regulation can have horrific unintended downstream consequences, e.g. reducing middle class participation in an entire category of wealth creation.
9
465
Sometimes verifiers are actually very expensive to run, and it’s much cheaper to use an LLMaJ instead. One example: suppose you’re RLing a model to build HFT strategies, and each rollout requires an extensive event-driven backtest that takes hours to complete. (1/n)
Unsaturatable reward means that, no matter what you do, the model probably won’t be perfect. In other words, a backtest will likely never perfectly predict prices out-of-sample. If you cannot observe a perfect trajectory, it is difficult to construct a rubric that can measure the path to it. Unfortunately, many awesome problems (quant research, math, formal) fall into this bucket. (3/n)
1
7
172
One of the core problems in continual learning is training without forward-looking bias: how do you improve a model’s performance without that improvement being conditioned on privileged information from the future?
2
22
1,015
You should care much more about precision being high. If the model consistently builds the features it plans, the planning prefix is a good surrogate. Low precision breaks the surrogate entirely, but low recall partly fixes itself. (3/n)
1
4
402
Poor recall might mean that we penalize the model when we shouldn’t, but this should introduce optimization pressure for it to disclose more features that it plans to build. However, this can turn into listing everything, which hurts precision. (4/n)
1
1
325
When RL rollouts take a long time, one optimization you can do is training on a planning prefix. For example, if you’re training a model to include a certain feature when building web apps, you can grade its plan or todo tool upfront for inclusion of this feature, rather than waiting for it to build it. (1/n)
Researchers commonly inspect loss curves during training. Batch composition affects these numbers significantly, especially if you are doing token-wise rather than sequence-wise loss averaging. (1/n)
1
17
764
Loss is unsurprisingly much lower on the on-policy data than the off-policy data. So the loss curve becomes a function of the ratio of aggregate number of tokens between these two arms and how it varies per batch. If you adjust for this, loss decreases pretty smoothly. (3/n)
1
4
63
In practice, you can mitigate this by: (1) decomposing loss curves into clear arms e.g. by distribution, task category, difficulty; (2) using sequence-wise averaging; (3) looking at baseline-relative curves, e.g. the log-prob ratio between the current model and the initial checkpoint on the same examples. (4/4).
4
52





