サメQCU retweeted
"the bitter lesson for search is to grep over a filesystem, just like how google works" - rich sutton
2: who benefits?
Well, you'll quickly discover reasons why you can't use adamw anymore; those arbitrarily large outliers that existed at init must be clamped down by construction to have any training signal across associativity variations. Pretraining will improve.
all that will be left of kimi will be their reverse engineered data mixture, the model and organization lost to the biting wind and chilling void of time.
we will find in retrospect that there was no deep learning theory supporting the inference-inefficient architectures; that they did not pretrain well, that they did not rl well, that they were not separated by architecture but only data mixture choices from their peers.
xitter.cf/teortaxesTex/status/21…
so my predictions? preregistered.
my forecasts? impeccable.
my hubris? perfectly calibrated.
my wisdom? peered, but they're pretty great peers.
because i'm far higher g than you, i had a bad theory of mind in the 2010s and thought that botnet acceleration was going to get worse and worse as programming became easeier for a larger talent pool and more poor people were connected to the internet, so i never used php
i think anthropic staff are far more concerned about evil lying chud claude problems than the rest of us particularly because Anthropic Is Not Good At RLVR, and the kind of personalities they get from bad rlvr on decent starting persona work are scary yudkowsky genies.