sudo retweeted
the best definition I have for "slop" is "low information density." I think we should not conflate it with other adjectives like "mundane," "verbose," "tasteless," or "hasty"
sudo retweeted
1/9 🚨 New Paper Alert: Cross-Entropy Loss is NOT What You Need! 🚨
We introduce harmonic loss as alternative to the standard CE loss for training neural networks and LLMs! Harmonic loss achieves 🛠️significantly better interpretability, ⚡faster convergence, and ⏳less grokking!
sudo retweeted
(Yet) another tale of Rise and Fall: DeepSeek R1 is claimed to match o1/o1-preview on olympiad level math & coding problems. Can it handle versions of AIW problems that reveal generalization & basic reasoning deficits in SOTA LLMs? (arxiv.org/abs/2406.02061) 🧵1/n
time is really great at filtering and compressing good information. twitter does not have this
1
105
i’ve noticed i don’t have a mental cache for preferences. “what kind of music do you like?”… mind is blank
65
best way to make conversational AI is give them rich inner lives and only bring that into context when relevant to the conversation
43
sudo retweeted
We’re excited about our 2nd KAN paper, improving the method and showing how it can help science in many ways:











