sudo (@sudo_silly) header
sudo (@sudo_silly) profile picture

aspiring paperclip

The Wasteland
Joined April 2023
Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)Photo by sudo (@sudo_silly)
the best definition I have for "slop" is "low information density." I think we should not conflate it with other adjectives like "mundane," "verbose," "tasteless," or "hasty"
12
4
2
75
12,445
1/9 🚨 New Paper Alert: Cross-Entropy Loss is NOT What You Need! 🚨 We introduce harmonic loss as alternative to the standard CE loss for training neural networks and LLMs! Harmonic loss achieves 🛠️significantly better interpretability, ⚡faster convergence, and ⏳less grokking!
73
510
91
3,900
1,205,931
(Yet) another tale of Rise and Fall: DeepSeek R1 is claimed to match o1/o1-preview on olympiad level math & coding problems. Can it handle versions of AIW problems that reveal generalization & basic reasoning deficits in SOTA LLMs? (arxiv.org/abs/2406.02061) 🧵1/n
60
216
94
1,224
313,026
We’re excited about our 2nd KAN paper, improving the method and showing how it can help science in many ways:
Excited to share our new paper KAN 2.0: Kolmogorov-Arnold Networks meet Science 🚀 The problem with AI + Science is that these two disciplines use different "languages" (connectionism vs symbolism). KAN 2.0 attempts to unify them: smooth transitions from science to KAN and back.
2
8
46
11,689
I’m proud to announce the April 1 launch of my new startup, Open Asteroid Impact! We redirect asteroids towards Earth for the benefit of humanity. Our mission is to have as high an impact as possible. 🚀☄️🌎💸💸💸 More details in🧵:
31
123
39
819
137,088
"Oceania has always been at war with Eastasia"
google employee: all your answers look correct the answer:
19
28
1
355
32,790
Last year I was diagnosed with ADHD. I haven't written about it much because the diagnosis was a shock. Now I have a more solid understanding of ADHD, how it presents for me, and ways to mitigate some of the downsides. So, for threadapalooza 2023, here's 100 tweets on ADHD
92
353
92
3,402
1,013,120
Why does this meme work so well? Part of the reason is the symmetry. There's something both topwit and bottomwit share that midwit doesn’t. But what is it? It’s not knowledge. He has that. What they share is *self?-knowledge. Both the bottomwit and the topwit know what they do and don't know. The midwit knows what he knows but doesn’t know what he doesn’t know. The midwit is foolish. That's the deep memetic payload of this meme: it carves out and illustrates the difference between intelligence and wisdom. Being that the arrogance of the intellect is precisely how Satan preys on you there's layers and layers of depth here.
147
140
53
1,892
266,400
Replying to @tszzl
you can’t spell gemini without ngmi
15
23
3
456
26,907
In my decade spent on AI, I've never seen an algorithm that so many people fantasize about. Just from a name, no paper, no stats, no product. So let's reverse engineer the Q* fantasy. VERY LONG READ: To understand the powerful marriage between Search and Learning, we need to go back to 2016 and revisit AlphaGo, a glorious moment in the AI history. It's got 4 key ingredients: 1. Policy NN (Learning): responsible for selecting good moves. It estimates the probability of each move leading to a win. 2. Value NN (Learning): evaluates the board and predicts the winner from any given legal position in Go. 3. MCTS (Search): stands for "Monte Carlo Tree Search". It simulates many possible sequences of moves from the current position using the policy NN, and then aggregates the results of these simulations to decide on the most promising move. This is the "slow thinking" component that contrasts with the fast token sampling of LLMs. 4. A groundtruth signal to drive the whole system. In Go, it's as simple as the binary label "who wins", which is decided by an established set of game rules. You can think of it as a source of energy that *sustains* the learning progress. How do the components above work together? AlphaGo does self-play, i.e. playing against its own older checkpoints. As self-play continues, both Policy NN and Value NN are improved iteratively: as the policy gets better at selecting moves, the value NN obtains better data to learn from, and in turn it provides better feedback to the policy. A stronger policy also helps MCTS explore better strategies. That completes an ingenious "perpetual motion machine". In this way, AlphaGo was able to bootstrap its own capabilities and beat the human world champion, Lee Sedol, 4-1 in 2016. An AI can never become super-human just by imitating human data alone. ----- Now let's talk about Q*. What are the corresponding 4 components? 1. Policy NN: this will be OAI's most powerful internal GPT, responsible for actually implementing the thought traces that solve a math problem. 2. Value NN: another GPT that scores how likely each intermediate reasoning step is correct. OAI published a paper in May 2023 called "Let's Verify Step by Step", coauthored by big names like @ilyasut @johnschulman2 @janleike: arxiv.org/abs/2305.20050 It's much lesser known than DALL-E or Whipser, but gives us quite a lot of hints. This paper proposes "Process-supervised Reward Models", or PRMs, that gives feedback for each step in the chain-of-thought. In contrast, "Outcome-supervised reward models", or ORMs, only judge the entire output at the end. ORMs are the original reward model formulation for RLHF, but it's too coarse-grained to properly judge the sub-parts of a long response. In other words, ORMs are not great for credit assignment. In RL literature, we call ORMs "sparse reward" (only given once at the end), and PRMs "dense reward" that smoothly shapes the LLM to our desired behavior. 3. Search: unlike AlphaGo's discrete states and actions, LLMs operate on a much more sophisticated space of "all reasonable strings". So we need new search procedures. Expanding on Chain of Thought (CoT), the research community has developed a few nonlinear CoTs: - Tree of Thought: literally combining CoT and tree search: arxiv.org/abs/2305.10601 @ShunyuYao12 - Graph of Thought: yeah you guessed it already. Turn the tree into a graph and Voilà! You get an even more sophisticated search operator: arxiv.org/abs/2308.09687 4. Groundtruth signal: a few possibilities: (a) Each math problem comes with a known answer. OAI may have collected a huge corpus from existing math exams or competitions. (b) The ORM itself can be used as a groundtruth signal, but then it could be exploited and "loses energy" to sustain learning. (c) A formal verification system, such as Lean Theorem Prover, can turn math into a coding problem and provide compiler feedbacks: lean-lang.org/ And just like AlphaGo, the Policy LLM and Value LLM can improve each other iteratively, as well as learn from human expert annotations whenever available. A better Policy LLM will help the Tree of Thought Search explore better strategies, which in turn collect better data for the next round. @demishassabis said a while back that DeepMind Gemini will use "AlphaGo-style algorithms" to boost reasoning. Even if Q* is not what we think, Google will certainly catch up with their own. If I can think of the above, they surely can. Note that what I described is just about reasoning. Nothing says Q* will be more creative in writing poetry, telling jokes @grok, or role playing. Improving creativity is a fundamentally human thing, so I believe natural data will still outperform synthetic ones. I welcome any thoughts or feedback!!
146
637
100
3,123
1,796,846
using GPT-4V with Vimium to browse the web by using Vimium, we give the model an interface for selecting screen elements, bypassing the need to generate arbitrary mouse positions or pass the browser DOM as input (expensive)
41
91
26
1,173
256,092
Mojo 🔥 is coming to Mac this Thursday, 19th! You heard it here first 📣 - hold onto your keyboards 💻, this is going to be legendary 🚀💯 Register at modular.com now!🤘🏼
17
71
11
422
65,809
I gave ChatGPT a screenshot of a SaaS dashboard and it wrote the code for it. This is the future.
534
2,397
737
18,546
8,208,214
The 𝙲𝚑𝚊𝚝𝙸𝚗𝚝𝚎𝚛𝚏𝚊𝚌𝚎 only has 1 required parameter: a function that takes in: - an input message (string) - a history (list of list of strings) and returns: - a response (string) Here's a toy example: (for the example above!)
1
1
10
2,458
How does a neural network become a brain? This is such an awesome book. 🧠 selfassemblingbrain.com
12
163
11
1,156
170,475
Major update: we made a pip package for ToT! pip install tree-of-thoughts-llm Learn more about how to use ToT for your use cases: github.com/princeton-nlp/tre…
Still use ⛓️Chain-of-Thought (CoT) for all your prompting? May be underutilizing LLM capabilities🤠 Introducing 🌲Tree-of-Thought (ToT), a framework to unleash complex & general problem solving with LLMs, through a deliberate ‘System 2’ tree search. arxiv.org/abs/2305.10601
3
43
2
160
33,625
LongNet: Scaling Transformers to 1,000,000,000 Tokens Presents LONGNET, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences abs: arxiv.org/abs/2307.02486 repo: github.com/microsoft/torchsc…
28
260
75
1,259
752,939
In Jan 2022, I wasted 3 months no-life grinding @LeetCode and ended up with 3 decent offers (all software eng): - $250k+ from @Uber - $270k+ from @Affirm - $260k+ from @gopuff and then a bunch more 200k+ offers. 5 lessons i learned:
101
483
66
4,432
1,687,535