Replying to @habibislop @tszzl
I think there's no clear standard for “complete” understanding. Suppose an SI does it for us, we still have to verify its interpretability, or we’re just trusting a model about a model.
51
Replying to @tszzl
Even if mech interp is solved, which I think is highly unlikely, I still don’t think it solves alignment. There are too many variables to make alignment a solvable problem.
For example, aligning a model is just biasing it toward one set of values. You can push that bias as hard as you want, but the other values it absorbed from its training data are still in there, so misalignment opportunities will always exist.
Even if there is an interp breakthrough in this scenario, how would you remove those values without stripping crucial knowledge a model needs to work?
1
193
Replying to @tonywestonuk @lieroes98
This is equivocation, grapes don’t contain alcohol but fermenting them produces alcohol.
Asking where consciousness may exist in a matrix multiplication is like asking where alcohol exists in a singular grape. By your own argument, no one can point to it in a single neuron either.
1
2
61
Replying to @max_spero_ @tautologer
I think a major problem is pangram does not provide any further information apart from a random number which is of course quite unhelpful for a user.
(obviously does not make sense for pangram to reveal such info) but it pushes people to entrust a detector as faithful whilst ignoring all possible edge cases like how AI was used in any given piece of text.
For example, you can go on arxiv cs.AI and grab a few latest pdf’s, almost all of them show as 70-80% ai generated. But how much of that is true? (For example a paper could already include AI output to explain a point).
If pangram can clarify it’s detector is agnostic in the sense of how it’s scoring (again, whilst omitting details that invite dual usage) it would help because right now a user could suspect the detector could be biased where it’s seen some AI content and may misconstrue the entire text portion as AI.
3
504
Replying to @NPCollapse
Is it really? I did a write up on it yesterday
rectifies.ai/blog/is-moltboo…
1
2
766
Replying to @fchollet
It’s important to note that this shouldn’t be framed as a user issue. The problem is on the model side, eg (alignment, RLHF)
3
462
Replying to @fchollet
Its a valid form of sentence structuring that has become too common now, this problem will only increase over time.
My prediction is online discourse will be impossible to differentiate between AI and human pretty soon
1
8
1,272
Replying to @AnthropicAI
Probably one of the only frontier companies that cares about interpretability and safety. Great work Anthropic.
7
1,243
Replying to @bayeslord
This is because karpathy is well respected and it’s his own opinion based on all the work he’s done on LLMs.
The truth is no one truly knows how long it will take to achieve AGI and the definition of AGI is still very broad
4
818
Replying to @unusual_whales
mech interp can only go so far, the works of @NeelNanda5 has been exceptional so far but the reality was that this would happen, we just didn't know when.
3
441
Replying to @Sauers_
Yeah but it's less about predicting datapoints and more about shaping frames that physics or cognitive science later quantified.
Example: early atomism (Democritus) guessed matter was discrete way before empirical proof. Later, information theory and computation vindicated deep ideas from formal logic (Leibniz, Boole).
1
13
701
Replying to @zivdotcat @sagevedant
This is a really nice website I recommend exploring: bbycroft.net/llm
You have made an error here by saying, “It just keeps going until it deems it’s done.”
To know when a point has been made requires understanding and reasoning as we know it.
If I state,
“A pineapple is yellow when you slice through it”
You could argue,
“Well, not exactly, it could just be a pigment in the chemicals that causes it to appear yellow”
Now, just using this example, how much further could you go? How much reasoning is required to reach a sufficient conclusion?
At some point, you stop. Not because of an inherent, mystical self-awareness, but because your internal model of reasoning deems further speculation diminishingly useful.
LLMs do exactly this: they don’t blindly run forward, they terminate based on learned heuristics for coherence, sufficiency, and contextual completion.
You argue that humans know when they’re guessing. But introspection itself is unreliable, we often believe we reason deductively when in reality, we’re post-hoc rationalizing intuitive guesses. The difference, then, is not in the process, but in the illusion of conscious control over it.
1
4
216
Then let’s scrutinize ‘thinking.’
For example, when Einstein reasoned about quantum entanglement, was he not at some level making probabilistic inferences based on prior knowledge?
If LLMs ‘guess’ by weighting possibilities over vast contexts, how different is that from human intuition?
1
144
Reasoning isn’t an ontological privilege of biological neurons.
If structured inference over patterns leads to novel conclusions, it is reasoning, just not the kind that flatters human exceptionalism.
Dismissing it as ‘regurgitation’ ignores its emergent capabilities.
1
8
218
rectifies retweeted
OpenAI o3-mini is now available in ChatGPT and the API.
Pro users will have unlimited access to o3-mini and Plus & Team users will have triple the rate limits (vs o1-mini).
Free users can try o3-mini in ChatGPT by selecting the Reason button under the message composer.
Replying to @ProfFeynman
Reminds me of what Geoffrey Hinton said:
“I refused to give up. I spent years thinking about this problem, and I knew there had to be a way to make neural networks work.”
6
394


