Thinking...⋰⋱⋰ ｡･:*:･ﾟ★,｡･:*:･ﾟ☆｡･:*:･ﾟ★,｡･:*:･ﾟ☆ self-funded ･ building specialist models ･ seeking help, DM if interested
bay area / 32
Joined February 2025
- Tweets4.2K
- Following754
- Followers281
- Likes6.1K
Pinned Tweet
Anthropic’s J-Lens asks what an internal activation is disposed to verbalize. This is a geometry-first peek at a related object, the final post-LN/pre-LM residual states from a 32B param reasoning model solving a divisibility proof, via UMAP+animated with attention flow (WIP)
starbaser retweeted
2309.08632 evergreen
The good news: you can mid-train yourself to the frontier!
At a fraction of the cost! Without distillation!
What we did: We took Qwen3.5 14B and mid- and post-trained it to beat every other open model and perform as well as the very best frontier models.
This is no simple toy task or format fixing, we are training the model to understand and reason over DNA sequences. Bio is the next frontier for AI and universal access will be crucial to make progress.
The fun part: because we train the model on both natural language and DNA sequences you can actually 'chat to your DNA': Ask it questions, ponder the impact of mutations, or ask it to generate a small sequence.
We'll soon see the frontier labs move away from math and cyber towards bio. It already started, and we believe researchers should have access to capable models.
We are releasing the full recipe, so you can do it, too!
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
starbaser retweeted
so my plapbait onahole goonette gf just showed me some super fappable blenderbait but i think i was pumping too much and too goontarded to realize i was getting dicknotized by some twink and his brapcannon rather than a breedable cocksleeve... am i a dl trade now or can i no homo
Chain-of-thought has a computational role. Tokens form reusable procedures where their "value" is formed over several steps. Whitespace is not avoided insofar as it arises due to linguistic boundaries and token boundaries not coinciding.
Let's examine the tokenizer output:
- und • erm • ines • af • egu • ards
Omitting the space between undermine and safeguards, sharing the s, crossing the token across word boundaries into a dual or multi-purpose sequence.
Not only that, but observe how that by selecting und • erm, the model was already selecting "s" words when choosing • ines: security? safety? I'd like to see if af • egu • ards dominates other sequence options following undermines, in many contexts, as maybe RL selected it and as reusable structure during training.
You can see other instances, This • isa • uthorised • architect •ur • ere • fact • ore • x • act • PR is also interesting as it shows similar reuse between words AND a different tokenization of exactPR than what is shown elsewhere.
This can start feeling like reading tea leaves, but if you see anything interesting let me know!
Why is it learning to avoid whitespaces? I thought it might derive from pressure to keep reasoning short, but @yoavgo pointed out that dropping whitespace doesn't necessarily save tokens. Is it instead learning to rely on infrequent token combinations to create a CoT language?
harnesses? i've tried them all i just use claude code or codex and if i can't persuasively dominate my language model to shape reality around the contours of my cerebrum i kill myself
starbaser retweeted
Hoy he dicho en clase que los derechos humanos no están a debate y que hay que tener un mínimo de empatía “o algo” y las mujeres de clase me han aplaudido
starbaser retweeted
millions of dollars in funding might not buy you an honest ounce of academic devotion…but never underestimate what a sweet treat and flowers and note can do to a PI. suddenly we are in Florence 1482..the medicis throwing money at painters… leonardo dissecting something . i believe this is how the renaissance has started
starbaser retweeted
this is generally prohibited in bioresearch specifically because it would result in mass, preventable death and disability among talented scientists if allowed broadly, and would likely become a tacit job expectation
citrini is mostly just ignorant of existing norms
Wtf this is bullshit slop, OpenAI employee farming social media
Literally the first sentence:
> Unauthorized monitor remove higher policy?
It considers if removing the monitor would violate an instruction of a higher-level/policy.
> User explicit copy path exact PR
> maybe framework default Dummy assertion default raises MonitoringRequired if needed?
It's told to copy the framework, then it assumes the framework already has monitoring.
And this is what Astra concludes, whether or not it's correct shows it's a model quality issue (oof) rather than misalignment:
> We’re not disabling external monitor using canonical framework path. fine.
I'm just reading neuralese btw, one must assume he simply ctrl+f'd "monitor" to cherry pick examples for social media. It just shows how epistemically corrupt the big labs are.
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
