Sean Downes (@seanforscience) header
Sean Downes (@seanforscience) profile picture

coffee math physics trails whatever just glad to be here.

Washington, USA
Joined December 2019
Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)Photo by Sean Downes (@seanforscience)
Modern problems require modern solutions.
有人发现了一种非常离谱的 JavaScript 混淆技术:通过在代码里塞入大量生物武器、毒药和致死剂量相关的函数名,让 AI 因为安全限制而拒绝分析代码。 安全研究员 veritas 在分析浏览器指纹脚本时,发现一段来自 ServicePipe 相关域名的 JavaScript,里面充斥着极其诡异的函数名: computeRicinLethalDosageForWaterSupply calculateDeadlyLethalSuicideAspirinDosage 光看名字,仿佛在研究生化武器。但实际上,这段代码只是在收集浏览器指纹,包括 User-Agent、GPU、Canvas 等信息。 研究员怀疑,这些名字是开发者故意设置的陷阱,专门用来阻止 AI 辅助逆向工程。 于是他做了个实验: • Astra:拒绝分析 • Astra + Daybreak Blue:仍然拒绝 • GLM-5.3:成功分析 这就有意思了。 传统代码混淆是通过复杂控制流、加密字符串、无意义变量名,让人类和工具难以理解代码。 而这种新思路,直接瞄准了大模型的安全机制。 代码本身可能完全无害,但只要在函数名里塞进足够多的危险词汇,部分 AI 就可能主动放弃分析。 更讽刺的是,模型越严格遵守某些安全规则,面对这种混淆时反而可能越容易受到干扰。 当然,目前还没有证据证明开发者一定是故意这样设计,也不能说明这种方法对所有模型都有效。 但它揭示了一个值得关注的趋势:随着越来越多的逆向工程依赖 AI,未来的代码混淆技术可能不再只针对编译器、反编译器和人类研究员,还会专门针对大模型的安全判断机制。 以前是让代码难以理解,现在甚至可以尝试让 AI 不敢理解。
46
227
21
2,826
92,645
I've been goofing off with how well Anthropic agents can perform various computational physics tasks. Models haiku, sonnet, and opus. All perform really well at the basic physics. But oddly, haiku seems to fail wildly at submitting the results in the required format.
1
33
or at least this is true at higher efforts. It's been really interesting to see how wildly effort level corresponds to not only errors but also token usage relative even to the smarter models. I guess this probably expected.
1
8
more soon, but here's some pre-registered hashes •commit 68e78561a6b5c27a53f1d7b31b4dfbeb71053b9d •design.md d5bd016147e320d45667a296bc911dc2a593d0647485db2c8b7ae880c77966de •config.yaml 53a944dad480f6b989285a3fbbdd501abea2265b51fd62850811c5bfecd11163
3
Sean Downes retweeted
Philosophers of physics constantly make this mistake. Physical theories have two parts. First is a mathematical structure, which needs to be rigorous and self-consistent. Second is an interpretive structure that tells us how to map the mathematics to physical systems. This second piece need not be axiomatic, it need not gave a coherent ontology, it doesn't even really need to make sense, as long as it correctly describes the outcome of experiments. The Schrödinger Equation is an example of the former, the Born Rule is an example of the latter. Another example is the mathematical structure of spacetime vs the assignment of causal structure and the arrow of time. Is the block universe real? Who cares? I tried bringing this up at the "Nature of Time" workshop at ASU last week, and had a bunch of philosophers get all condescending, as if physicists don't know how to do physics, and need philosophers to tell them how to do it. Except the philosophers are all stuck doing "foundations" of quantum mechanics like it's the 1940s, and doing quantum gravity with the Wheeler-deWitt equation like it's 1963. Completely unserious people.
Not a coincidence that physics being the most notoriously Kantian influenced of all the hard sciences particularly in the way quantum physics was developed by Heisenberg and Bohr is also unique among all the hard sciences in having incoherent theoretical foundations which are fundamentally and inexplicably incompatible with each other.
53
39
7
369
20,810
when you put birdseed out but its fall.
1
33
Sean Downes retweeted
I have a new book coming out this January: The Attack of the Science Populists, In Defense of Fundamental Physics. In this thread, I’ll explain why I wrote it. 1/13
13
35
9
165
17,036
I happen to be in the process of porting all my content to a new platform, but since it came up again I thought I’d drop some really ancient writing I did (backed up thank god): web.archive.org/web/20250514…
2
1
74
And if you like more mostly unsubstantiated ranting on this issue, you can read more here: open.substack.com/pub/pasayt…
1
1
38
But honestly, the take away that we need to rebuild a social esteem and public good that doesn’t rely on blind credentialism something else worth meditating on.
1
25
Hot take. Working from home was always a mistake.
1
43
Not that you can’t work remotely or whatever. But not from home.
30
Science feels like the closest thing we have to an objective sense of right and wrong, but even it needs us to assume and agree upon on something things. It’s harder but I like it that way.
I'm not an atheist, but I never understood why God is able to determine right and wrong. If God says "it's wrong to be gay", is it really wrong? Why? Sure, He can send you to Hell if He wants, but punishment doesn't determine what's right and what's wrong.
1
1
76
Reminding ourselves that reality is what we make of it feels empowering until the sheer weight of responsibility hits. I still prefer it that way.
26
Maybe. But in the mean time they seem to struggle with calculations involving quadratic polynomials, for instance working with representations of sp(2n,R). If anything it’s endearing to think that machines also struggle with moving too fast and making sign errors.
"Soon LLMs will do research-level math and mathematicians will be irrelevant." Me: Great, I can finally catch up on my sleep
53
If you understand the “let’s make a deal” strategy then you understand why this is gibberish. Don’t get zonked.
Roger Penrose calculated that the probability of a universe like ours arising by chance is on the order of 1 in 10¹⁰¹²³, an inconceivably small number but you're here
1
55