Empowering @SentientAGI builders, researchers, and the ecosystem in advancing open-source AGI 🤝
Joined July 2024
- Tweets1K
- Following74
- Followers27.1K
- Likes1.7K
no need to learn. no need to learn.
@sentient_zh and @Anitahyou got you covered!
Planted words then 📖
Planted fingerprints now
Models cost millions to train and minutes to copy. Fine-tuning and distillation can erase where they came from.
Introducing the Fingerprinting Leaderboard, an open benchmark for model provenance — built so the proof lives in the weights, not the paperwork.
fingerprint.sentient.foundat…
Before and after model fingerprinting
Models cost millions to train and minutes to copy. Fine-tuning and distillation can erase where they came from.
Introducing the Fingerprinting Leaderboard, an open benchmark for model provenance — built so the proof lives in the weights, not the paperwork.
fingerprint.sentient.foundat…
Robustness, Utility, and Verification
@qzxcle breaks down the three scores that make up @sentient_found’s Fingerprinting Leaderboard ↓
Why are there three leaderboard scores?
Model provenance carries clues about the history, training data, development process, and later modifications. Fine-tuning and distillation obscure provenance to the point where it’s virtually impossible to track the origin of the model back to its creator.
And if you as an owner want to reliably prove attribution and claim ownership of your open weights, you should choose a fingerprinting scheme based on these three dimensions:
>> Retention: whether the fingerprint is detectable without sacrificing normal model quality.
In other words, how embedding fingerprints affects model performance compared to the unfingerprinted model's nominal performance.
>> Robustness: whether the fixed attack can remove the fingerprint, preserving output quality.
In other words, how resistant the fingerprinted model is to malicious attempts to obscure its origin.
>> Verification: whether model owner/creator can reliably read the fingerprint back from a suspected copy and verify his IP property.
In other words, how reliable model attribution would be under scrutiny/inspection.
Release the weights.
Keep the receipts.
opening a model shouldn’t mean giving up the ability to prove it’s yours.
that’s the problem we’re tackling at @sentient_found.
this leaderboard tests how well model fingerprints survive attacks and how much model quality they preserve.
This is the missing piece for real open-source AI.
Training costs millions; copying takes minutes. Embedding provenance directly into model weights, rather than relying on paperwork, means models can remain open without losing their origin or credit.
The historical parallels are spot on, and an open leaderboard is exactly what this research needs to move forward.
A big step for open-source AI.
Kudos to the @SentientAGI for mmaking this possible.
an open benchmark solution to test whether ai models can prove where they come from 🤗
as AI models become easier to copy, fine-tune, and distill, proving where they came from matters more than ever.
renaissance painters left hidden artist marks on their masterpiece and now, open source ai creators and builders should have a way to mark the intelligence they build & prove where it came from. 🤗
Now, Chinese AI creators and builders (too) can verify their models’ provenance and prove where the intelligence they built came from
模型训练可能耗资数百万美元，但复制它只需要几分钟
微调和蒸馏可能会抹去模型的来源痕迹
我们推出 Fingerprinting Leaderboard——一个用于测试模型来源证明的开放基准
我们的目标：让证明本身存在于模型之中
fingerprint.sentient.foundat…
模型来源的证明通常存在于模型之外的日志和元数据中
但这些痕迹可能随着复制、微调或蒸馏而消失
模型本身需要携带证明
这就是模型指纹技术
@SentientAGI Labs 与 @Princeton 的研究人员 @edoardocontente、@viswanathpramod、@sewoong79、@oleg_golev 和 @ZeruiCheng 在训练过程中嵌入秘密的查询-响应对（query-response pairs），以帮助识别模型的来源
The solution to model provenance starts with @sentient_found's fingerprinting leaderboard ↓
Nothing we ship slips past this Spark ✨
Congratulations @PreyWebthree, you're our Spark of the Month!
Here’s a few of his best tweets ↓
EvoSkill made it into Franklin Templeton’s latest report, and @PreyWebthree was ready with his take.
For him, the question is no longer which model is better, but what you can build on top of it ↓
xitter.cf/PreyWebthree/status/20…
The real power of open models may not lie in the model alone.
How well a model performs matters, but so does what is built around it: skills, harnesses, and mechanisms that allow it to learn and improve.
Franklin Templeton’s report highlights Sentient’s EvoSkill as an interesting example of this shift.
The idea is simple but powerful: agents can learn from their past tasks and turn that experience into skills they can reuse on future tasks.
This suggests that higher performance can be extracted from lower-cost open models through the systems built around them.
So the competition in open AI may not simply be about “which model is better?”
It may increasingly be about what can be built on top of the model.
@SentientAGI
@sentient_found
@SentientEco
