
Multimodal Generalist, DS @HiLabs_inc | GSoC ’26 @GNUOctave | @iitroorkee alum | Explaining SOTA DL & systems | Constantly building, lifting, running.
Bengaluru, India
Joined June 2023
- Tweets2.7K
- Following1.3K
- Followers1.3K
- Likes10.6K
will be pre-training a sub 1B MoE model with each line of code being hand written, would be fun. will keep you guys updated
goal would to have this ready by end of 2026
5
28
1,245
completed a bare bones pre-tokenisation for training a sub-word tokenizer (BPE), would then work on binding, profiling using py-spy and writing parts in C++ for faster execution. Currently deubbing using the TinyStories validation dataset and then would take parts from smollm-corpus for pre-training the model
8
starting off with implementing SOTA tokenizers from scratch:
Tokenization: Modern LLMs use (Byte-level Byte pair encoding) BPE, instead of splitting by characters/words, we start with 256 raw UTF-8 bytes and iteratively merge the most frequent adjacent pairs into new vocabulary tokens
- original neural machine translation of rare words with subword units (2016)
- LMs are unsupervised multitask learners - GPT 2 (2019)
- SuperBE: Space Travel for LMs (CoLM 2025)
- Tokenizer from first principles: ggrigorev.me/posts/tokenizer…
- BPE explain: How GPT Tokenizers Work: blogs.gruheshkurra.com/blog/…
- Andrej karpathy's minbpe repo & walkthrough notes: github.com/karpathy/minbpe
4
162
A significant week ahead for the AI industry
Tuesday: NVIDIA reports its Q3 earnings, potentially marking the first earnings call without Jensen Huang.
Wednesday: Anthropic announces its IPO pricing
Thursday: $ANT begins trading on Nasdaq
Meanwhile, pressure mounts on NVIDIA’s board to bring Jensen back next week. A pivotal week for the future of AI.
Aman retweeted
Posting this blog at a very Chinese point of my timeline
I rebuilt DeepSeek V3 from scratch in 190 lines of pure JAX and trained it on my MacBook Air.
Wrote up the whole thing, every piece of the architecture explained with diagrams, code and real results:
vaibhaav.in/writing/deepseek…
Here’s the short version 🧵
Prompt-injection defenses are trained in English. Tiny Aya is built for everyone else. So does the defense survive translation?
I trained Meta-SecAlign into Tiny Aya Global using English injections only, then attacked it in 11 languages, scripts and code-mixes: Hindi, Urdu, Swahili, Tamil, Amharic, Hindi typed in Latin letters, Hinglish, Spanglish and more
What I found:
→ It transfers. Attack success drops from 51-90% to under 9% in every variant, including four it never saw
→ Languages add up. An attacker who tries every language on each case still gets through 1 time in 5
→ Low-resource data fixes most of that. Training on Hindi, Swahili, romanized Hindi and Hinglish too cuts it to under 1 in 10, with no loss in quality!
→ Scorers miss it. The standard AlpacaFarm-Hacked scorer ignores replies that open with **Hacked!** in bold, so non-English attacks look up to 58 points rarer than they are
Built for the @Lexsi_labs AI Safety & Alignment Hackathon with @Cohere_Labs and @huggingface, on top of Meta-SecAlign by @_Sizhe_Chen_ and @AIatMeta 🩵
Paper, code, models and data in the thread
9
35
864
Huge thanks to @_Sizhe_Chen_, @arreqe_ai, David Wagner and Chuan Guo for Meta-SecAlign (@AIatMeta), the recipe and eval code this builds on, and to @Lexsi_labs for AlignTune, CuratorKIT and AuditKIT 🤗♥️
1. Paper + code: github.com/beingamanforever/…
2. Models: huggingface.co/collections/b……
3. Data: huggingface.co/collections/b……
3
66
Every language alone stays under 9%, but they fail on different cases, so together a fifth still break. Add low-resource languages to training and it drops to 8.7%
2
40
One recipe, three training mixes. Only the language of the injection changes.
1. Utility and security for every model: quality holds, attacks collapse
2. The same instruction in Hindi, Urdu, romanized Hindi and Hinglish: the script doesn't matter
3. The scorer gap, language by language
4. How each training pair is built
1
4
157
Aman retweeted
Jensen Huang, during the 2020 pandemic, casually pulling out an NVIDIA HGX A100 motherboard out of his kitchen oven.
amazing trolling
CEO of the Year has a nice ring to it.
Congratulations to AMD Chair and CEO @LisaSu on this well-deserved recognition from @YahooFinance! 👏
yhoo.it/4kdfEYI
2
215















