BBA + CS @ Yonsei Interested in data-driven product and AI strategy. Posting notes, code snippets, and readings. Open to research collabs.
Seoul, Republic of Korea
Joined September 2025
- Tweets227
- Following71
- Followers79
- Likes567
Yunmin Cha retweeted
New #NVIDIA Paper
We introduce Motive, a motion-centric, gradient-based data attribution method that traces which training videos help or hurt video generation.
By isolating temporal dynamics from static appearance, Motive identifies which training videos shape motion in video generation.
🔗 research.nvidia.com/labs/sil…
1/10
Yunmin Cha retweeted
i was made aware of miscitations thanks to the GPTZero team (cc @alexcdot). ji won and i quickly checked them ourselves and have posted what happened on openreview: openreview.net/forum?id=IiEt…. we have already notified NeurIPS'25 PC's about this issue.
i truly thank the GPTZero team for bringing this to our attention as well as raising the awareness of this serious issue (gptzero.me/news/neurips/), and at the same time i sincerely apologize to all for our error.
Yunmin Cha retweeted
nanochat d32, i.e. the depth 32 version that I specced for $1000, up from $100 has finished training after ~33 hours, and looks good. All the metrics go up quite a bit across pretraining, SFT and RL. CORE score of 0.31 is now well above GPT-2 at ~0.26. GSM8K went ~8% -> ~20%, etc. So that's encouraging.
The model is pretty fun to talk to, but judging from some early interactions I think people have a little bit too much expectation for these micro models. There is a reason that frontier LLM labs raise billions to train their models. nanochat models cost $100 - $1000 to train from scratch. The $100 nanochat is 1/1000th the size of GPT-3 in parameters, which came out 5 years ago. So I urge some perspective. Talking to micro models you have to imagine you're talking to a kindergarten child. They say cute things, wrong things, they are a bit confused, a bit naive, sometimes a little non-sensical, they hallucinate a ton (but it's amusing), etc.
Full detail/report on this run is here:
github.com/karpathy/nanochat…
And I pushed the new script run1000 sh to the nanochat repo if anyone would like to reproduce. Totally understand if you'd like to spend $1000 on something else :D
If you like, I am currently hosting the model so you can talk to it on a webchat as you'd talk to ChatGPT. I'm not going to post the URL here because I'm afraid it will get crushed. You'll have to look for it if you care enough. I'm also attaching a few funny conversations I had with the model earlier into the image, just to give a sense.
Next up, I am going to do one pass of tuning and optimizing the training throughput, then maybe return back to scaling and maybe training the next tier of a bigger model.
Yunmin Cha retweeted
🚨 This paper might be the bridge between logic and intelligence.
It’s called Tensor Logic, and it turns logical reasoning into pure tensor algebra no symbols, no heuristics, just math.
Here’s the wild part:
Logical propositions become vectors. Inference rules become tensor contractions. Truth values propagate as continuous operations meaning deduction and neural computation now speak the same language.
This isn’t symbolic AI or deep learning. It’s both.
Tensor Logic proves that Boolean reasoning, probabilistic inference, and even predicate logic can all be embedded inside a single differentiable framework.
Every major AI model today struggles with consistency and reasoning because logic is discrete and gradients are continuous. Tensor Logic erases that boundary.
In experiments, the system performs logical inference as matrix math, allowing neural nets to reason with symbolic precision — and symbolic systems to learn like neural nets.
If this scales, we might finally get models that don’t just predict truths — they can prove them.
The fusion of logic and learning just got real.
Paper: “Tensor Logic: A Unified Framework for Differentiable Reasoning”
Yunmin Cha retweeted
You know that frustrating moment when you're talking to an AI, and it almost gets what you want, but not quite?
You try to correct it—"No, make it more creative," or "Add some stats"—and it feels like you're talking to a wall.
Well, what if your corrections actually made the AI smarter? A new paper shows how. 🧵
1/12
For years, we've trained AI on massive, static datasets. Think of it like studying from a textbook. It's full of "correct" answers labeled by experts, but it's totally disconnected from how you actually talk and think.
This is why AI can feel so generic and impersonal.
2/12
But researchers at Meta & Johns Hopkins just flipped the script with a method called RLHI (Reinforcement Learning from Human Interaction).
Instead of textbooks, the AI learns directly from our messy, real-world conversations.
It's like learning on the job instead of just in the classroom.
3/12
Here's how it works. When you say, "That's not right, add more statistics," the AI doesn't just try again from scratch.
It creates a preference pair:
👎 The original, unhelpful response.
👍 A new response that incorporates your feedback.
It learns from the correction itself.
4/12
This is already a huge leap. The AI is learning to adapt in real-time based on your specific needs in that moment.
But that's not even the most interesting part.
What about making the AI feel like it actually knows you across conversations?
5/12
This is where it gets brilliant. The system creates a "user persona" by summarizing your entire chat history.
Do you prefer casual or formal tones?
Do you like bullet points or long paragraphs?
Do you ask for code, or for poems?
It builds a profile of your unique preferences.
6/12
Now, when you ask a question, the AI doesn't just give a generic answer.
It generates several options and uses your "persona" to pick the one you're most likely to prefer. It's aiming for personalized quality, not just general correctness.
(I know, right?)
7/12
Now, you might be thinking: "But my chats are messy and full of typos!"
The researchers knew this. A critical part of the system is a quality filter that sifts through the noise to find the genuinely useful feedback, so the AI doesn't learn bad habits from our chaotic conversations.
8/12
And it works. In tests, models trained with RLHI were significantly better at personalization and instruction-following.
They even got better at reasoning tasks just by learning from simulated users pointing out mistakes in math problems.
9/12
So what does this mean for you?
It means the future of AI assistants might feel a lot less like a clunky tool and more like an adaptive partner that learns your style.
Next time an AI seems to remember your preferences, this is the kind of tech making it happen.
Yunmin Cha retweeted
I'm at COLM! I'm doing:
- COCONUT poster (Tues 11am)
- Multi-Token Attention poster (Weds 11am)
- 🐏Organizing RAM 2 workshop🐏(Friday)
facebookresearch.github.io/R…
Reasoning, Attention & Memory – 10 Years On.
Invited speakers:
-Yoshua Bengio, Univ. of Montreal
-Juergen Schmidhuber, KAUST
-Kyunghyun Cho, NYU & Prescient Design
-Yejin Choi, Stanford & NVIDIA
-Azalia Mirhoseini, Stanford
-Sainbayar Sukhbaatar, Meta
Yunmin Cha retweeted
no more designing data intensive applications :(
This Post is from an account that no longer exists. Learn more
Yunmin Cha retweeted
“7M parameters achieve 45% on ARC-AGI”?
Not exactly.
1000× data augmentation
1000× ensemble voting
3.75× recursive compute
That’s 7M × 480,000 forward passes — not tiny, just time-hungry.
The result isn’t less is more, it’s more is more at test time.
Yunmin Cha retweeted
no, python is absolutely a bottleneck.
one of my (least?) favorite debugs at my previous job was figuring out why a background pytorch thread that deallocates memory mapped tensors was blocking the main process. it was the GIL.
github.com/pytorch/pytorch/p…
Yunmin Cha retweeted
Happy to share our 👼Angles don’t lie paper got featured by @DeepLearningAI Let’s chat more!
Researchers introduced GAIN-RL, a method that fine-tunes language models by training on the most useful examples first.
It ranks data using a simple internal signal from the model. On Qwen 2.5 and Llama 3.2, this method matched baseline accuracy in 70 to 80 epochs instead of 200 (about 2.5 times faster). This can cut compute costs and shorten iteration cycles for teams building math- and code-focused assistants.
Read our summary of the paper in The Batch: hubs.la/Q03M9ZjV0
Yunmin Cha retweeted
Excited to share our new paper, Router-R1, accepted to #NeurIPS2025
We teach a Large Language Model to reason about how to select and fuse insights from multiple other LLM APIs. It acts like a smart router, learning which specialist AI to call upon and synthesizing their collective intelligence. 🧠🔗
This allows for more robust, nuanced, and powerful reasoning by leveraging the diverse strengths of the entire AI ecosystem.
📜 Paper: arxiv.org/abs/2506.09033
💻 Code: github.com/ulab-uiuc/Router-…
Router-R1 just got accepted by NeurIPS'25 🎉
Agentic RL + LLM Routing → “Think–Route–Aggregate”
✔️ Policy LLM as Router
✔️ Cost Reward for perf–cost tradeoff
✔️ Multi-LLM coordination at scale
Big Thanks to all collaborators @taofeng_uiuc @youjiaxuan & See you in San Diego!
Yunmin Cha retweeted
🧵 (1/6) Bringing together diverse mindsets – from in-the-trenches red teamers to ML & policy researchers, we write a position paper arguing crucial research priorities for red teaming frontier models, followed by a roadmap towards system-level safety, AI monitoring, and oversight. Links to paper and our blogpost are in the thread.
Main position: LLM red teaming should prioritize: (1) product safety specifications with realistic threat models over abstract social biases or ethical principles; and (2) system-level safety over model-level robustness to most effectively mitigate real world risk.
Yunmin Cha retweeted
New @nvidia paper shows how to make text to image models render high resolution images far faster without losing quality.
53x faster 4K on H100, 3.5 seconds on a 5090 with quantization for 138x total speedup.
It speeds up by moving generation into a smaller hidden image space.
It means the model does not create the full detailed image step by step.
Instead, it first works inside a smaller hidden version of the image that has fewer pixels.
Because it processes this smaller space, there are fewer data chunks (called tokens) to handle.
Most models use 8x compression, this uses 32x or 64x, so far fewer tokens.
A direct swap is unstable because old and new latent features do not match.
They add a quick embedding alignment step that trains the patch embedder and output head.
Then they fine tune small LoRA adapters so the main weights stay mostly intact.
For FLUX guidance distilled weights, they fix the training target using outputs with and without text.
----
Paper – arxiv. org/abs/2509.25180
Paper Title: "DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space"
Yunmin Cha retweeted
Ettin mentioned 👀
Very impressive results with such small models - definitely take a look if you’re looking for VL models!
Yunmin Cha retweeted
Reinforcement Pretraining got a lot of hype but I'm annoyed that we're diluting the notion at this point. These are models pretrained with normal NLL on 20-36T tokens, which then get ≤1B tokens to predict with GRPO CoTs. It's really just another post-training stage.
[LG] RLP: Reinforcement as a Pretraining Objective
A Hatamizadeh, S N Akter, S Prabhumoye, J Kautz... [NVIDIA & CMU] (2025)
arxiv.org/abs/2510.01265
Yunmin Cha retweeted
imagine building a high traffic lookup service where every request hits a distributed cache, avoiding costly DB lookups. Sounds efficient?
But what about the cache misses? Each miss is a needless network trip, costing latency and resources at scale. Solution: Bloom Filters👇
Yunmin Cha retweeted
LLMs can learn long reasoning and composition tasks. The key is data mixture. Must have a mixture of short reasoning chains and long reasoning chains! One of the rare theory papers that directly sheds light on how data should be designed.
LLMs can solve complex tasks that require combining multiple reasoning steps. But when are such capabilities learnable via gradient-based training?
In our new COLT 2025 paper, we show that easy-to-hard data is necessary and sufficient!
arxiv.org/abs/2505.23683
🧵 below (1/10)
