Sing Chen Yeo (@yeo_sing) profile picture

Data Scientist and cognitive scientist - Developing AI with emotion - Opsis Emotion AI

Singapore
Joined April 2019
Photo by Sing Chen Yeo (@yeo_sing)
Sing Chen Yeo retweeted
It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches, audio chunks, action choices, molecules, or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens), you can "throw an LLM at it". Actually, as the LLM stack becomes more and more mature, we may see a convergence of a large number of problems into this modeling paradigm. That is, the problem is fixed at that of "next token prediction" with an LLM, it's just the usage/meaning of the tokens that changes per domain. If that is the case, it's also possible that deep learning frameworks (e.g. PyTorch and friends) are way too general for what most problems want to look like over time. What's up with thousands of ops and layers that you can reconfigure arbitrarily if 80% of problems just want to use an LLM? I don't think this is true but I think it's half true.
549
1,156
236
10,407
1,284,656
Sing Chen Yeo retweeted
Excited to release Zephyr-7b-beta 🪁 ! It pushes our recipe to new heights & tops 10x larger models 💪 📝 Technical report: huggingface.co/papers/2310.1… 🤗Model: huggingface.co/HuggingFaceH4… ⚔️Evaluate it against 10+ LLMs in the @lmsysorg arena: arena.lmsys.org Details in the 🧵
18
136
18
570
183,561
Time to graduate! Welcome anyone who is interested in #sleep and #learninganalytics research to join my public defense. Please note that the date and time are in Singapore local time. nus-sg.zoom.us/j/88172127548…
1
Nice to see my research on @newscientist. University students with morning lectures tend to have lower grades newscientist.com/article/227…
1
Sing Chen Yeo retweeted
We are so pleased to be able to support 31 students to attend #LASI19! Thank you to our #LAK19 sponsors, @MHEducation, @CengageLearning, @CanvasLMS, @ACT and @Blackboard who play a pivotal role in providing this funding to expand the field of #LearningAnalytics! #ErikDuval
Arc of Enlightenment in a rational, civilized world: Data → Facts → Information → Knowledge → Understanding → Wisdom.
551
5,466
380
23,302
SCIENTISTS: “We’ve produced the first-ever image of a supermassive Black Hole, 55-million light years away” RESPONSE: “Oooh!” SCIENTISTS: “We’ve concluded that humans are catastrophically warming Earth” RESPONSE: “That conflicts with what I want to be true, so it must be false”
3,303
88,931
3,143
290,982