
Cognitive scientist @UCMerced // PI of Language, Interaction, & Cognition (LInC) lab: https://teapawt.org/t.co/rFNQ8DuaXq // Now mostly on Bsky: https://teapawt.org/t.co/E1BWpcYB5S
- Tweets700
- Following657
- Followers1.1K
- Likes4.3K

ALT At the top is the title of the paper: "Embers of autoregression show how large language models are shaped by the problem they are trained to solve". Below on the left is a screenshot of ChatGPT being asked to count how many words are in a list. The correct answer is 29, but it says 30. Next to it is a plot showing ChatGPT's accuracy at counting elements in a list; in general, it does well on multiples of 10 but poorly on other numbers. The explanation offered at the bottom of the image is: In training sets, round numbers are much more common than other numbers.




























