Sebastian Olschewski

Humans are more creative than LLMs on open tasks requiring imagination. A new fascinating paper compares humans with three LLMs on two creative tasks. The closed task: write a story incorporating ten specified words, such as “house”, “cow” and “planet”. The open task: imagine you could invent anything simply by thinking about it. What would you create, and why? Creativity was rated from 1 to 10 by human evaluators who did not know whether responses came from humans or AI. The results are super interesting: In the closed task, LLMs outperform humans on average. By contrast, in the open task, humans outperform all three LLMs. Why? To provide a first potential answer, the authors model imagination as the human capacity to generate ideas beyond what instructions prescribe. Then they estimate its contribution from differences between human and AI scores. Their model attributes between 22% and 45% of humans’ creativity scores in the open task to imagination. Humans may be more creative than LLMs thanks to their capacity to jump beyond what is prescribed. * Paper in the first reply.