Nicoconut

So my👇 thread about our papers investigating the verification and self-critiquing inabilities of GPT4 has apparently resonated with a lot of folks. Here is a quick response to several issues raised (either in replies or other quote-tweet threads). [Using one of 'em long "vanity tweets" 😛] 1. I am not an LLM luddite--I think they are amazing "idea generators" (in either language or code form). They just can't do their own planning/reasoning with any guarantees. So they are best used in LLM-Modulo settings (with either a sound reasoner or an expert human in the loop). Self-critiquing needs verification, which is a form of reasoning (..and so I was surprised at all the claims about LLM self-critiquing abilities). 2. This should be pretty obvious to anyone with Intro #AI background--unless folks writing papers making these claims have skipped that..? (c.f. teapawt.org/rao2z/status/168146127…). More likely, it happens because the authors anthropomorphize LLMs ("I mean, people do produce wrong solutions, and repair them themselves, right?). (c.f. teapawt.org/rao2z/status/169710160… ). Finally, the use of haphazardly curated Q&A datasets to make and evaluate self-critiquing claims has also, in my view, contributed to the confusion/conflation in the community (see teapawt.org/rao2z/status/171183590… ) 3. While LLMs can't plan, reason or verify, they can however help with those in LLM-Modulo settings We show a way of doing this for planning in a #NeurIPS2023 spotlight: arxiv.org/abs/2305.15771 (thread: teapawt.org/rao2z/status/166193833…) 4. Some have pointed out that while GPT4 cannot verify colorings, it can *help* come up with python code (especially given how common coloring problem code is in github 🙄), which once "corrected" as needed, can serve as the external verifier. Yes--but once again, it is the human in the loop which guaranteed the correctness of the verifier. Without that, there is no guarantee that verification amounts to much. 5. Continuing on 4, another of our #NeurIPS2023 papers (arxiv.org/abs/2305.14909) shows how a planning domain model can be teased out of an LLM (for any reasonably commonsense domain), refined with the help of humans and specialized external reasoners in the loop, and then thrown over to a plan verifier (such as VAL) or a domain-independent planner. This can then piggyback on the LLM-Modulo approaches mentioned in 3 above. (thread: teapawt.org/rao2z/status/166194489…) 6. [[Yet "band aid" to ensure that LLMs' verification capability is better than it is out of the box would be to *specifically* fine tune it for verification. We tried this in a previous paper teapawt.org/rao2z/status/166352362…. Note that this doesn't make LLM suddenly do better reasoning of verification--but just makes it better at faking verification (c.f. teapawt.org/rao2z/status/169992889…) Enough, at any rate, to make self critiquing work a little.]] (For further elaboration on all of the above, see the thread about a tutorial on the role of LLMs in planning at teapawt.org/rao2z/status/167942751…, or the CACM blog piece teapawt.org/rao2z/status/170161286…) Thanks for reading!