Hall of Hallucinations is up to 142 papers. Sign up, contribute a few of your own.
veruscite.com/hallucinations
Can see the post at veruscite.com/blog/veruscite…
I suggests a human review after my tool has flagged. Which is of course is so much faster than going through citations yourself one by one.
New VerusCite blog post, my false positive rate is under 1% in my benchmark and likely outside of it.
I go through my Grok Bot scan yesterday to show an out of sample example. Of the 200+ citations scanned, my tool flagged 10. Only one is potentially a false positive.
Replying to @sccarlson @akoustov
Daniel vs Alexander is a big difference, but again no way to know for certain unless the author admits it (which they never will).
If you would like to see additional examples of author hallucinations, one of my scans for the day found a paper with 4 examples, veruscite.com/share/Z_4zMYwH…. Those are more egregious but better examples of hallucinations.
A not uncommon pattern is to get the first author right (like the first example) but all the secondary authors are wrong. A few of the examples in that paper though just 100% of the authors are wrong.
Replying to @sccarlson @akoustov
I will never have a smoking gun. The design philosophy behind when to identify a hallucination is a whole cloth made up word -- a name, a title, a journal, etc. Minor errors (as you can see in the linked report), like a year off, a misspelling, etc. will not be flagged as hallucination.
In my public benchmark, I have a false positive rate of under 1%, github.com/apwheele/veruscit…, so I do suggest humans to review anything I flag after the fact. And of course identifying minor errors is useful for copy-editors as well.
Replying to @akoustov
My tool identifies just that citation as hallucinated FYI and no others, veruscite.com/share/vOGgnMBQ…
There are some legacy systems that are terrifying and I have no clue how they still manage to function (e.g. email). I am going to add 811 to that list.
Got an email from North Carolina that has a parcel map that is at least over 10+ years old.
It does work fast and gets the various companies out, but gosh that must be a tangled web to work on as a developer. (And the utilities must drop a significant amount of labor cost for something that should not be that hard.)
Replying to @ian_t_adams
I asked grok comparing to lotto like powerball. To reach that volume is closer to the monthly level for some of the bigger states.
Given you can bet for pennies on these ridiculous sequences and do it for so many things I imagine it is a better kick though for gamblers.
Damien Charlotin's AI hallucination tracker in court cases has flattened. Possibly via not collecting as many, but also just as likely the models are getting better. (I am skeptical individuals are using them more effectively!)
Replying to @CharlesFLehman
I have never used these, is this just a UI pattern? Like they just give a pop-up "Win 10 million dollars" for these random events coinciding. (Which does not go against your point, but just genuinely curious.)
Or are are people going in and making their own combinations?
presentofcoding.substack.com…
PILLAGE! Is Not a Management Strategy
DOGE, curiosity, and lost opportunities by Abigail Haddad
Abigail Haddad has a good blog post in response to the Glock piece on DOGE that I recommend folks read. Unlike private sector, it is hard to measure "how effective the gov is".
And she is pretty skeptical of Glock's claim that it is just as effective even after the exodus.
Not what Jukka meant, but there are actually more objective measures of quality in theatre that have no analog in Sociology -- like ticket sales and external reviews.
I used to build sets while in undergrad, so cost/time/ease to move are pretty objective for the carpenters as well.
Andy Wheeler retweeted
Anyone know how to get an org accepted on @pypi
Had a request sitting for months now and doesn't seem to be any way to contact/pay to do anything
😩












