Comment by EdNutting
7 days ago
I think you missed the point. Yes it was meant to be humorous, and also to emphasise one of the reasons AI-generated citations are completely untrustworthy, especially with the growing number of AI-generated (junk) papers being published.
No, I had no intention of trying to offer a real source for the accuracy of AI generated citations. It is not hard to Google, search HN or even (ironically) use AI to search, to find numerous relatively recent studies discussing the problem or highlighting specific cases of respected journals/conferences publishing papers with junk citations.
It feels like allowing fake citations in the output from the AI means that you didn't do even the barest minimum of verification (i.e. tell the AI to verify it by sending a new AI to download the pdf that matches that DOI and verifying that it matches what the citation says).
Yeah I tried building such a tool. The problem was two fold:
1) Automated fetching of papers is difficult. API approaches are limited, and often requires per-journal development, scraping approaches are largely blocked, and AI- approaches require web fetch tools which are often blocked and when not, they consume a lot of credits/tokens very quickly.
2) AI generates so many hallucinated citations it’s very hard to know what a given citation was even supposed to be. Sure you can verify one link, but when you start trying to verify and correct 20 to 40 citations, you end up having to deal with hundreds or thousands of citations just to get to a small number of accurate and relevant ones, which rapidly runs you out of credits/tokens on Claude, and API pricing is insane for this use-case. It’s not possible to just verify the link, as “200 Status” isn’t enough to be confident the paper actually exists and actually contains the content the AI was trying to cite. And if it requires human review anyway, then the whole thing is pointless because a human could more quickly search, read and create citations than the AI tool approach (bearing in mind most researchers aren’t starting from scratch - they build up a personal ‘database’ of useful papers relevant to their work, and having an AI search it isn’t optimising any meaningful amount of work; so the focus has to be on discovering new citations).
All in all, AI is a very poor tool for this part of the problem, and the pricing for AI tools and/or APIs is high enough that it’s a barrier to this use case (partly due to tokens, and partly because the web search and web fetch tools are so relatively expensive).
Interesting, tools like Zotero seem to have sorted out the pdf fetching (and metadata + abstract fetching even without institutional access to the pdf). Did you try building the fetching on top of that?
2 replies →