Comment by bArray
5 months ago
It's a nice project, but the LLM itself seems to struggle with actually comprehending the subjects. It can point me very well to parts of the paper, but it could not explain parts of the equations based on other knowledge outside of the paper.
I've had better results running it with the Gemini 2.5 Pro model but it's much more expensive. The website is using the very cheap 2.5 flash lite model.
You can run it yourself using the better model to see if you get better results as well:
https://github.com/montanaflynn/asxiv