← Back to context

Comment by bobmarleybiceps

18 hours ago

A recent review of mine had a LLM-ism at the very end, "would you like me to format this into a formal peer review report?" So they very likely copy-pasted their whole review :). I'm pretty down on academia atm ;-;

8 comments

bobmarleybiceps

Reply
Skyy93  17 hours ago

I think this is more of a systematic issue. I review now since 2-3 years, I do not get paid, which is fine. However, it takes always a huge amount of time without really having anything from it, but I do it because it is important work.

There now so many researcher that need to publish which explains the flooding, LLM only speed it up, so reciprocal reviews take place. So now you are forced to review and you are having less and less time. So it’s a natural choice for you if you already took an LLM to write a paper to use it to review.

Perhaps one solution would be much harder entry barriers, and enforcing some guidelines. For example that a supervisor can not have more than 5 papers and PhD students only need one real paper on a major conference/journal.

coolness  11 hours ago

Super unfortunate. On the other side, I reviewed four papers for a top-tier AI conference and three were clearly fully Claude generated, as in all text, figures, results, everything. Actual good reviewer time is wasted on such papers and your (i hope) human written good paper receives AI responses. It's a sad state of affairs for sure.

  • stingraycharles  7 hours ago

    I find it incredibly sad that this is what the world is becoming, and I mostly blame people for this, not the LLMs.

    It’s the same type of people that would have no issue letting an LLM open a pull request on GitHub wasting valuable time of other humans, and whatnot.

    I’m using LLMs all the time myself, but it’s so incredibly important to use it to improve the quality of your work, not degrade it. People seem to be totally oblivious about this.

    On the flip side, it does make it easier to recognize people who are wasting my time.

    • jamienk  3 hours ago

      I think we underestimate the psychology of this kind of use of LLMs. I don't think these people (students, academics, lawyers, etc etc etc) are all just lazy morons. I think that using an LLM gives a strong knee-jerk feeling of "OMG this is exactly what I have to say. This is MY idea, these are my thoughts, this is - in a real sense - MY writing!" The feeling floods you when you see the output being churned out, way before you you read the thing (if you ever do) - it's the initial *seeing* it. Then when you hand the generated text in, unread by you, you do not have the feeling of cheating, you have the feeling of having exercised your powers, pushed right to the edge of your expertise and thoughtfulness, you successfully overcame obstacles because of your experience and unique capabilities.

      This is not always the psychological situation, but I think it might be a lot of the time.

      I also think that until we recognize this, we won't be able to help people to not do it. Calling them lazy or being bewildered by them or feeling rage or contempt towards them or threatening them is not going to be practically helpful.

baq  14 hours ago

You’d better not look at the average CI pipeline in software shops then