The "if" is doing a lot of heavy lifting. Can it do what it claims to do? LLMs are always hallucinating that they've found the most severe bug in software history.
Good point, I don't know! That's why I formulated it this way.
Edit: I guess what I'm trying to say is, with security relevant things it might do no harm to be a bit more credulous when it comes to initial reports. (In theory, no idea how to best handle the growing flood of LLM generated stuff. Pointing LLMs at each other doesn't seem great.)
> LLMs are always hallucinating that they've found the most severe bug in software history.
That is a function of the training data/method, like many of AI's problems. This doesn't stop it being a problem of course, but it shouldn't be a surprise.
Big bugs, like heartbleed, over the years have had breathy websites going over how serious the problem is because the problem was that serious, so the models associate that sort of wording with bug reports. Less serious bugs don't get the same excited treatment, repeated in many many places, so more measured descriptions figure in the training data relatively less.
mind boggles, issue is they're swamped with AI slop and high number of findings, they're using AI assistance operated by a 3rd party to handle researcher submissions so many legitimate findings slip through the cracks.
That's unfortunate. They need to do some kind of triage then, I hope this doesn't mean that they're constantly busy now with equally bad or worse bugs, but I guess it wouldn't be too surprising.
False negative? I honestly don't know what we can do about it. If this is a real issue, maybe another llm will rediscover it and it eventually will get triaged right?
Ironic comment from someone who literally just submitted an AI slop report and went so far as to give it a website and cutsey name to attempt to feign legitimacy and skill behind the "researcher."
The "if" is doing a lot of heavy lifting. Can it do what it claims to do? LLMs are always hallucinating that they've found the most severe bug in software history.
Good point, I don't know! That's why I formulated it this way. Edit: I guess what I'm trying to say is, with security relevant things it might do no harm to be a bit more credulous when it comes to initial reports. (In theory, no idea how to best handle the growing flood of LLM generated stuff. Pointing LLMs at each other doesn't seem great.)
> LLMs are always hallucinating that they've found the most severe bug in software history.
That is a function of the training data/method, like many of AI's problems. This doesn't stop it being a problem of course, but it shouldn't be a surprise.
Big bugs, like heartbleed, over the years have had breathy websites going over how serious the problem is because the problem was that serious, so the models associate that sort of wording with bug reports. Less serious bugs don't get the same excited treatment, repeated in many many places, so more measured descriptions figure in the training data relatively less.
Why are you believing an entirely Claude generated webpage?
A wordpress page would be more believable, right?
mind boggles, issue is they're swamped with AI slop and high number of findings, they're using AI assistance operated by a 3rd party to handle researcher submissions so many legitimate findings slip through the cracks.
As far as I can tell the next ASB will fix this issue... maybe they simply forgot to tell you :/
EDIT: actually already fixed in https://review.lineageos.org/c/LineageOS/android_frameworks_...
There's yet another PDU fixup that stems from negative index
That's unfortunate. They need to do some kind of triage then, I hope this doesn't mean that they're constantly busy now with equally bad or worse bugs, but I guess it wouldn't be too surprising.
They triaged them - that's why it gets wontfix.
False negative? I honestly don't know what we can do about it. If this is a real issue, maybe another llm will rediscover it and it eventually will get triaged right?
2 replies →
Ironic comment from someone who literally just submitted an AI slop report and went so far as to give it a website and cutsey name to attempt to feign legitimacy and skill behind the "researcher."