← Back to context

Comment by blinkingled

2 months ago

It's great to hear about $topic from someone no-nonsense and in-the-know like Greg KH. You can verify all of this too - since, well Linux kernel. (As opposed to what Microsoft or Apple claims to fix as far as LLM finds.)

Mythos may not be great today but it is not far fetched to imagine bug discovery, analysis and fixes can be made much quicker, accurate and even newly possible with specialized models trained on say Linux kernel specifics - with codemap/coding standards/threat models, good and bad coding patterns, tools to validate etc. an LLM can be much more relentless than humans and if it has the help to be accurate it will be worth the electricity burned. Oh and another model trained on triage data to validate the first one's findings would be good.

(I think Microsoft is doing this internally - different models trained internally alongside Mythos - there was some talk about it on the tubes, don't recall where exactly.)

> Mythos may not be great today but it is not far fetched to imagine...

I...

Look. Mythos was hyped up as the absolute best bug hunting tool ever made... no software was safe from its awesome bug-finding and exploit-writing capabilities. So strong was it that access _had_ to be limited to a select few pre-vetted entities, lest these awesome capabilities fall into the hands of Evildoers(!!!). Mythos' claimed capabilities were absolutely an important part of the "The LLM-based tools we're building are so dangerous that we must have new laws made to regulate us, or else all of humanity is likely to die!" story that the major LLM manufacturers have been building for a while and are telling now.

Now? Not even six months after release? "Well, yeah, okay, it's actually not that great. But imagine how great the next one could be!"... which is the story I've been hearing roughly every six months for what feels like five years now.

As an aside: I often wish we lived in a world where it was illegal for companies to use hype or any other types of emotional manipulation when advertising (or otherwise speaking in an official capacity) about tools that are to be used in a professional setting. Is it anything other than a bare statement of verifiable facts? Big fines, and repeat offenders get jail time. I know it's never going to happen, but it sure would be nice.

  • Every 6 months without exception people claim that the new generation of tools is absolutely incredible, and the old ones were total garbage, and that you only think they're crap if you were using the old tools. This just gets repeatedly memory holed again and again, and we're expected to always uncritically buy into the idea that they're actually good now against all evidence from the real world

  • Just for the 'record' - I am totally with you on the hype and just in general the normalization of sleazy behavior surrounding it but I'm not sure we as normal people have any say anymore including where our money is going to be invested.

    • > I'm not sure we as normal people have any say anymore [in regards to] where our money is going to be invested.

      Unless that was your money being invested, and it was a substantial fraction of the total pool of money being invested, was there ever a time when "normal people" had a real say in where the money was being invested?

      AFAIK, the only thing "normal people" can do is vote with their "feet" and pick a different prepackaged investment product, different investment company, or take their money and do the investment themselves.

      Honestly, this comment of yours seems a non-sequitur and doesn't really address anything I said... you don't have the power to bend investment firms to your whim, but that doesn't mean that you need to -knowingly or not- carry water for the major LLM manufactures by perpetuating the "But think of how great the tools will be in the future!" meme. It has been years now, and everyone who has been paying attention can say with confidence that the LLM-based tools of the future are never that great... they're often not useless, but they're not worth the billions of dollars that have been and continue to be poured into their manufacture.

      Related to your "We little people don't have any power anymore!" commentary, I note that TFA mentions that the kernel community has found that these LLM-based bug-finding tools have a false positive rate of ~50%. TFA goes on to mention that Coverity spent a huge number of years trying so hard to get people to buy its automated scanning software that had only a 20% false positive rate, and could not get enough people to buy the software.

      Coverity went under because everyone hated how stupid and annoying the tooling was... at a 20% false positive rate. Once the hype machine starts slowing down, no one working at the coal face is going to buy a tool with a 50% false-positive rate. I personally very strongly believe that even if the tools had a 10% false positive rate, no one would pay the actual price that OpenAI and/or Anthropic would have to charge to recoup the research and manufacturing costs of a cutting-edge LLM-based bug-finding tool.

      2 replies →

    • Don't lower yourself like this. The people working on this are "normal people" too. Don't put them on a pedestal. That's part of the problem here: lionizing people who are actively lying to others to protect their bags and reputation. Stop acting like a pawn for these people. Buying into their delusions of grandeur is what's perpetuating this mess. LLMs are a tool, not a monolith.

      6 replies →

His presentation style may be no-nonsense, but the content is brimming with nonsense. I would like to hear Greg Kroah-Hartman explain how the Mythos output was 'only 10 real bugs' but there were simultaneously over 1300 CVEs issued last month. It seems not much counts as a 'real bug' when an LLM comes up with it, but when it's time to bully distros into shipping an LTS release, anything goes?

  • What he means is that the headline-grabbing mythos output amounted to 10 actual bugs out of 79^H6 (and all of those got CVEs because that's how linux does it). The other 1300 CVEs came from other sources (the big increase likely being everyone else running LLMs through the codebase and filtering through the false positives). The Mythos output is mainly meant as an example of how even the top-tier models still have a high false-positive rate and that can be pretty tiring to deal with.

    I do think he repeats some myths in the video, or at least confidently states some things that are not demonstrated to be true, but his core point of 'you still gotta check these things' seems pretty solid.

    • > he repeats some myths in the video, or at least confidently states some things that are not demonstrated to be true

      I am genuinely curious what the myths/unproven things he states - I watched the video and it's repetitive sure but not much felt controversial to me.

  • Fair point, but he was talking about Mythos in particular and the point wasn't so much that LLMs will always have false positives rather he was saying they are causing a lot of them right now and how to deal with it.

    Also as other replies said Linux kernel process is to assign CVE to everything - some of them may be just DDOSes, very hard to exploit and everything in between. All of them are bugs so they all get fixed and it's not a bad thing if distros ship those fixes and people update their kernel.