← Back to context

Comment by aeneas_ory

6 hours ago

Nobody is sleeping on anything. Linting for the most part is static code analysis which by definition does not find runtime bugs. You even say it yourself "runtime bug, ask the LLM if a static lint rule could be turned on to prevent it".

To find most runtime bugs (e.g. incorrect regex, broken concurrency, incorrect SQL statement, ...) you need to understand the mental model and logic behind the code - finding out if "is variable XYZ unused?" or "does variable X oveshadow Y" or other more "esoteric" lint rules will not catch it. Likelihood is high that the LLM just hallucinated some false positive lint rule anyways giving you a false sense of security.

5 comments

aeneas_ory

Reply
kaoD  39 minutes ago

OP isn't claiming all runtime bugs can be prevented with static lints suggested by LLMs but, if at least some can, I don't see how your comment is contributing. Yet another case of "your suggestion isn't perfect so I'll dismiss it" in Hacker News.

Why is this such a common occurrence here? Does this fallacy have a name?

EDIT: seems to be https://en.wikipedia.org/wiki/Nirvana_fallacy

normie3000  5 hours ago

> static code analysis which by definition does not find runtime bugs

I'm not sure if there's some subtlety of language here, but from my experience of javascript linting, it can often prevent runtime problems caused by things like variable scoping, unhandled exceptions in promises, misuse of functions etc.

I've also caught security issues in Java with static analysis.

  • aeneas_ory  4 hours ago

    The usefulness of using static code analysis (strict type systems, linting) versus not using static code analysis is out of the question. Specifically JavaScript which does not have a strict type system benefits greatly from using static code analysis.

    But the author claims that you can catch runtime bugs by letting the LLM create custom lint rules, which is hyperbole at least and wrong at most and giving developers a false sense of security at worst.

    • thunky  2 hours ago

      > But the author claims that you can catch runtime bugs

      I think you misinterpreted OP:

      Every time you find a runtime bug, ask the LLM if a static lint rule could be turned on to prevent it

      Key word is prevent.

      1 reply →