Comment by aidenn0

I have never lived in SF but I have failed similar Chinatown-during-lunar-new-year tests in all 3 places I lived for more than a year. I have no doubt that an AI, particularly one that continually gets traffic update from some external source, would do better than I in predicting such hiccups. OTOH, an unconstrained LLM seems very likely to route me down streets that don't exist at least some of the time. If only we had some sort of database of actual streets and routes that were capable of checking the work of an LLM...

I see the same thing with LLMs in software development. If you say "find a bug in this code" it will regularly confabulate bugs. If you ask it for a test-case, run the output through some deterministic thing that tries the test-cases, and tells the LLM it's wrong, the output of that system will mostly be legitimate bugs[1].

For now, transformer-based generative AIs seem at a minimum like a very useful tool for dealing with "squishy" problems when you have some way to validate their output. Many of the 404's to the blog are probably people validating the output of generative AI, which is the opposite of the inference made in TFA.

1: It will also occasionally hack your test-runner; I suppose that's also finding bugs, just not in the software you wanted to find bugs for.