Comment by Quothling

In the Danish public sector we provide services based on need assessments of citizens. Then we subsequently pay the bills for those services. Which amounts to thousands of small invoices having to be paid by a municipality each month. An example of this could be payments for a dentist visit, transportation and similar. Most of these are relatively small in size, and we've long since automated the payments of anything below a certain amount through automation. Systems which are faster and less error prone as far as putting valid data everywhere goes. They are more prone to decision making errors, however, and while fraud isn't an issue, sometimes citizens have invoices approved that they aren't entitled to. Since it's less costly to just roll with those mistakes than to try and fix them, it's an accepted loss.

The systems are hugely successful and popular, and this naturally leads to a massive interest in LLM's as the next step. They are incredibly tools, but they are based on probability and while they're lucky enough to be useful for almost everything. Decision making probably shouldn't be one of them. Similarly ML is incredibly helpful in things like cancer detection , but we've already had issues where they got things wrong and because MBA's don't really know how they work, they were used as a replacement instead of an enhancement for the human factor. I'm fairly certain we're going to use LLM's for a lot of things where we shouldn't, and probably never should. I'm not sure we can avoid it, but I wouldn't personally trust them to do any sort of function which will have a big influence on peoples lives. I use both Co-pilot and OpenAI's tools extensively, but I can still prompt them with the same thing and get extremely different quality outputs, and while this will improve, and while it's very to get an output that's actually useful, it's still a major issue that might never get solved well enough for what we're going to ask of the models way before they are ready.

I hope we're going to be clever enough to only use them as enhancement tools in the vital public sector, but I'm sure we're going to use them in areas like education. Which is going to be interesting... We already see this with new software developers in my area of the world, where they build things with the use of LLM's, things that work, but aren't build "right" and will eventually cause issues. For the most part this doesn't matter, but you really don't want the person designing your medical software to use a LLM.