Comment by ozim

5 days ago

Key part is *where reliability matters*, there are not that many cases where it matters.

We tell stories of Therac 25 but 90% of software out there doesn’t kill people. Annoys people and wastes time yes, but reliability doesn’t matter as much.

E-mail, internet and networking, operations on floating point numbers are only kind of somewhat reliable. No one is saying they will not use email because it might not be delivered.

6 comments

ozim

Reply
abustamam  5 days ago

10% is still quite a lot!

Reliability matters in lots of areas that aren't war. Ignoring obvious ones like medicine/healthcare and driving, I want my banking app to be reliable. If they charge me $100 instead of $1 because their LLM didn't realize their currency was stored in floating point dollars and not cents, then I may not die but I'd be pretty upset!

  • ozim  4 days ago

    I was writing about Therac 25 that’s not war that’s medical equipment and code written by a human that killed people. Without LLM.

iugtmkbdfil834  5 days ago

<< 90% of software out there doesn’t kill people.

As we give more and more autonomy to agents, that % may change. Just yesterday I was looking at hexapods and the first thing it tells you ( with a disclaimer its for competitions only ) that it has a lot of space for weapon install. I had to briefly look at the website to make sure I did not accidentally click on some satirical link.

  • ozim  5 days ago

    Main point is that there is many more lines of code of CRUD business apps running on AWS and instances of applications than even non-autonomous car software even though we do have lots of cars.