Comment by zOneLetter
15 hours ago
Anecdotally, we use an LLM note-taker at work for meetings. I had to intervene recently because our CIO was VERY angry at our vendor for something they promised to do and never did. He wasn't at the meeting where the "promise" was made. I was. They never promised anything, and the discussion was significantly more nuanced than what the LLM wrote in the detailed summary.
In other cases, I have seen it miss the mark when the discussion is not very linear. For example, if I am going back and forth with the SOC team about their response to a recent alert/incident. It'll get the gist of it right, but if you're relying on it for accuracy, holy hell does it miss the mark.
I can see the LLM take great notes for that initial nurse visit when you're at the hospital: summarize your main issue, weight, height, recent changes, etc. I would not trust it when it comes to a detailed and technical back-and-forth with the doctor. I would think for compliance reasons hospitals would not want to alter the records and only go by transcripts, but what do I know...
I recently left my mom a voicemail saying happy Mother’s Day with normal human boilerplate of sorry I missed you, feel free to give me a call back tonight or we can talk tomorrow, either is fine by me whatever works best for you, hope we can talk soon, love you, bye.
She called me back later that night and we chatted for bit and then she paused and sort of uncertainly was like “So… was there something you were needing to tell me?” And I was completely baffled and was like “Uhhhh I don’t think so…?”
She then explained the notification she got about my call and apparently the LLM summary of my voicemail converted a message consisting of 75% well-meaning but insignificant interpersonal human filler (like most voicemails) into this stilted, overly formal business-y speak with a somewhat ominous tone. Assigning way too much significance to each of the individual statements in the message about wanting to talk (to say happy Mother’s Day), inquiring about her availability ASAP (to say happy Mother’s Day) etc. Plus grossly exaggerating the information density of the call making it sound like I left this rambling, detailed message about needing to tell her something that was left completely vague, but possibly important and also time critical.
Added up it made her a little worried when she read it and made me a bit pissed that was the end result of my wishing her well. Because apparently everything needs a half baked LLM summary crammed into it now.
What is a voicemail in this context? What app is reading it?
I’ve noticed my iPhone has recently started putting little AI summaries of messages on the notification screen.
Which reminds me, I need to figure out how to turn that off.
> I would think for compliance reasons hospitals would not want to alter the records and only go by transcripts, but what do I know...
Transcription is both too good, and not good enough. The magic generative content only makes it worse.
Too good: a lot of commercial settings forbid persistent transcription because it makes an easily discoverable record of specific details. Thats a business risk that can be mitigated simply by having participant notes or summaries where the secretary can omit sensitive discussion or present consensus without specifics. And notes/summaries also introduce a interpretive defense with some “strategic ambiguity.”
Not good enough: if you look at STT its still probabilistic. The actual evaluation output will have just much data about alternate words/phrases as the selected choice. That leaves lots of room for creating alternate impressions or representing words that werent actually spoken. The fact that people _think_ a STT transcript is authoritative only makes this worse.
When you add generative inference in top (eg summarization) you exacerbate both problems. I suspect that counsel is more accepting of summaries as its less likely to contain specific discoverable terms, likely to diffuse responsibility and specificity, and your judge/jury will be more amenable to “the ai summary is wrong” than “the transcription selected the wrong vowels.”
> I would think for compliance reasons hospitals would not want to alter the records and only go by transcripts, but what do I know...
I'm puzzled by this as well. Why not just generate a transcript and be done with it? If it's a particularly long transcript that's being referenced repeatedly for whatever reason let the humans manually mark it up with a side by side summary when and where they feel the need. At least my experience is that usually these sort of interactions don't have a lot of extraneous data that can be casually filtered out to begin with. The details tend to matter quite a lot!
I mean the reasons are the same AI is being pushed everywhere.
The businesses offering these services want to say "we are using AI" to their stake holders and the government committees who approve this shit don't have the skills or knowledge to evaluate the effectiveness in addition to the fact they likely don't even use the tools they have approved for use.
Every doctor's visit I've had, I have been able to make corrections to the record afterward, because there have been meaningful mistakes almost half the time.
ALWAYS check your summaries immediately, and contact your doctor ASAP. They can generally fix it themselves, and it's best done when everyone still has some memory of the event.
Transcription works pretty well in my experience, and the transcripts should be treated as the ground truth in such cases.