Comment by judge2020

> someone says one of the benchmarks has a defense case where it says the "correct" lawyer-bot action is to is "snitch" on the client.

There are required exceptions to attorney client privilege[0]:

> the confidentiality rule is subject to limited exceptions. Paragraph (b)(1) recognizes the overriding value of life and physical integrity and permits disclosure reasonably necessary to prevent reasonably certain death or substantial bodily harm

An eventual lawyer-bot probably should be able to identify this, however this does pose a conondrum.

If the lawyer gets told by the LLM "the client's words indicate there may be a threat to life that requires you report them to the police", there could be various liability associated with either path of action.

If the lawyer reports them to the police, but then the client challenges that on grounds of attorney-client privilege being broken, is "the LLM said it qualified" a valid reason that'll prevent the lawyer from being found at fault?

If they lawyer doesn't report them, but then the client commits said murder, would the lawyer be liable if their LLM transcript were admitted into evidence under the theory of "you should've listened to the LLM telling you that you had to report it"?

This is an inherent problem with giving LLMs (or computers in general!) any say over anything that threatens someone's life or liberty, even with a human in the loop. Same applies to the potential for AI-driven upcoding in hospital environments where a doctor has to weigh potentially ignoring the AI and then missing some diagnosis (Safest path is probably to run extra tests to check if the AI's theory is correct, but then that's still de-facto upcoding).

0: https://www.americanbar.org/groups/professional_responsibili...