Comment by suriya-ganesh

this is not what they're claiming.

the reason why clinical trial exists, is to prevent reward hacking. either by organizations or LLMs. the reasoning about why a drug works is usually post facto discovery (if that happens at all). the R&D environment around drug discovery is hilariously lacking (my wife works in this space. sometimes my mind just blows hearing some stories.)

if the model comes with a ridiculously shaped drug that no human could understand. you could basically put through the clinical trial process (mouse -> apes -> RCT with humans.) and decide whether we want to continue or not.