Comment by acedTrex
7 hours ago
"Lets have the system that fails sometimes that we are trying to ensure does not fail check it self"
This is such an unserious approach.
7 hours ago
"Lets have the system that fails sometimes that we are trying to ensure does not fail check it self"
This is such an unserious approach.
A separate model with separate context is used for review.
Like I said above, some people will never be happy with LLMs being allowed to do anything and nothing is going to make them happy about it.
It’s only fair to discuss what the real current status of these systems is. Every time I highlight that things are actually being done, the goalposts move again. There is no possible solution which will satisfy someone who has zero tolerance for letting an LLM execute tool calls because they will always find something.
> A separate model with separate context is used for review.
Thats fine, theres still a chance it fails.
> There is no possible solution which will satisfy someone who has zero tolerance for letting an LLM execute tool calls because they will always find something.
This is generally correct, security goes completely out of the window with this stuff. It will/currently is a security disaster and theres no actual solution to it.
"Let's make sure our model fails sometimes so that we can bill more for a second agent to validate, sometimes correctly, the work of the first model."
If you’re implying that the LLM companies are trying to train their models to make malicious tool calls so they can collect a few more tokens on the review, then I don’t know what to say. I guess threads like this are just a breeding ground for conspiracies now?
I think that solving a problem they have that has a side effect of generating more revenue is not lost on them. Particularly in the situation where the cost to improve the models goes vertical for minimal improvement in an asymptotic fashion. The decision to spend 6-7 figures regularly on training or bill every customer 3-4 figures per month isn't hard. While I don't think they will explicitly impair the models, they will certainly make choices that externalize the shortcomings of the model to their customers.
1 reply →