Comment by meander_water
3 days ago
The problem with stuff like this is that it's hard to evaluate. You don't even know when the agent is using a skill, or if the skill even made a difference. Using tools lets you at least instrument tool calls, and control what gets executed.
I agree, I think traceability will be extremely important in evolving and improving a system like this. Since scripting is involved in searching for and managing skills, I feel like there is probably a way to achieve some kind of use tracing, but I'm not quite sure. Seems like this, if implemented, could also be fed back into the system for self improvement.