← Back to context

Comment by HenryNdubuaku

20 hours ago

thanks for these haha, you can actually edit the tools and/or their descriptions, the demo is just a "get started" preset. But still we do have room for reasoning improvement!

3 comments

HenryNdubuaku

Reply
IanCal  17 hours ago

What kinds of things do you expect to work?

Edit - I’m struggling to get anything useful. Reasoning is often utter nonsense and the actions are very often very wrong. To the point of seemingly needing very precise sentences to work at which point you may as well do regexes. Very simple things like clean one room then another with the vac fails.

  • HenryNdubuaku  17 hours ago

    Thanks for the feedback! Implications and relations are hard for the model to understand (things like go to the living room, then the kitchen, and back), so yes the cleanest use cases involve direct language. Reasoning isn't true reasoning in the way general LLMs do it, it is more like grounding for the model that it generates itself. This can often become nonsensical specifically when the model gets things wrong, providing signal to the confidence.