← Back to context

Comment by mosselman

9 hours ago

It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.

7 comments

mosselman

Reply
earcar  9 hours ago

RubyLLM author here.

I'm not sure where you got that.

`chat.with_temperature(0.2)`

https://rubyllm.com/chat/#controlling-response-behavior

`chat.with_thinking(effort: :high, budget: 8000)`

https://rubyllm.com/thinking/#controlling-extended-thinking

Max tokens is the only one of your list that require provider specific params:

https://rubyllm.com/chat/#provider-specific-parameters

I'm one guy doing it for free. Happy to see your contribution!

  • mosselman  8 hours ago

    Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that.

    I will have a deep dive into which things I felt we needed to adapt per provider.

    I didn't mean to imply that you have to solve all of our wants of course.

    One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.

  • realty_geek  5 hours ago

    Well put!!!

    You work your arse off for free and the guy who made the disparaging comment didn't even bother to research to see if he had the details right.

    Hat's off to you Carmine for all your work. Many people really do appreciate it.

    • jaredsohn  2 hours ago

      FWIW, I don't think the GP post is disparaging (at least as I read it right now.)

      I think it is fair to list limitations from using a library that provides an abstraction; it can suggest why a tool isn't right for a person's use cases.

      But it also sounds like this API handles those pretty well.