← Back to context

Comment by gslepak

2 months ago

Note that these are Python-only results, the model will not do as well with other languages.

I'm glad to see more domain-focused SLMs, we need more of them! A programming focused MoE should work well across many languages.

9 comments

gslepak

Reply
rcarmo  2 months ago

If it writes functional Python instead of cosplaying as a Java programmer and cramming code with classes and accessors, it's already better than Opus...

nsingh2  2 months ago

Lots of confusion about what this model is actually focused on.

It is a cheap specialist for closed-world, verifiable reasoning tasks like math, self-contained coding problems, and similar.

"Closed-world" means the needed information is already in the context. It is not a tool-using agent that can discover missing context. "Verifiable" means answers are hard to generate but easy to check.

So no open ended research, repo wide agent work, factual Q&A, or SVG generation. More of a compact reasoning module for bounded problems.

  • nsingh2  2 months ago

    To follow up on this, I had it solve a nasty ODE problem that I saw in the recent Mathematica 15 release post:

        Solve the following first-order ODE for f(x):

    ((-1 - 2*x)*f(x)*tan(1 + x - exp(-61 - 2*x)*f(x)/x)
    + exp(61 + 2*x)*x*(1 - x*tan(1 + x - exp(-61 - 2*x)*f(x)/x))
    + x*tan(1 + x - exp(-61 - 2*x)*f(x)/x)*f'(x)) = 0

    Find the general solution f(x).

    And surprisingly it found a valid solution! Extra impressive because it runs 25 tok/s on my measly RTX 2070 super.

        f(x) = x*exp(61 + 2*x)*(1 + x - arccos(C/x))

    C is an arbitrary constant.

    Apparently Mathematica 14.3 couldn't solve this ODE.