← Back to context

Comment by anonymous908213

11 days ago

> you want to write a handful of paragraphs and have a sota model vibe out a +1000loc diff

Wrong. What I want Luna to be able to do is ~deterministically make small edits or generations of 1~10 LoC according to a specification, and it literally cannot even do that. It violates specifications no matter how small you aim.

that doesn't even make any sense. deterministically generating 10loc based on a spec? you should be generating that code based on specific instructions of what you want those 10 lines of code to do, not based on some spec. and there is nothing deterministic about using llm's for codegen. i feel like you're just lost here mate.

  • The specification constrains the implementation details. Repeating the demonstrated example, again: filename must be lowercase is a perfect example of such a specification. You can have a hundred different tasks that require different things but all require that common specification.

    > there is nothing deterministic about using llm's for codegen

    And if you're okay with that, you're more like a farmer eating the slop meant for his pigs than an engineer. Using Sol Max, you can get near-deterministic output for a much wider range of tasks and specifications. Not all specifications; at a certain degree of sufficiently detailed specification it will begin failing again. But some, which is better than none.

    • i just asked luna to write a function that transformed a file path while upholding an invariant about lowercase file names. completed that task just fine. i see no reason it wouldn't work if i asked it for 99 other functions with that same constraint.

      again, you're expecting to use it without steering. that's not how you get the most out of the smaller models. i don't want to say you're using them wrong, but you clearly are. it could be deficiencies in your harness as well which the stronger models mask. try pi.

      4 replies →