← Back to context

Comment by anonymous908213

11 days ago

> underspecified prompts

This is the same stupid argument trotted out repeatedly in the thread, and it doesn't hold up to reality. Would you care to explain how adding specifications helps a model that demonstrably ignores specifications? A model that cannot even handle the trivial specification of "the filename must be lowercase" is not going to get better when you add more specifications it ignores. I would guess that you're actually significantly underspecifying relative to me. Perhaps you're happy with whatever slop you get and don't notice Luna's overwhelming incapability to actually provide what one requires, because you don't have requirements.

yes, your prompts are underspecified relative to the size of changes you are making. you want to write a handful of paragraphs and have a sota model vibe out a +1000loc diff, where it makes all sorts of decisions and architectural/functional choices for you (that you didn't even realise needed to be made). if you use a smaller model, it's just not going to be sufficient. you need to work more iteratively with the model, with more steering and more human judgement. astra/opus can do this relatively well in some domains, especially the relatively simple ones like web, agent harnesses, etc. but someone with intention and skill can make use of the less powerful models, it's just a different workflow (and one that I prefer, when I'm not being lazy).

  • > you want to write a handful of paragraphs and have a sota model vibe out a +1000loc diff

    Wrong. What I want Luna to be able to do is ~deterministically make small edits or generations of 1~10 LoC according to a specification, and it literally cannot even do that. It violates specifications no matter how small you aim.

    • that doesn't even make any sense. deterministically generating 10loc based on a spec? you should be generating that code based on specific instructions of what you want those 10 lines of code to do, not based on some spec. and there is nothing deterministic about using llm's for codegen. i feel like you're just lost here mate.

      6 replies →