Comment by 9dev
11 hours ago
I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?
It's obviously not perfect, but it's not like you're going to YOLO it into prod without evaluating it on whatever you care about.
If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.
That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?
It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.