Comment by NoDodgeQuestion
4 days ago
How can system be tuned for a specific model? Model is fungible, often one model strictly greater on both quality and price.
4 days ago
How can system be tuned for a specific model? Model is fungible, often one model strictly greater on both quality and price.
Think of it this way: you are at an enterprise business. You have a workflow implemented a year ago that is working just fine. Swapping out the model for a new one changes behavior in unpredictable ways. Eventually, you'll do it once cost is low enough, but it takes serious labor to validate this, so you'll wait a long enough time for Google to make money.
Prompts can certainly be tuned to a particular model, where updating the model actually results in worse performance. This is perhaps less true today than a year or two ago, but we have seen this on newer models as well. Typically, the less specific the instructions are, the less it's a problem. But sometimes you really need to get into specifics to get good results. Area of work is code porting and translation.
Models are not fungible, if you're building certain types of products on them.
this is becoming less true with every generation of model.
a decent model with a decent harness will determine when the knowledge base is lacking and attempt to fill the holes; thus the good general models can be very easily brought up to speed on niche domains.
Ugh? It is not just the knowledge
Some model are more aggressive by default, some are more verbose by default. To get the result you want for your specific application, you run experiment with prompts and parameters.
You're thinking like an engineer.
Think like a regulator.
The business model is to replace the entire human economy.