Comment by teaearlgraycold
7 hours ago
I don't think the other answers you're getting are right at all. AI isn't good at product/design decisions because it's not as easy to generate trillions of objectively right/wrong examples in those other domains. The training data isn't there yet.
Also, it’s a moving target. The training data must change every second. What worked yesterday will not work tomorrow.
These LLMs don't work such that training data === output data. You don't need to have 3+2=5 in the training data for the LLM to know the answer to that question.
Training data gives the LLM it's knowledge of the world, it can then bisect that space and output most likely answers.
If you are able to write a prompt such that you get the LLM to bisect the training data space, you will get the output that you want.
The trick though here is that guessing what is a profitable business venture is to some extent the same as trying to predict the future.
However, I would assume, that as more and more people start living more of their life via LLMs (i.e. asking where should I go for vacation, what are good shoes to buy, which is the most popular sports team near me) consumer choices will start to become more homogenous which would make it easier and more likely that for businesses to succeed they have to follow the LLM advice as that is also what the consumers are doing.