Comment by gitremote
5 days ago
Indeed. Animals without linguistic ability (like fruit flies) need sleep, but after ChatGPT's release in 2022, now tech bros think LLMs specifically might model the animal brain in general because of anthropocentrism and anthropomorphism.
It's also a fundamental misunderstanding of how LLMs work, mixing up inference with training.
Come on, don't be uncharitable, language isn't inherently necessary for models like LLMs, you can train something similar on visual inputs. Fruit flies have neurons that pass around ~probabilities/signal strengths to each other to represent their environments and basic concepts, it's not way off as an analogy.
It was applicable to all neural networks, not just LLMs.
Can we say that after ChatGPT's release in 2022, now antitech bros think everything is about LLMs specifically?
The statement was "AI frenzy almost convinced me that sleep was the training of our neural network with all the prompts of the day."
Prompts are specific to LLMs. Most neural networks don't have prompts.
Additionally, prompts happen during LLM inference, not LLM training. There are many non-technical people who claim they have experience "training" LLMs, when they are just an end user who added a lot of tokens to the context window during inference.
You're being pretty pedantic about the specific term used. Everything they said makes sense if you change "prompts" to "training examples" and you wouldn't expect someone who hasn't implemented an AI model to know the difference.
It's like someone said while driving the car "let's give it some gas" and you said "but the tank is almost full" when they obviously meant "let's press the accelerator pedal"
1 reply →
> Additionally, prompts happen during LLM inference, not LLM training.
It is pretty common during the fine-tuning phase.
1 reply →
> There are many non-technical people who claim they have experience "training" LLMs, when they are just an end user who added a lot of tokens to the context window during inference.
Since in-context learning is a thing, “adding tokens to the context window”, at least with the intent and effect of having a particular impact on capabilities when inference is run on the context to which they were added, is, arguably, a kind of training.