Comment by Tuna-Fish
21 days ago
Backprop has significant costs, if a local solution truly matches its performance, it will be dropped like a hot potato.
21 days ago
Backprop has significant costs, if a local solution truly matches its performance, it will be dropped like a hot potato.
Agreed, but how will we find out? Pre-training alone on frontier-class models costs billions of dollars.