Comment by SecretDreams
3 months ago
> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
Is this at all ironic considering we power modern AI using custom and/not non-general compute, rather than using general, CPU-based compute?
GPUs can do general computation, they just saturate under different usage profiles.
I'd argue that GPU (and TPU) compute is even more general than CPU computation. Basically all it can do is matrix multiply types of operations!