Comment by bjourne

Tooling mostly. To write fast code for CPUs you need a good optimizing compiler, like clang or gcc. Imagine how much work has gone into making them good. We're talking thousands of man years over several decades. You need just as good tooling for FPGAs and it takes just as much effort to produce. Except the market is orders of magnitudes smaller. You can also not "get help" from the open source community since high-end FPGAs are way to expensive for most hackers.

Intel tried to get around this problem by having a common framework. So one compiler (based on clang) with multiple backends for their CPUs, FPGAs, and GPUs. But in practice it doesn't work. The architectures are too different.