Comment by llm_nerd

2 hours ago

Apple's hardware does not support FP8 (neither the ANE NPU, or the new "neural accelerator" tensor cores), though the most recent variant supports INT8.

0 comments