← Back to context Comment by Tuna-Fish 17 hours ago it's the widening (for example: fp16 * fp16 = fp32) instructions, which are useful on some loads. 2 comments Tuna-Fish Reply augment_me 15 hours ago I assumed the comment above is referring to operand dtype and not accumulator dtype since they mentioned ops. In general even for nvfp4 you use fp32 accumulators because you are usually reducing over large Ks which mean large values.
augment_me 15 hours ago I assumed the comment above is referring to operand dtype and not accumulator dtype since they mentioned ops. In general even for nvfp4 you use fp32 accumulators because you are usually reducing over large Ks which mean large values.
I assumed the comment above is referring to operand dtype and not accumulator dtype since they mentioned ops. In general even for nvfp4 you use fp32 accumulators because you are usually reducing over large Ks which mean large values.