Comment by mchiang 6 months ago OpenAI has only provided MXFP4 weights. These are the same weights used by other cloud providers. 4 comments mchiang Reply irthomasthomas 6 months ago Oh, I didn't know that. Weird! reissbaker 6 months ago It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4). irthomasthomas 6 months ago Interesting, thanks. I didn't know you could even train at FP4 on H100s 1 reply →
irthomasthomas 6 months ago Oh, I didn't know that. Weird! reissbaker 6 months ago It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4). irthomasthomas 6 months ago Interesting, thanks. I didn't know you could even train at FP4 on H100s 1 reply →
reissbaker 6 months ago It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4). irthomasthomas 6 months ago Interesting, thanks. I didn't know you could even train at FP4 on H100s 1 reply →
irthomasthomas 6 months ago Interesting, thanks. I didn't know you could even train at FP4 on H100s 1 reply →
Oh, I didn't know that. Weird!
It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4).
Interesting, thanks. I didn't know you could even train at FP4 on H100s
1 reply →