← Back to context Comment by wongarsu 9 hours ago Which conveniently fits on one 8xH100 machine. With 100-200 GB left over for overhead, kv-cache, etc. 1 comment wongarsu Reply storystarling 1 hour ago The unit economics seem pretty rough though. You're locking up 8xH100s for the compute of ~32B active parameters. I guess memory is the bottleneck but hard to see how the margins work on that.
storystarling 1 hour ago The unit economics seem pretty rough though. You're locking up 8xH100s for the compute of ~32B active parameters. I guess memory is the bottleneck but hard to see how the margins work on that.
The unit economics seem pretty rough though. You're locking up 8xH100s for the compute of ~32B active parameters. I guess memory is the bottleneck but hard to see how the margins work on that.