Comment by iwontberude

2 months ago

And this is the investment case for AMD, models fit entirely in a single chassis, and side benefit: less tariffed network equipment to interconnect compute. Map/reduce instead of clustered compute.

Edit: when downvoting, please offer some insight why you disagree

34 comments

iwontberude

dragonwriter 2 months ago

How is the a unique advantage for AMD?

latchkey 2 months ago
AMD is consistently stacking more HBM.
H100 80GB HBM3 H200 141GB HBM3e B200 192GB HBM3e MI300x 192GB HBM3 MI325x 256GB HBM3e MI355x 288GB HBM3e
This means that you can fit larger and larger models into a single node, without having to go out over the network. The memory bandwidth on AMD is also quite good.
- ryao 2 months ago
  
  It really does not matter how much memory AMD has if the drivers and firmware are unstable. To give one example from last year:
  https://www.tomshardware.com/pc-components/gpus/amds-lisa-su...
  They are currently developing their own drivers for AMD hardware because of the headaches that they had with ROCm.
  
  26 replies →
- krapht 2 months ago
  
  So the MI300x has 8 different memory domains, and although you can treat it as one flat memory space, if you want to reach their advertised peak memory bandwidth you have to work with it like an 8-socket board.
  
  1 reply →
- dragonwriter 2 months ago
  
  MI355X isn't out yet, and the upcoming B300 also has 288GB HBM3e
  
  1 reply →

rixed 2 months ago

> when downvoting, please offer some insight why you disagree

And remind that (down)voting is not for (dis)agreement.