Replying to @⁨absquatulate@lemmy.world⁩

I don’t see a price there, so I’m not sure how “overpriced” it is. But it’s a class of product that is expensive across the board, compared to a typical desktop workstation or gaming PC.

The use cases are mostly local AI. That’s very demanding and requires compromises one way or another. You want high memory capacity and high memory bandwidth most of all. You can’t get both without paying out the nose, so you need to find the sweet spot for your specific use cases.

Discrete GPUs give you high bandwidth, but low capacity.

Standard desktop memory gives you high capacity, but low bandwidth (and also low compute speed since CPUs are no good for the workloads).

This type of product is a middle ground. Integrated memory gives you better bandwidth than DDR5, but not as good as a discrete GPU. Integrated GPU/NPU gives you much better performance than a desktop CPU, and can access the full integrated memory directly. Upgradeable memory is not really possible for this class of product.

The real competition for this is a tricked-out MacBook Pro. Those are faster, but they ain’t cheap, and I don’t think Apple even sells any 192GB Macs anymore, and when they did it was close to $10k.

Replying to @⁨AnAmericanPotato@programming.dev⁩

You also have the benefit that, since it’s a single memory pool, it can be used for other things, and allocated dynamically.

Where in a conventional machine, VRAM is exclusively for the GPU to use, and RAM is generally for the system. The capacities are usually fixed (I think it’s possible to offload GPU memory into system memory for some uses, but that’s about it, and your performance usually falls off).

Downside is that because it’s a shared memory pool, everything uses it, so you lose some of the actual capacity for things like your OS.

en