Originally posted by: 4P_Bulldozer
The SSI Mid-Range E-Bay (SSI MEB) Specification (v 1.0) mentions a "Memory Riser Board" on Page: 11.
SuperMicro's newest e-Catalog (Mar 2011) shows a new Motherboard (the "H8QGL-6F") on Page: 4, it has "1 HyperTransport slot".
I have Googled furiously for "HTX memory card" with no results, there seems relatively few HTX Cards available, mostly concerned with interconnectivity.
Does anyone know of an "HTX Memory Riser Board" that would work with SuperMicro's "H8QGL-6F" (or any other Motherboard with an HTX Slot) that allows one to expand the Memory capacity of that Motherboard ?
They do not exist, and for good reason. One, the latency is much higher and bandwidth much lower compared to directly-attached RAM when accessing memory over the HT bus. That's why thread scheduling is so important to performance in NUMA systems like multiprocessor Opterons- if you make too many remote-CPU memory accesses, your performance goes to pot. You roughly double RAM latency (in 2P, it can be worse with 4P) and halve bandwidth compared with direct RAM access with the current HT 3.1 Opteron setups. Secondly, it would require a lot of custom logic to implement a memory controller/HT node in a 4P G34 Opteron setup that already has all of the coherent HT links accounted for. Latency would increase even more in that setup as it would require being routed through up to three other HT nodes to reach the memory on the riser card, as compared to a maximum of two "hops" in a current 4P G34 setup. The old 8-way Opteron 800/8000 setups were "three-hop" setups and latency and snoop traffic made their scaling pretty poor as compared to the "two-hop" 4P setups.
The memory riser cards were really only heavily used by Intel. They fell in love with serial RAM interfaces and single enormous memory controllers, both of which lent themselves well to using riser cards. The TYAN i5000PW
is an excellent example of that methodology. It has a great big old monolithic four-channel northbridge and each channel can run up to 8 serial-protocol FB-DIMM modules. The i5000PW puts 16 RAM modules in a big old double-sided riser card that sticks up like an overclocker's CPU heatsink. And like an overclcoker's heatsink, the FB-DIMMs on that riser card run scorchingly hot and throw off a lot of heat. I believe memory riser cards are also the rule with most Xeon MP setups as well. AMD's IMCs would not go all that well with riser cards as the trace length would get too long and RAM speeds would have to drop precipitously to allow for those long trace lengths.