RCC implementing liquid-cooled servers for High Performance Computing

In March, the Research Computing Center will deploy 28 liquid-cooled servers to the High-Performance Compute (HPC) cluster. This will be a first for the RCC and will put Florida State University at the forefront of data center infrastructure in Florida. Up until now, all servers in the HPC have been air-cooled.
Liquid-chilled servers are emerging as the solution poised to revolutionize the landscape of data centers and computing infrastructure. As the demand for higher computational power and energy efficiency intensifies, especially for high-performance computing applications, traditional air-cooling systems are proving to be increasingly inadequate.

There are many benefits to liquid cooling compared to the current air-cooled approach. The most salient is that it mitigates the load on the limited air-cooling infrastructure on the data center floor. Liquid cooling also improves CPU performance, energy and efficiency and allows for more density.
Modern CPUs with dozens of cores generate a lot of heat. For example, a four-core CPU might use 60-65 watts, whereas a 96-core CPU uses up to 360 watts. If both have a maximum operating temperature of 95 degrees F, the temperature sensor in the 96-core will throttle down to avoid overheating, thus reducing the computing capacity of the chip. Liquid cooling helps mitigate this issue, and it is especially salient in the world of HPC, where every CPU cycle counts.

The 28 Hewlett-Packard Enterprise servers that we are deploying this Spring have AMD Epyc 9454 48-core processors with 384 GB of RAM and 200Gbps InfiniBand networking, adding 2,688 to the HPC. They will be housed along with the rest of the HPC cluster at the recently renovated Sliger Data Center. As part of the renovation, several upgrades were made to the liquid-cooling system to prepare for the implementation of liquid-cooled systems.

Going forward, RCC staff plans to add more liquid-cooled servers to the HPC. Additionally, we plan to add a second chilled water loop to the Sliger Data Center dedicated to liquid-cooled systems.

Installation of the second chilled water loop in Sliger is planned to begin later this year (2024) and prep work is already underway. Additionally, we plan on a second round of HPC compute nodes this year.