1. The importance of cooling the world's supercomputers
particles - where science matters
  • The importance of cooling the world's supercomputers

    By Kelly Hall, 3M Storyteller

    Inside a data center

    • Supercomputer. You may have heard of it, but to fully explain what it is, we’ll need to start with the basics.

      A supercomputer is a group of computers that act as one collective machine, and it’s used to process enormous amounts of data.

      Picture a scientist conducting nuclear research or a meteorologist forecasting weather patterns. Now, think about how much data they need to do their jobs. In short, it’s a huge amount that requires a super-fast speed.

      While supercomputing designers began formulating their idea of supercomputers in the 1920s, the first official supercomputer wasn’t released until 1964. It was called the CDC 6600, and it came with a force to be reckoned with.

      It was about ten times faster than all other computers at the time – debuting with more than 100 miles of wiring and 400,000 transistors. Things haven’t slowed down since then.

    • Infographic comparing the different sizes of files

      Speeding up exponentially

      Today’s top ten most powerful supercomputers in the world reside in China, Switzerland, the United States and Japan. They process information billions of times faster than the CDC 6600 – at petascale speeds – with capabilities measured in terms of one quadrillion calculations per second. You can compare that to today’s average laptops, which operate at gigascale speeds, with one billion calculations per second.

    • “The amount of data we have, and the amount of people connected to the internet, is not slowing down.” - Lucas Beran, senior research analyst with IHS Markit

      Faster = hotter

      With that extreme speed comes extreme heat. Supercomputers can consume as much power as a small city. Studies show that data centers in the United States consume 2 percent of the country’s total energy consumption. Much of that energy is used to run chiller units and fans to cool computing hardware.

      Without an efficient cooling system, the monster machine supercomputers housed in data centers cannot operate properly.

      One common industry solution is to cool the servers through air-cooling. The downside of this approach? It can require a lot of energy to cool and flow the air across the server. It can also be corrosive to servers in places where air pollution is high.

      With our lives becoming increasingly data driven, data center engineers have joined efforts to find more efficient solutions to cool supercomputers.

      “The amount of data we have, and the amount of people connected to the internet, is not slowing down,” says Lucas Beran, a senior research analyst with IHS Markit. “It’s continuing to grow, and we need something different to help maintain and reduce the energy footprint of data centers.”

    • Closeup image of computer hardware immersed in 3M™ Novec™ Engineered Fluid

      An innovative way to compute

      Scientists at 3M are leading efforts to cool server equipment utilizing a different solution: liquid immersion cooling.

      With liquid immersion cooling, the liquid does the cooling passively, using only a fraction of the energy used in an air-cooled system. This can result in a significantly smaller environmental footprint.

      In total, the most efficient liquid immersion cooling methods can help improve data center energy efficiency by up to 97 percent since they eliminate the need for chillers and air conditioning units.

      How does immersion cooling work? The process involves putting electronics directly in a non-conductive fluid, allowing heat to be transferred directly from components into the heat transfer fluid.

      3M scientists utilize both single-phase and two-phase immersion cooling techniques. In single-phase immersion cooling, the fluid has a higher boiling point and remains in its liquid phase throughout the process. Electronic components are submerged in a non-conductive bath filled with a liquid. The heat from the chip is transferred to the fluid. Then, the heated fluid is pumped to a heat exchanger, where it is cooled and cycled back into the bath.

      3M scientists also utilize a passive, two-phase immersion cooling (P2PIC) process, where component racks are submerged in a bath of 3M™ Novec™ Engineered Fluid.

      Phil Tuma, who works with the application of this technology at 3M, explains how it works. “Novec fluids remove heat through direct contact with the chip or other heat source, which causes the fluid to boil and capture the heat as vapor,” he says. “The vapor generated rises from the liquid to a condenser coil, and then falls back into the bath. No energy is required to move the vapor, and no chiller is needed for the condenser, which is cooled by facility water supplied by a dry cooler.”

      The Novec fluid cools while maintaining a thermal Power Usage Effectiveness (PUE) under 1.02, meaning that less than 2 percent of the electrical power used by the IT equipment is needed to cool it – well below the government standard of a PUE of less than 1.5.

      It has zero ozone-depletion potential and is designed to be nonconductive and nonflammable. The chemical makeup of the fluid also allows it to be low in toxicity and to have a low global-warming potential.

      Because of its efficiency, the immersion cooling fluid allows a tighter packing of components, reducing the amount of floor space required. It enables 100 kilowatts or more of computing power per square meter, compared to just 10 kilowatts in a typical air-cooled system. This means that places like data centers could be housed in 10 times less floor space.

    • Photo of supercomputer immersed in immersion cooling fluid

      Bitcoin cooling

      One industry with a need for effective cooling solutions for high-performance computing is the blockchain transaction industry.

      “Bitcoin mining is a very cost-sensitive industry. The profits are dictated by the costs of the hardware, the cost to deploy it and the cost to operate it,” says Phil. “So, energy efficiency is very important, because it dictates the amount of energy required to cool the hardware.”

      The BitFury Group, a major bitcoin blockchain infrastructure provider and transaction processing company, wanted to embrace a solution for this need when they embarked on deploying one of the world’s most powerful two-phase immersion cooling projects for their new data center in the Republic of Georgia in Eastern Europe.

      Their data center houses 40 megawatts of processing power – about the average power used in 40,000 homes.

      To pull it off, BitFury acquired Hong Kong-based engineering company Allied Control, which specializes in 2PIC using 3M’s Novec fluid.

      Kar-Wing Lau, CEO at Allied Control, says that Allied Control’s system uses less than 10 percent of the electricity that would have been required for a conventional air-cooled system. “With 250 kilowatts per rack and not having reached the limits yet, we now have the ability to grow quickly and spread the cost of the cooling system over many hardware generations.”

      Allied Control’s success with this technology inspired BitFury to utilize Novec fluid for their data center project.

      The results may help to support the next generation of supercomputers, which are only gearing up to get even faster. The hope is that immersion cooling technologies will benefit high-performance computers evolving toward a national exascale system, which would process information at speeds of a quintillion calculations per second. Can’t picture it?

      An exascale computer would be a thousand times as powerful as today’s fastest supercomputers.

      EXPLORE DATA CENTER SOLUTIONS