Why Density per Rack is Going Up

Average rack density (the amount of power the equipment within a rack uses) has gradually been increasing over the past decade. While the rise was slow and steady, IT advancements are now rapidly pushing the average rack density up and threatening to disrupt traditional practices in data centers.

This article explains the reasons behind the recent increase in density per rack. Read on to learn what is causing this trend and what steps data centers will have to make to remain competitive.

A Steady Rise in Rack Density

For years, data centers housed equipment in racks that required 2 to 5kW of power on average. These setups were easy to accommodate with single-phase power and blown-in air for cooling.

However, recent IT advancements are increasing the demand for compute-intensive workloads that require more power. Some of these rapidly developing technologies are:

• Cloud services.
• Virtualization and containerization.
• Artificial intelligence (AI), machine learning (ML), and deep learning workloads.
• Internet of Things (IoT).
• Blockchains and cryptocurrencies.
• 5G networks.
• Edge computing.

These technologies require high processing power and high-density racks that go beyond the traditional 2 to 5kW average. As a result, enterprise and on-premises data centers are increasing the average density, a concern that was once unique to high-performance computing servers and hyperscale centers.

As data centers look to meet the growing demand, the average server rack (computer rack) density jumped to 8.4 kW per rack in 2020, up from 7.3 kW in 2019. Analytics and surveys run by the Uptime Institute report the following split:

• 46% of data centers have server rack densities in the 5 to 9kW range.
• 25% operate within the 1 to 4kW average range.
• 13% have rack densities between 10 and 19 kW/rack.
• 16% report 20kW or higher averages.

A closer look at the top 16% reveals the following densities:

• Over 6% of facilities have an average of 20 to 29kW/rack.
• 5% operates in the 30-39kW range.
• Just under 5% have 40 kW and higher densities.

As an additional indicator of the trend, over 45% of data centers expect their average rack density to be over 11 kW in the near future.

What is Driving the Rise in Rack Density?

Data consumption is on the rise as the use of cloud services goes up and new tech gains ground. The demand for compute processing is also growing as companies increasingly rely on power-hungry workloads such as:

• BI reporting and dashboards.
• Custom, ad hoc, and self-service analytics.
• Hybrid operational and analytic processing (real-time advising, real-time operational analytics, etc.).
• Transactional operational workloads (CRM, ERP, financial transactions, sales order entries, etc.).
• Data science and discovery.
• AI/ML and advanced analytics.
• Batch processing and HPC.

AI and ML are at the top of the causes behind the recent increase in average density. While AI and ML are still in relatively early stages, they already require high amounts of power.

AI/ML systems ingest large datasets, learn from them, and draw conclusions when they get new data. These systems require high levels of processing and are often coupled tightly with a single shared memory pool. As a result, 1kW/rack unit is the current standard for AI servers. A rack can have over 30 units, so setups typically run in the 20 to 40kW per rack range. Similarly, it is not uncommon for HPC deployments to reach the 50kW/rack mark.

Alongside AI data-crunching, there is an increase in advanced modeling and data analytics, which is another factor causing the need for higher-density racks. Other drivers of the trend are:

• Power-hungry government and military-grade apps.
• Demand for high-definition videos.
• Augmented and virtual realities.
• Intelligent transportation systems.
• The e-sports industry that requires high-performance and low latency.

IT Equipment is Also Pushing Rack Density Up

In response to the rising appetite for processing capacity, manufacturers are boosting the power of chips.

In the early to mid-2010s, mainstream server processors used under 100W, while dual-processor servers consumed about 200W at full load. Nowadays, Intel server chips (which are in more than 90% of server processors) go past the 200W barrier and bring the total average server consumption close to 500W.

Specialized chips are becoming even more power-hungry due to complex analytics and multimedia elements. The new types of chips must support capabilities such as:

• High parallelism (with potentially thousands of cores operating at the same time).
• Solving memory bandwidth problems to process large amounts of data (typically achieved by running a high number of operations in close proximity).
• Variable precision (using different quantities of digits in calculations).

These accelerator chips include graphics processing units (GPUs), field-programmable grid arrays (FPGAs), and custom application-specific integrated circuits (ASICs). These components require higher energy per chip than a standard CPU. For example, a GPU card can now draw about 300W, so each setup with three cards per rack unit draws as much as 1kW/rack unit.

A Challenge for Legacy Data Centers

Besides the increase in power-hungry processes, modern data centers are also accelerating the move towards higher rack density. Colocation providers see the trend as an opportunity to optimize resources as raising the density bar allows a data center to:

• Maximize the use of cooling by having more computing power in each cabinet.
• Free up square footage for other equipment (or reduce the overall size of the data center).
• Combine virtual machines onto fewer dedicated servers and reduce the number of necessary cabinets and pieces of equipment.
• Expand vertically instead of horizontally (up instead of out) and facilitate more users in the same space they already possess.

While some data centers embrace the change, some facilities will be under pressure to improve cooling and deploy new technologies. Changing operating strategies is a challenge, especially if a business with legacy equipment hopes to make a smooth, efficient transition at scale.

During the transition period, some data centers will likely offer higher density as an added service. This strategy will help balance out the costs and get some extra time to remodel the equipment.

Key Business Planning Takeaways

Current rack density averages do not suggest an immediate need for technical overhauls, but data centers should prepare for evolving consumer demands. Here are the key factors to consider during business planning:

• Customer expectations are changing. As companies develop software that requires more power, high density will eventually become the industry norm.
• High density per rack will become an even more significant consideration for data center design and capacity planning.
• AI and ML are at the core of digital transformation as companies look to better use data and improve decision-making. As AI/ML systems are most effective when running in proximity, businesses will have much higher needs for energy delivery per rack.
• Chips are becoming more energy-hungry. This trend, along with the increase in the use of accelerator chips, will make the shift towards higher density racks a natural choice when updating equipment.
• Standard air-cooling systems (heat sinks and fans) cannot efficiently control the temperature of the latest chips or large numbers of racks with AI/ML workloads. Facilities that offer direct liquid cooling (DLC) will house new chips more efficiently as this strategy can efficiently cool 50-100kW per cabinet.
• As the number of stacked equipment increases, racks will become heavier. Some facilities will require new racks with larger load ratings or floors that can withstand heavier loads.
• Cloud providers will have to adapt their operating strategies to deal with higher-density requirements.
• Some enterprises will no longer be able to support the required densities in an on-premises data center. There will be a rise in colocation hosting as setting up equipment in a third-party provider's facility becomes an even more sensible decision.

Average Rack Density Is Going Nowhere but Up

We expect rack densities to keep rising as data centers look to meet new customer demands.

The change will not happen overnight, but the industry is currently evolving in ways that favor higher-density racks. While there is still no immediate pressure, forward-thinking organizations are planning accordingly.