Jumbo Frames and MTU in Bare Metal Cloud

Introduction

Network performance is vital in Bare Metal Cloud (BMC) servers. It ensures optimal and efficient data transfers, resource utilization, and communication.

Two significant concepts in network utilization are the Maximum Transmission Unit (MTU) and jumbo frames.

This article will explain what MTU and jumbo frames are and how to view and modify the current MTU settings on BMC.

Jumbo Frames and MTU in Bare Metal Cloud

Prerequisites

What Is MTU (Maximum Transmission Unit)?

The Maximum Transmission Unit (MTU) is the largest data packet size (in bytes) that can be transmitted over a network without fragmentation. The packet includes the payload (data) and IP header related to the protocol.

Choosing the adequate MTU sizes has a significant impact on network performance and response time:

  • Smaller MTU sizes. The relative overhead size is large compared to the payload. Transmitting data requires more packets, increasing processing power and causing network delays.
  • Larger MTU sizes. The relative overhead is small compared to the payload. Performance improves due to sending more data per packet and processing fewer headers.

Typical MTU Values

The table below outlines typical MTU values depending on the network type:

Network TypeMTU Size (bytes)
Ethernet (standard)1500
Ethernet (jumbo frames)9000
PPPoE (Point-to-Point Protocol over Ethernet)1492
FDDI (Fiber Distributed Data Interface)4352
Token Ring4096
ATM (Asynchronous Transfer Mode)9180

Fragmentation

A packet larger than the network MTU size breaks into smaller IP packets. Fragmentation has several impacts:

  • Performance degradation. Fragmentation results in delays and requires additional processing power, decreasing the overall network performance and reliability.
  • Data loss. If a packet is fragmented and any fragment is lost, the entire packet requires retransmission. The mechanism results in inefficient network transmission and the potential for packet loss.
  • Increased overhead. Fragmented packets have headers for every fragment. The total overhead increases, reducing data transmission efficiency.

IPv6 does not allow fragmentation, and Layer 2 (data link layer) devices drop all frames larger than their MTU. On the other hand, IPv4 allows fragmentation on the Layer 3 (networking) level. If the IP header contains the 'Don't fragment' (DF) flag, the outgoing interface drops the packet instead of fragmenting it.

What Are Jumbo Frames?

Jumbo frames are Ethernet frames with an MTU size larger than 1500 bytes up to 9000 bytes.

Since jumbo frames carry more data per packet, they are beneficial for specific network environments, such as high-speed networks and those that require efficient data transfers for large amounts of data.

Jumbo frames vs. standard Ethernet frames

Environments where jumbo frames are an advantage include:

  • Storage Area Networks (SANs). Jumbo frames reduce overhead when transferring large data blocks between servers and storage arrays.
  • High-Performance Computing (HPC). HPC clusters handle large datasets and require high throughput. Jumbo frames reduce the number of transferred frames and improve network latency.
  • Virtualization. Communication between virtual machines (VMs) and VM migration benefit from the reduced network overhead with jumbo frames.
  • Data centers. High-speed connections are standard in different data center tiers. In these environments, jumbo frames improve networking efficiency and reduce overhead processing.
  • Backup and replication. Backup and replication processes transfer large files and datasets. Jumbo frames reduce the frames required to transfer data, resulting in faster transfers.

A critical factor when working with jumbo frames is that the entire network structure must support them, including adapters, switches, and routers. Networking devices that do not support jumbo frames lead to compatibility issues and performance degradation.

Modern network interface controllers (NICs), switches, routers, and operating systems support jumbo frames but may require manual adjustments. Implementing jumbo frames requires adjusting device and network interface MTU configuration to ensure support across the whole network.

MTU Configuration on BMC

Bare Metal Cloud (BMC) instances on phoenixNAP use jumbo frames as a default MTU value for public and private networks. The size aligns with the requirements and uses for BMC servers, resulting in network efficiency, improved resource utilization, and performance optimization.

BMC is a highly customizable solution, so the MTU size is not a fixed value. Adjusting the MTU is beneficial in some use cases, depending on the environment. The value is simple to check and adjust when troubleshooting network issues, and in some circumstances, changing the value ensures seamless connectivity with external networks, network segments, and services.

phoenixNAP recommends the following configurations depending on network type:

  • Public Networks. MTU of 1500 bytes is recommended on the server bond public interface. Since this network type is predominantly utilized to communicate to the Internet, MTU of 1500 bytes is ubiquitous in almost every use case. BMC supports an MTU of 9000 for this network type. However, phoenixNAP recommends not using this value if your public networks will talk outside of BMC unless path MTU discovery (and similar) are enabled in tandem with the 9000 bytes MTU setting.
  • Backend Networks. MTU of 9000 bytes is recommended on the server bond backend VLAN interface. If a gateway device, such as Netris, is utilized, the MTU decision depends on the configuration.

To view the current settings, follow the steps below for your server OS.

Windows Server

Access the server via a remote console and follow the steps below:

1. Search for Device Manager in the search bar.

Windows Server search Device Manager

2. In the Network Adapters section, locate the two Intel(R) Ethernet Controllers X550.

Device manager Network adapters Intel controllers

Repeat the following steps on both adapters.

3. Right-click the controller and select Properties.

Device manager Network adapter properties

4. Switch to the Advanced tab and locate the Jumbo Packet property.

Advanced tab Jumbo Packet property

5. The current value is 9014: the jumbo frame (9000) and the Ethernet header (14 bytes).

Enter the new number (including the Ethernet header value) to change the value. For example, to change to the standard Ethernet value (1500), enter 1514. Click OK to confirm the changes.

6. Repeat the process for the second network adapter and restart the server for the changes to take effect.

Linux Server

Use SSH to access the Linux server, and follow the steps below to view or edit the MTU value:

1. Run the following command in the terminal:

ip a | grep mtu
ip a | grep mtu BMC server terminal output

The ip command shows all known network interfaces, while the grep command highlights the MTU field from the output. The private and public interfaces contain the VLAN ID in the network interface name. The current MTU values are set to jumbo frames.

2. To change the MTU to a different value, run:

sudo ip link set dev [interface] mtu [value]

Replace [interface] with the interface name and [value] with the new MTU value. The change does not persist after restart.

3. To change the MTU value permanently, edit the following config file using a text editor such as Vi:

sudo vi /etc/cloud/cloud.cfg.d/50-curtin-networking.cfg

4. Locate the interface by VLAN ID and adjust the MTU value.

50-curtin-networking.cfg mtu value vlans

5. Save the changes and reboot.

Conclusion

This guide explained MTUs and jumbo frames and their role in BMC server networking. Continue testing and monitoring the network interface for compatibility issues and performance degradation. An optimal value will minimize fragmentation and improve network performance.

Next, see our comprehensive list of the best network bandwidth monitoring tools for Linux.

Was this article helpful?
YesNo
Milica Dancuk
Milica Dancuk is a technical writer at phoenixNAP with a passion for programming. With a background in Electrical Engineering and Computing, coupled with her teaching experience, she excels at simplifying complex technical concepts in her writing.
Next you should read
BMC Public IP Management via Portal and API
February 22, 2022

Follow the instructions in this guide to learn how to buy a public IP allocation, assign it to a resource and delete it via the BMC...
Read more
Configure BMC Server to Work with New Public IP Block
April 6, 2022

The BMC portal offers server instances and public IP allocations that cater to multiple...
Read more
How to Configure BMC Server After Adding It to a Network via Portal
July 26, 2022

This guide shows you how to add a BMC server to a network...
Read more
Netris for Bare Metal Cloud
April 13, 2023

This article provides an overview of the Netris solution and how to set up the Controller and SoftGate nodes in the Bare Metal Cloud environment.
Read more