How do I benchmark network throughput between Amazon EC2 Linux instances in the same VPC?

Key Takeaways

[music] >> Hello, I'm Alwyn, a cloud support engineer here at the AWS office in Cape Town. Today, I'll show you how to benchmark network bandwidth between Amazon Elastic Compute Cloud Linux instances in the same virtual private cloud. Let's get started. To accurately measure network throughput between EC2 instances, we'll use the benchmarking tool iPerf3. We'll use two instances in the same VPC to test. First, let's make sure that we have two instances ready to test. As a best practice, use the same instance type and launch them in the same availability zone. To achieve the lowest latency and highest throughput, launch the instances in a cluster placement group. Also, verify that you turned on enhanced networking on your instances. Current generation instances use Elastic Network Adapter or ENA Express drivers to activate enhanced networking by default. ENA Express can boost single flow performance up to 25 gigabits per second. To check whether you turned it on, open AWS CloudShell and run the describe instances AWS command line interface command. Make sure to include your instance ID. As you can see, the output shows that enhanced networking is activated. The output shows true, enhanced networking is enabled. If you haven't turned on enhanced networking, then see the related link in the description of the video. Also, make sure that your security group allows traffic on the port that you'll be using for testing. For this demonstration, I'll use port 5201. Edit the inbound rules and add both TCP and UDP rules. Open the Amazon EC2 console. In the navigation pane, choose security groups and then select your security group. On the action tab, choose edit inbound rules. You can also choose the edit inbound rules from the inbound rules tab. Choose add rule and then enter custom TCP rule for port 5201. Set the source to the other instance security group or private IP address. Then add the custom UDP rule for port 5201 with the same source. After you're done testing, make sure to remove the port rules from your security group. Now I'm going to use Session Manager, a capability of AWS Systems Manager, to connect to the first instance in a separate terminal. This will be our server. Install iPerf3. The command depends on your operating system. I'm going to run sudo command for Amazon Linux 2023. Next, run this command to start iPerf in server mode, listening on port 5201. Now connect to the second instance. This will be our iPerf client. Run a sudo command to install iPerf on this instance, too. Then run a TCP throughput test with 40 parallel connections. In the command, include your server instance's private IP address. The test runs for 10 seconds and shows the bandwidth for each connection and the total. The sum line shows the aggregate throughput. For larger instances in a placement group, such as a C5N 18x large, you can tune the instance to reach throughput that's close to 100 gigabits per second. Smaller instances will show lower numbers based on their network allocation. Now, let's test the UDP performance. On the client, run a UDP test. Set the bandwidth target with the B flag, for example, 5 gigabits per second. UDP results show the bandwidth achieved, jitter, and packet loss percentage. UDP results depend on the bandwidth target that you set with the B flag. And you'll see jitter and packet loss metrics that TCP doesn't show. To get more detailed information, you can swap the client and server roles and run the test again because network performance can differ by direction. To improve your results, you can increase the MTU to use jumbo frames for instances that support them. Check the current MTU and note your network interface name. If it shows 1,500, then you can run this command to increase it to 9,001 for jumbo frames. In the command, include the network interface name that you noted. For jumbo frames to work, set the same MTU on both instances. The MTU increase is temporary and will reset on reboot. Thanks for watching and happy cloud computing from all of us here at AWS. >> [music]