How to hunt down VM network timeout issues with performance charts
Setting up the performance counters in the vSphere client can help troubleshoot a dropped packets issue.
We moved an application -- running on Windows 2008 R2 -- to our vSphere 5.1 environment. While the performance is much improved, our users are experiencing timeout issues. Despite a lengthy application timeout setting of 60 minutes, the users have been dealing with timeouts after 10 minutes have passed. Using a network protocol analyzer, we're seeing the server issue a reset flag to the app. How can we correct this?
When network performance issues arise, it can be difficult to determine where the source of the trouble is, especially in a highly complex virtual infrastructure. Without diving deep into your system, it's hard to give you a straight answer. Some troubleshooting on your end will have to happen. But I can direct you to the tools you need to use to explore the system and come up with a diagnosis for these timeouts.
To start, I would suggest using the performance counters in the vSphere client to analyze the networking statistics of your host and VM. In the screenshot below, while the Performance tab can show packets being lost, it's difficult to force packet loss to happen and therefore determine how it's occurring. A packet can get dropped when there are networking problems or when the vmkernel is overloaded.
To set up the statistics, configure the chart options for your ESXi host to show real-time network counters to display when both transmit and receive packets have been dropped.
I would also suggest checking the usual suspects, such as duplicate IP addresses, and try to use another physical network interface card to connect the virtual switch to the physical switch to eliminate physical problems.
Dig Deeper on VMware ESXi, vSphere and vCenter
Related Q&A from Rob Bastiaansen
Is Kubernetes a hypervisor? How do they differ?
Kubernetes, on the surface, appears to perform the same task as a hypervisor. Does that make Kubernetes a hypervisor? The short answer: No. Let's ... Continue Reading
How do you enable centralized vCenter logging?
Centralize vCenter log files with vRealize Log Insight. Configuration with vSphere is simple and enables the centralization and transmission of event... Continue Reading
How do you reconfigure access to a VMkernel port?
Use ESXi firewall configuration settings and VMkernel ports to control the levels of access that different services, such as Secure Shell, have to ... Continue Reading