vSphere - Monitoring vSMP VMs (VMA & resxtop)

A server was built in the VI and I received the healthcheck email the next morning stating there was a 4 vCPU VM - which is unusual as the max vCPU we normally have is 2! The supplier was quite insistent that the server needed to have 4 CPUs.

The warnings VMware give with regards vSMP VMs is (as quoted from the VMworld 2011 VSP3866 session):

"vSMP VMs may not always use those vCPUs. Test your applications and verify that the threads for that application are being split among the processors equitably. An idle vCPU incurs a scheduling penalty.

Pay attention to the %CSTP counter on vSMP VMs. The more you see this, the more your processing is unbalanced. (The ESX 4.x relaxed co-scheduler is needing to catch all vCPUs up to a vCPU that is much further advanced.)".

In short - if there are too many vCPUs on a VM you could get the opposite affect to what you want.

How to check:

1. Log into the vMA using putty (or similar).
2. Set a target - in this case the Virtual Center server.

# vifptarget -s <vc-server>

3. Run resxtop.

# resxtop --server <esxi-host>

4. Login using the ESXi account when prompted (Host needs to be out of Lockdown Mode - Hosts are connected using AdAuth) .

5. Look for any increase from 0.00 in %CSTP.

6. Exit resxtop CTRL-C.

You can then decide what to do - add/remove vCPUs.

Description of %CSTP (extract from DOC-11812)
The percentage of time the world spent in ready, co-deschedule state. This co-deschedule state is only meaningful for SMP VMs. Roughly speaking, ESX CPU scheduler deliberately puts a VCPU in this state, if this VCPU advances much farther than other VCPUs. VCPU with high %CSTP is "stopped" from executing so that another VCPU in the same virtual machine could be run to "catch-up".

References:
VSP3866 VMworld 2011 sessions (USA and EMEA)

VMware communities interpreting esxtop 4.1 stats
http://communities.vmware.com/docs/DOC-11812

VMware communities interpreting esxtop stats
http://communities.vmware.com/docs/DOC-9279