Time to roll up our sleeves and talk about memory, and in particular the sharing of memory between virtual machines. In many virtualization initiatives targeting existing environments the CPU utilization of the target servers is fairly low, particularly in Windows environments. In an earlier post I went into some of the reasons why this is the case, but this time I want to talk about how this may cause the emphasis of a virtualization project to shift toward memory.
Low CPU utilization often implies high stacking ratios, especially if the workloads aren't overly "peaky" in nature. But in practice this simply means that something else will become the bottleneck, and because it is uncommon for servers with low utilization to have high I/O rates, the next bottleneck is often memory. In general, the memory requirement for a process is not related to its activity level unless it is competing with other applications, a situation that is rectified at the OS level by memory paging. But in most steady-state production servers enough memory is added to prevent paging, thus giving the application a nice big space to spread out and laze around as it does nothing.
When stacking applications together in virtual machines, an opportunity exists to share read-only memory pages between VMs, but this opportunity will be squandered if they are virtualized the wrong way. Most users of Solaris 10 Zones are familiar with the "sparse root" model and the advantages it gives with respect to memory sharing. Simply put, the more similar you make each zone the more memory sharing will occur, and this is often a major consideration when determining what apps to put together on the same physical system.
What most people don't know is that VMware does this too, albeit in a different way. If VMs are sharing identical pages of memory VMware will eventually notice this (using checksums) and start sharing them. But this will only happen if the VMs are running similar stuff (i.e. the same OS, Rev, service pack, app revs), and if every VM is different then there won't be much to share. And the difference can be significant - over half the memory in use can be shared with the right configuration, effectively doubling the potential virtualization ratio for a given environment.
Unfortunately most Windows virtualization initiatives ignore this and focus primarily on utilization information. This tends to create configurations that have relatively low memory sharing potential (since this is not an optimization criteria), and consequently have lower virtualization ratios. By adding configuration uniformity as a criteria when analyzing servers it can have a surprising effect on the level of virtualization attained, and all it takes is a some foresight in the planning phase.
