Tag Archives: VMWare

vSphere 5 Memory Management

Written by William Roush on November 12, 2013 at 7:14 pm

A very informative video describing how the VMware hypervisor vSphere (AKA: ESXi) handles your virtual machine’s memory. The video below shows how to dig into your virtual machine’s memory usage statistics, read esxtop, and discusses how memory compression and Transparent Page Sharing (TPS) works!

VMWare – vSphere 5 Licensing

Written by William Roush on July 16, 2011 at 9:03 pm

We’ve been eagerly awaiting vSphere 5 for awhile now, and it has been less than a week since VMWare made their big announcement including new features, but more important to most of us at this time, their new licensing model.


Now we’re looking at these numbers for virtualization licensing costs:

  • $995 – per CPU + 24GB vRAM
  • $2875 – per CPU + 32GB vRAM
  • $3495 – per CPU + 48GB vRAM


This has sparked quite the uproar in the VMWare community, as of writing which includes a 28,000 view, 40 page threadnaught which includes complaints from clients and vendors about canceled orders and requests to migrate, and some of the largest responses their Facebook community has ever seen, to which VMWare almost makes fun of the situation with cute vRAM mascots.



What is vRAM?

VMWare’s responses have mainly been that we just “don’t understand” how vRAM works, it’s simple enough: how much RAM is currently being allocated to your VMs? Or if you’re still physical, how much RAM do you currently have in your machines?

Not that hard.


One of the biggest responses from VMWare has been repeating like a broken record is that “most of our customers will not experience an increase in costs”, however I believe they’re missing critical points.

Let me preface it with what we’re looking at personally as a company: We haven’t yet bought VMWare, we’re currently discussing with a VMWare partner to get new hardware together and plan a new setup, virtualizing two racks of servers.  We leverage RAM to help increase the speed of our SQL servers, so we’re looking at 24GB/36GB servers (x3). On top of that we’re looking at our partner for Disaster Recovery (DR) services.


We’re not concerned with our current setup, how much can we grow now?

One of the biggest benefits for virtualization is the ability to grow, in vSphere 4 we can purchase more RAM than we’ll consume being as it’s relatively cheap, and provision it to our cluster of servers as we see fit. VMWare even supports RAM hot plug, which makes this even more tasty!


Again lets preface this with a real world application of planning for growth:

I’ve proposed the possibility to leverage AMD’s 12-core Opterons (16-core Opterons are due to be out Q4 2011) and a quad socket board so that we can leverage a high number of cores per socket, and high memory density.  We can start out with 3×1 CPU boxes with 96GB of RAM. We can purchase 3 licenses for $10,485, we’ll have N+1 failover of up to 192GB of provisioned RAM with no over-allocation, 288GB of usable memory if we allow certain services to fail when we go into a degraded mode.

Model CPUs vRAM Licenses Costs vs vSphere 4
v4 x3 512GB 3 $10,485 100%
v5 x3 192GB 4 $13,980 133%
v5 x3 512GB 11 $38,445 366%


So what we are to take away from this is on v4 I am not limited by vRAM, so I’m going to max it out at N+1 failover of 1/4 of the total RAM you can put in these boxes (to be fair and compensate for not allocating all sockets on the board). I can upgrade the box to 256GB of RAM before purchasing a new license. To cover our ability to grow in v4, our v5 pricing will be nearly 4 times as much.

Under the new licensing system, to just break even with our capacity of our N+1 failover, we’ll be looking at having to buy a 4th license, a 33% increase, and that hard limits us to no over-allocation, and no degraded mode.


Virtualizing memory intensive servers becomes way too costly.

Here we’re going to really hit home with how much this hurts, to virtualize one SQL server is going to use the vRAM of their cheapest license. So to virtualize one of our 3 servers is going to cost us $995.

Every time we go to provision a VM, we basically have to look at vRAM as a limit to what we can give it, and if that limit is reached, we must discuss with accounting, which is something we wanted to get away from having to do every time we provisioned a VM. Under the v4 licensing model I could easily allocate all 3 of these servers on a single socket 6 core Xeon with 64GB of RAM, with failover it’s still only 66% of the v5 cost.


Over-allocation of RAM is just a way for VMWare to charge us for hardware we don’t have.

One of the major benefits you get with VMWare is the ability to allocate more RAM than exists on your boxes, depending on the workload, this can work brilliantly in your favor. As certain services run through their steps they’ll allocate and deallocate a massive amount of RAM, the total RAM consumption can be higher than the total amount of RAM you have available on your system (this is standard for our development servers on VMWare), under the new model, VMWare is charging me for hardware I do not actually have!

This also goes for Transparent Page Sharing (TPS), basically memory deduplication which is another nifty feature of VMWare.


Penalize those with newer hardware.

There has been no talk about doubling vRAM allocation every 18 months (to keep up with Moore’s law), on top of that with the example we have above, we’re already highly penalizing anyone that purchases new equipment specifically aimed at consolidating workloads.


DR services, mostly pointless.

We were looking into purchasing disaster recovery services from a VMWare partner, primarily because of the additional hardware and licensing costs associated with spinning up a DR box offsite. However with the vRAM setup, we can have a standby box for next to nothing in licensing (well, in theory… the next issue destroys this).


Wait! We’re still bound by sockets?

Yes, we still have to pay per-socket, so we can’t even take the benefit of dropping the pCPU (physical CPU) model, which penalized those that were running older hardware that couldn’t consolidate as many cores per socket.


Home users get shafted too.

This one really hurts me personally too. Another employee at our company and I both run VMWare products at home, with the 8GB limit on RAM we’re limited severely on our home boxes. I’m pretty frugal with hardware and mainly virtualize Linux, but my coworker runs a lot of Windows boxes, and I can see how he’ll hit the 8GB limit quickly.

At which point, we both might as well just run XenServer, I’ll get additional hardware support (and no RAM limit when I need to allocate more than 8GB of RAM), and my coworker can get his additional RAM. When we both get familiar with XenServer, guess what we recommend at work?


No discussion about Moore’s Law.

Moore’s Law in pure simplistic terms says that computing capabilities will just about double every two years, we’ll need vRAM to rapidly run alongside hardware developments as 2TB, 4TB, etc. supported boards hit the server market and are easy to purchase. VMWare wont comment on this, and it’s hard to move an entire system over to VMWare when we don’t know if we’re going to get strangled quickly and are unable to use bleeding edge technology.


48GB for Enterprise systems?

This is assuming 4GB/VM per core on 12-core opterons, and we should be looking at consolidation ratios of 4-5VMs/core, meaning that I’m getting less than a GB of RAM per VM, I hope you’re running Linux!


Where do we go from here?

Personally I have no problem with a vRAM model itself, as long as it keeps the price the same for potential growth.

  • Drop the pCPU model entirely, if we want to go with vRAM it’s time to stop penalizing those running 4/6 core Xeons (I have my own speculation that the Intel/VMWare alliance is partially the reason for the move to vRAM away from pCPU).
  • Bump up vRAM 256GB/license for Enterprise licenses, this aligns with the current 4 socket 1TB boards on the market, keep up with the ratio for the newest available equipment.
  • To be somewhat fair, put caps on what each tier of licensing can get, if you’re allocating terabytes of RAM to machines, you’re an enterprise customer.

Now here is the painful part and the reason I know why VMWare doesn’t want to do this: this will reduce the cost for people that aren’t using their full allocation of vRAM. However there are three points to this:

  • XenServer and Hyper-V are quickly catching up with their feature sets, you need to be more competitive.
  • Most people will probably keep their vRAM over-licensing because they like the ability to easily grow, they will see this as money they’ve already got cleared with accounting, and that the freedom to grow upward is completely worth the cost. It will also allow us to cover our smaller systems (such as development) with vCenter when the cost cannot be justified under the pCPU model.
  • You’ll never be able to convince customers that giving them less at the same price is a deal.


I really do like the vSphere product todeath, but VMWare’s business descisions are scaring me away from it.