Virtual machines for Programmers

Notes and links for a talk to Australian Delphi Users Group

Essential utilities to install on the virtual machine

TweakUI PowerToy for setting VMs to autologin to a domain

A decent text editor if you don't already have one: Notepad++ is free; Textpad costs money.

An archiver that opens most common formats and does more:

XP-Antispy disables lots of crud on any PC, and you want it to zero the pagefile on shutdown if nothing else.

Other things that might be useful:

Wikipedia has useful stuff on VMware and a comparison.

There's more on VMware at Chitchat and at the extensive VMware forums . There's more VirtualPC stuff at RobertMoir SJ and of course the blog of Microsoft's Virtual PC Guy (you'll see me hassling him there sometimes).

Invirtus VM Optimizer claims to do wonderful things and at least did no visible damage when I tested it. It looks as though it's primarily designed to make dynamic disks (that grow as you put stuff on them) suck a bit less by clearing free space and packing them down. For $US79 I think you might be better off with a decent defragger and sdelete, unless you buy into their magic claims about guest OS performance.

Remove Windows Messenger XP-Antispy will disable it to some extent, this will get rid of it (until the next critical update from Microsoft, of course).

lots of useful links from the blog of one of the VMWare team

Microsoft Bootvis officially doesn't speed up your boot and shutdown, but it might... get it from MajorGeeks anyway:

Super compression is not all its cracked up to be according to some people but use it anyway: www.7- 10% smaller than WinRar and five times slower to compress. But for putting a 20GB virtual disk onto one DVD... I care a lot. If you're in a hurry, crank the compression down and burn two DVDs. At least with 7-zip you can crank it up if you need to. 7zip is also multi-threaded, which most archivers aren't, and that makes a big difference to compression times. I did some tests on 7zip.

Differencing disks can eat you alive, and performance sucks according to some geek with a blog

XEN is a free virtualization system that is free and open source but requires the OS to be recompiled (so you can get BSD and some Linux flavours, but nothing from Microsoft until the chipsets support their approach (which is coming)).

Virtuozzo have yet another approach to virtualisation, where they use the host OS to do all the work and somehow virtualise on top of that. It means you only need one OS license, but I suspect it also only takes one problem to dump the whole lot. I only found out about this today but the downloadable white papers are interesting to read (you don't need to give them a real email address).

The Talk

My notes and references for this are on the web at I won't mention many sites because I assume you'll hit that page and follow the links if you are interested. There are links to FAQ's and other useful things there.

Virtualisation is basically simulating a PC in software, from the CPU upwards. Both VPC and VMware simulate a particular motherboard, video card and CPU, with optional stuff like USB and networking added. Both allow you multiple networking options, and can mount multiple virtual partitions, or a real partition. You can't run either inside itself, but apparently they will run inside each other.

Most of this talk focuses on using a virtual machine for software development, where performance is important and you'll be using fairly large files that change a lot (every time you compile you make a big steaming puddle of object and executable files). If you follow this you'll end up with a fairly fast VM that will work fine for testing, but for demoing software or similar install once then don't change it uses you could ignore much of the tweaking).

VMware has been around for a long time (10 years???), while VPC was until recently a Mac-only thing. Microsoft have bought it and are talking about embedding it in Windows to handle tricky stuff (backwards compatibility, I presume). VMware just released a free player that will let you run images from either program, but you can't change the way they're set up using it.

In use they're very similar. The vast bulk of what you do will be identical for both of them. The key differences to me are that VPC has no USB support, and doesn't deal well with the display. Either can full-screen, and both only support one (virtual) monitor. One big difference in usage is how many monitors you have - I find that on a single-monitor machine I tend to maximise the VM and leave it that way, while on a multi monitor machine the non-maximised modes are really useful.

You can resize that monitor to cover all of your desktop if you have multiple real monitors. VMware win in this area because you can edit in "fake" resolutions and use their window-shade mode to cover your available desktop rather than going full-screen. VPC puts up scroll bars if you get too close to full-screen, and that sucks.

The reason to avoid full-screen mode is that on a multiple monitor machine you only get one monitor. You can move the mouse outside the virtual screen, but clicking causes problems. VMware drops out of full screen mode, while VPC blacks the virtual screen and puts the mouse cursor in the middle of it. This is annoying! With the Linux version of VMware you can fiddle Xwindows to get a single desktop and maximise to that. Some video cards allow the same trick under windows.

Both allow you to run multiple VMs at the same time, subject to physical memory constraints and licensing requirements. Microsoft are very nice about server operating systems, but when it comes to the desktop they are adamant that you buy as much of everything as possible. I think they're saying that you must have one license per installed virtual (google translation) machine (either brand), but not so clear on what happens when you copy one. They have apparently also declared that you can run one virtual copy of the OS on the physical copy of that OS (in the middle of a 45 minute MS-TV interview) but their lackeys deny everything. I have no idea whether making five copies of your VM but only running one at a time is legal and apparently neither does anyone else. The public newsgroup is also not very helpful. I'm going with the "buy one license per running copy" approach.

Since XP is activated already it's hard for MS to know that you have multiple copies unless you run all of them at the same time. Windows Update seems happy to repeatedly update "the same" VM. Note that MS and others also sell connection licenses, and your VM may well need some. Shared networking can save you from some of them, but you might just have to cough up or do without your server connections. The amusement comes when you compare this with companies like TextPad, who have a "per meat puppet" license option that lets me install and run as many copies as I like from the one license. Or I count as one user on a site license.

Speed is the big issue. Virtual machines are slower than real ones, and do not get the whizzy multimedia CPU instructions or benefit from your 3D video card. For this reason, turn off all the glitter and run the "windows classic" interface. That speeds you up noticeably. You gain a further benefit from having the two OS's on different hard disks, as OS's tend to assume they have a disk to themselves. Swapping on a VM is not a good idea - it's quite slow. So swapping hurts even more than you're used to. I run 2GB both at home and work, and allocate at least 500MB to a VM that's going to run any development environment. If you can get a dual core system do it - either two CPUs or a dual core one. Hyperthreading is not dual core, no matter how much Intel would like to think it is.

Virtual hard disks can be dynamic or fixed. Fixed disks are just like real hard disks - you get X much space and can partition it as you will. Dynamic ones grow as you add stuff to them, up to the limit you decide when you create them. So over time you get a virtual disk that's fragmented all over your real hard disk, and you get strange "out of disk" errors from some apps that are seeing lots of free space but can't write to it immediately. I'm sure there's a good reason for dynamic disks, but I don't use them.

On that note, it pays to defrag both your virtual disks and your physical ones. I use both a bought product, and the free "contig" utility from that works on single files. That's useful when you just want to fix one 20GB virtual disk image.

Both products let you suspend a virtual PC to disk. So never use the OS hibernate function, the virtual version is a lot better. I actually have a couple of lightweight VMs purely to bookmark help files with. You know how you dig around in help, and end up with 8 pages open but you can't save that state? A suspended VM does that trick nicely, and Win95 with 32MB of RAM and 500MB of disk will run most help systems just fine. The .NET SDK help, for instance.

Networking is potentially exciting. When you copy a virtual machine you suddenly have two PCs on the network that have the same MAC and other hardware IDs. VMware detect this and offer to generate new ones for you, Virtual PC just run with it and you can get very odd network behaviour as a result. So when cloning VPCs, you need to edit their network settings (or disable networking).

When you're setting up the first VM, it pays to make a list of what you want it to do before you start. I'm assuming that this is for your development systems, so you'll want a series of images. Start with XP/2000, install that and activate it. DO NOT install the VM tools if you ever intend trying the other brand VM, as they're hard to get rid of and will stop the other brand booting. Throw in the 5 essential VM tools and defrag the hard disk. Now shut it down and archive it off.

By itself that will 7z down to under a gig. Fire it back up and install the VM tools pack, then set up the appearance - you want non- traditional colours for the title bars so you can easily tell the difference. Set the performance to best, possibly re-enabling smooth screen fonts. Run Windows Update and so on. At this point I normally disable networking and archive again. Then start installing your development tools, again archiving out when necessary - after installing and patching Delphi but before adding components is a good idea. The goal is that you can grab an image to install updates of anything on - Delphi, components, other tools, so you want a "clean Delphi", a patched Delphi and so on. Assuming you have a spare machine lying round, the cost of archiving out is quite low - 10 minutes of your time and blank DVD.

VMware warn you that changing the amount of memory can prompt you to re-activate, but I haven't had that problem despite tweaking it up and down quite a lot. Windows activation kicks in if you change three things, and normally you can work around that - to move from host A to host B with different CPU types you might disable networking (so the MAC address doesn't change), copy over, boot, shutdown, enable networking and so on.

At some point you will realise that your virtual disk is too small. You can either start testing your tools by installing them on a second virtual partition, or you can resize the disk. This means exactly the same thing as on a physical disk - you make an image of the disk and copy that onto a new disk. I use SysRescue, you could also use Norton Ghost or some other bought tool. The steps and sites are detailed on my web page above.

I've been running my virtual machines with 500MG RAM and no pagefile. This costs you a little memory because there are things that can swap, but it means Windows doesn't page stuff out in a desperate attempt to keep 300MB of free RAM. If there's anything that really sucks, it's swapping on a virtual machine.

Compressing VMs For Archiving

One thing that is annoying about this is the number of huge files you end up with as part of the "make a copy and play with it" strategy. Which is where differencing disks come in. You can "freeze" an image and run it, with the VM engine creating a differencing disk based on what you do. The performance penalty is greater, but you save time and disk space. You can also run multiple variants off the one base image.

I don't do that, I just buy more disks. A 300GB disk is about $300, or around a days work for me. It holds 25 or so virtual machine images. Or I could faff with frozen images and differencing disks and get slower performance instead.

Most of the handy utilities above are selected to make archiving VMs easier. I use 7zip because it pack VM files 10% or so smaller than anything else I've found. This is important for me sometimes, as my 20GB virtual disk that has Visual Studio, MS Office, the MSDN libraries, Eiffel and our applications on it just barely fits on a single DVD. I've played with compression

The sequence to pack the things down really tight is:

  1. disk cleanup (manually get rid of unnecessary files, then run the disk cleanup wizard)
  2. defrag (if you have a bought defragger with "consolidate free space" so much the better). The default MS defragger needs to be run repeatedly until it's no longer gaining you anything.
  3. run sdelete -c to zero free space
  4. shut down the VM
  5. run 7zip a -mx9 -mmd=27 -mmt (archive) (VM directory) where mmd=27 assumes you have 1.4GB of free RAM and -mmt assumes you have more than one CPU. RTFM.
  6. wait 2-10 hours
If you're really desperate for better compression, WinRK is generally slightly better than 7zip but takes three times as long (I gave up after 24 hours on the 8Gb test image I was using, where 7zip did it in 5). Like I said, desperate. SDelete for zeroing free space so it compresses better:

Steve Arena says...

Unfortunately I will miss Moz's VMware presentation. I am very disappointed to be still sitting at my desk with more than a few customer tasks that must be done this afternoon. I had hoped to be well on my way to Sydney by now.