Building a VMware vSphere Home Lab

Why build a home lab?

After attending a vSphere course, many students are keen to consolidate or continue the learning experience, and want information on how to go about building their own vSphere lab at home. In some cases they want to gain more hands-on practice before starting a virtualization project at work. In other cases they are already using vSphere in production, but they don’t get a chance to use all the technologies covered in class. Often, a smaller organisation may not have the appropriate license to enable them to deploy the more advanced features of the product. Finally, building a lab is a great way to study for the VCP exam.

What hardware do I need?

The hardware that you need depends on many factors. There are several different approaches to building a VMware vSphere home lab, depending on your budget and goals.  For example, some early adopters quickly found that it was possible for ESX to “virtualize itself”. The concept is to install ESX on the bare metal, and then to install ESX as a virtual machine running on top of ESX. The possibilities that this approach opens up are enormous! For example, you can install as many instances of ESX as resources (CPU, memory) permit. As an example of the flexibility of such a set up, you could give each ESX virtual machine multiple virtual network cards even if the physical machine has only one NIC. In short, you can build an entire lab on just one box.

A similar approach taken by some is to install ESX on top of VMware Workstation or VMware Fusion. The main benefit of this is that you can often use an existing desktop or laptop without dedicating the machine to running a lab. When creating a virtual machine using VMware Workstation there is even an option to choose “ESX” as the guest operating system. If you are interested in using these methods of building a lab, then follow some of the links in this document where you will find plenty of useful information.

When I built my own home lab, I decided to take a more traditional approach. I designed the lab around physical servers running bare metal ESX, much as you would in production. My reasons for choosing this approach were as follows:

• I wanted the lab to be as “realistic” as possible.
• Performance will not be compromised.
• No special workarounds are necessary.
• There are some technical limitations using the other methods.

The rest of the document describes a lab setup that enables you to configure almost all of the technologies covered in the VSICM course and labs, including connecting to iSCSI and NFS storage, and features such as High Availability, vMotion, Storage vMotion, DRS and Fault Tolerance.

When building a lab for home use, it is common practice to use hardware which is not on the official VMware Hardware Compatibility Guide. Considerable cost savings can be achieved by doing this. However, you will want to be confident that ESX will run on your chosen hardware without any problems. Such hardware is commonly known as “whitebox” hardware, and lists are available at http://www.vm-help.com/esx40i/esx40_whitebox_HCL.php or http://ultimatewhitebox.com/.  These lists contain hardware that has been proven to work with ESX by members of the VMware community.

A good source of information on current “hot deals” for such hardware can be found at http://www.techhead.co.uk/. Part of this site is dedicated to building home labs. Some server distributers, such as http://www.serversplus.co.uk/ have sold lab bundles specifically for VMware. Of course, a whitebox system will not be supported by VMware and should definitely not be used outside of a lab environment.

My lab currently uses two HP Proliant ML 115 G5 servers. These contain Quad Core AMD Opteron CPUs. At the time of building the lab, these servers represented very good value for money and are a very popular choice in the community. However, these particular severs are now end of life and so I recommend you use the links above to research a suitable alternative.  You should not need to spend more than around £150 for a server. I have included a full list of my lab hardware together with the costs at the end of this document.

HP Proliant ML115 G5 Tower Server

I upgraded the servers with 4GB memory each, which I think is a useful minimum for a lab, and capable of running just a few VM’s.

The servers came with a 160GB internal disk. It is common practice to use the local disks for the installation of ESX. However, these servers also came with an internal USB slot on the motherboard, giving another choice for the installation.

USB stick plugged into internal USB slot

For the switch, I use a Cisco Small Business SLM2008 8-Port Gigabit Smart Switch at around £80. The switch supports the following features.

• 802.3ad Etherchannel
• VLANs and 802.1q VLAN trunking
• A web based GUI for management

 Support for these features is very useful for testing vSphere functionality such as Virtual Switch Tagging and IP based load balancing. However, if you don’t need these features you can simply use the switch as a “plug and play” device, without any configuration.

You will need several straight through Ethernet cables. They should be Cat 5e or higher in order to support the Gigabit speeds recommended as a minimum for most networking functions in vSphere.

Cisco SLM2008 8-Port Gigabit Switch

Which Hypervisor?

As you may know, ESX is due to be phased out in the next major version of vSphere. For that reason, I chose ESXi 4.1 Installable as the hypervisor for the lab. The 60-day evaluation version of ESXi 4.1 installable and vSphere 4.1 can be downloaded from the VMware web site https://www.vmware.com/tryvmware/index.php?p=vmware-vsphere&lp=1 . The evaluation version includes all the features of the Enterprise Plus edition. You will need to register using an email address. The image is an .iso file, which can be burned to a CD or DVD using a desktop or laptop.

Before the 60-day evaluation period expires you can simply re-install ESXi. This isn’t such a big issue as it might sound, as the installation only takes a few minutes. Once you have reinstalled the product, you can take advantage of using a backup/restore script or the Host Profiles feature to restore the system to a consistent state.

When these particular servers are first powered on, the fans run at full speed for a few seconds, creating a very loud noise, but then they settle down to a level that you may find acceptable in a living room or study.

If you are using whitebox servers, you may need to research any issues and workarounds that other users have found with ESX on your chosen server. One issue I found with these particular servers is that the built in CD drive is not supported by ESXi  4.1, and an error message was displayed part way through the installation.

The solution was simply to use a cheap external USB CD drive instead.

Successful installation of ESXi 4.1

The installation process creates a VMFS file system on the internal disk. The local datastore can be used to store virtual machines, templates and iso files until shared storage has been provisioned. The installation was repeated for the second host.

Installing vCenter Server

vCenter Server can be implemented either on a physical machine or as a virtual machine. While there are many pros and cons to consider in a production environment, it is likely that you will want to run vCenter Server as a virtual machine in a lab environment. This keeps the number of physical boxes needed to a minimum. The following steps describe how to proceed. You may need to use the VSICM course notes for more details on the individual steps.

• Use the DCUI to configure root passwords, IP addresses and hostnames. Other details such as DNS configuration and default gateways can be left until later.
• Download the 60-day evaluation version of vCenter Server 4.1 from the VMware web site.
• Install the vSphere 4.1 Client (included with the vCenter Server software) on a Windows machine, for example on your laptop, and connect it to one of the hosts.
• Create a Datacenter object.
• Create a virtual machine for vCenter Server, allocating the minimum requirements of 2 CPUs and 3GB RAM. Allocate space for the virtual disk. I used a 40GB, thin provisioned disk. The virtual machine files can be moved to shared storage later using Storage vMotion. This will allow vCenter to be continuously operational even if you need to shut down one of the physical servers for maintenance.
• Install a 64-bit version of Windows 2003 or 2008 Server guest operating system. A Microsoft Technet subscription is a useful way of gaining access to Microsoft software. Alternatively, Microsoft allow you to download an evaluation version of Windows 2008 server from their website, but bear in mind that the time period of the evaluation will be quite short.
• Install the DNS component of Windows Server, create forward and reverse lookup zones, and create records for the two ESXi hosts and vCenter Server. Using the DCUI or vSphere Client, configure the ESXi hosts as clients of the DNS server.
• Install vCenter Server on the Windows virtual machine. As this is a lab, I chose to use SQL Server 2005 Express for the database.
• Login to vCenter Server from the vSphere Client using the Windows administrator account and add the two ESXi hosts.

Which Storage?

Shared storage is an important pre-requisite for features such as vMotion, DRS, HA and FT. There are many choices for the storage. One option is to use free or evaluation third party NAS/iSCSI software such as Openfiler from http://openfiler.com/ or StarWind from http://www.starwindsoftware.com/. These products are installed either on a separate physical system or as a virtual machine. However, my choice was to use low cost external storage. An enterprise class storage array is likely to be outside a typical budget, but for a home lab performance is not such an issue, and I chose the iomega StorCenter ix2-200 at around £180. This unit is designed for small business or home use. It came with 2*500GB disks for a total of 1TB of storage. It supports iSCSI, NFS and CIFS, and is even on the VMware HCL! It has a web based GUI and is very simple to configure. Using the iSCSI sotware initiator, I connected the two hosts to the storage and created a Datastore.

The iomega runs very quietly so you can leave it powered on even if you don’t keep your servers powered on all the time. You could of course use it to store your music and videos, backups of your laptop, or anything else you want.

Iomega StorCenter ix2-200

Networking

The servers came with a Network Interface Card integrated with the motherboard, which was recognised with no problems by ESXi during the install. Devices such as network cards, together with storage adapters, have been problematic areas in the past when using unsupported hardware.

It would be technically possible to use a single NIC in each server for all the networking features in a lab. However, in order to configure and test features such as NIC teaming and network redundancy, you may want to purchase two extra NIC cards. Look for NIC cards which are on the HCL or whitebox list.

You may be happy to use the servers in a spare room or study. Alternatively you may want to hide them away in a cupboard, and access them from a laptop from a different room. For this lab, I used a spare shelf in a cupboard to house the equipment. Despite advances in wireless technology, I decided to run Ethernet cables to the cupboard so that I could connect to the lab using the vSphere Client at Gigabit speeds. I found this especially useful when using the datastore browser to upload large iso files to the storage. I also wanted to connect the lab to my internet router. I installed Ethernet cabling between the cupboard and the study. Wiring an Ethernet socket is only a bit more involved than wiring up a plug. You need a small plastic “punch down” tool to insert each of the 8 wires of the UTP into the socket. The socket will be colour coded to simplify the task.

Power and Ethernet from Screwfix and Maplins

If housing equipment in an enclosed space such as a cupboard, it is necessary to provide ventilation and cooling. I cut two holes in the door using a jigsaw, one at the bottom for cool air in, and one at the top for hot air out. This arrangement allows the air to circulate. Monitor the temperature in the cupboard. You may find that you need to additionally install a fan.

I wanted to access the lab over the internet from any location, so I used the following procedure:

• Enable RDP on the vCenter Server.
• Configure port forwarding on the internet router. Your internet router should have a menu or CLI option for this. If you have a BT Home Hub 2, for example, the feature is called “Application Sharing”. You are asked to specify the protocol (RDP) and the IP address for packet forwarding (vCenter Server).
• Installed the vSphere Client on the vCenter Server.
• My broadband connection uses a DHCP assigned address for the internet facing ADSL interface, which frequently changes, even while the router is powered on. I use a free pubic dynamic DNS service http://www.dyndns.com/, so that I can access the lab using a fully qualified domain name. When you register with the site, you can choose the host portion of the name. The BT Home Hub 2 includes an option to update the dynamic DNS whenever the dynamically assigned IP address changes.

Kit List

This is a full kit list of my current lab together with costs.