Automated installation is the way to deploy your ESXi configuration to your servers. After all it’s all about automating your IT operations and making your life easier. Nobody likes to do a simple task twice, so I would definitely recommend to automate your ESXi installation. ESXi uses kickstart for automating your installation. You can view more details on this in my previous post over here.
Disk selection
All kickstart scripts start with the installation of ESXi. Afterwards you can add first-boot script to configure your ESXi installation to your specific needs.
One of the first thing you need to decide is where to install your ESXi installation to. The autopart command in the kickstart file specifies where you will install your ESXi to. You’ve got 3 options to choose from :
* Local = install ESXi onto the first local disk (local hard drive)
* Remote = install ESXi onto the first remote disk (FC or iSCSi LUN disk)
* Driver = install ESXi to the device which uses this driver in the vmkernel to access the disk
So if your using local disk select local and if your booting from SAN use remote.
But wait! There’s a catch…
Some local disks, specifically SAS disks, are not presented to ESXi installer as local disks, but as remote disks. This is also acknowledged by VMware in this KB article.
The solution is rather simple, but you do need to know the specific driver for the controller of your local SAS disk. In my case the server was a HP BL460c G6 blade. Thanks to this (Dutch) VMUG post I was able to trace the driver (hpsa) for the SAS controller in the HP BL460c G6.
And this is where the driver option of autopart kicks in. If you have several types of servers and always want to install to the local disk, use the following command :
autopart –firstdisk=hpsa,local –overwritevmfs
In this case ESXi first tries to install to the disk access with device driver hpsa. If that fail it will try to install onto the first local disk. If both options fail, the installation of ESXi will fail and you will error will be shown onscreen.
Hope this gives you some better understanding in how ESXi installs itself onto disk. For more information have a look at the Setup Guide of ESXi here.
So you have to deploy ESXi and think about automating the installation. Then the kickstart file is going to be your new best friend. It’s a think once, apply many concept to deploy ESXi to your hardware.
At the moment I’m building an vSphere 4.1 infrastructure with 300+ HP blades. All need to be migrated from ESX 3.5 to ESXi 4.1. This requires a new installation of ESXi. I’m using kickstart to automate the installation process and to get a consistent installation state when deploying ESXi.
ESXi deployment options
In general there are 2 options if you want to install ESXi : interactive and scripted. As you will probably understand, kickstart is the core of the scripted installation. The kickstart.cfg file is the file that contains the configuration settings for your ESXi installation and more.
The picture below shows the different options you have for installing ESXi onto your hardware.
Both installation method use either CD or PXE to install ESXi. The key difference is the kickstart file. The most commonly used scripted installation is the PXE method. With PXE you boot your server into a PXE image which deploys your ESXi from a media depot hosted through HTTP(S), FTP or NFS. This media in combination with your kickstart file will created a custom ESXi server.
Deployment tool
For the deployment I have been using HP Insight Control Server Deployment 6.2 (former HP Rapid Deployment Pack (RDP)) an OEM version of Altiris Server Deployment 6.9. As far as I know the only commercial tool at the moment that supports ESXi 4.1 deployment.
There are however also two freeware appliances available for PXE ESXi deployment. Ultimate Deployment Appliance (UDA) and ESX Deployment Appliance (EDA) can be downloaded from the VMware Virtual Appliance Marketplace. But you can also take a look at the ESXi Installable and vCenter Server Setup Guide which gives good leads on building your own PXE deployment solution.
HP Insight Control Server Deployment presents you with a workflow tool to deploy a server. The workflow can be created by adding scripts to installation jobs. HP creates specials deployment packs for OS installations. Also one for ESXi 4.1 which has the following 4 jobs in them :
1. Configure BIOS (Very handy to set those BIOS settings for virtualization!)
2. Deploy GRUB image
3. Configure GRUB image
4. Create kickstart file based on default kickstart script
This deployment pack generally gives you a basic installation of ESXi if you don’t edit the kickstart file. The only thing you have to do afterwards is assign the job to a specific HP server and it will install ESXi out-of-the-box.
Kickstart file
The kickstart file below is used in conjunction with HP Insight Control Server Deployment. This tool has a database with all the configuration variables per server object. The @@VARIABLES@@ in the kickstart script below are replaced each time the installation job is executed.
This creates a server specific kickstart file for each server in your server park.
The lines until %firstboot are created to install ESXi. All commands after that line will be executed after the ESXi host has booted for the first time.
For more information about the specific commands in the kickstart file :
#————————————————————————-
# Customer default Kickstart for ESXi 4.1.x
# Created by : Martijn Baecke
# Date : 26-01-2011
# Summary : This kickstart script is used for the installation of
# ESXi using HP Insight Control Server Deployment 6.2
#————————————————————————-
########### Start : Networking ###########
# Add vMotion portgroup to vSwitch0
esxcfg-vswitch -A “vMotion Network” vSwitch0
# Add vmnic3 to vSwitch0
esxcfg-vswitch -L vmnic3 vSwitch0
# Add IP address to vMotion vmk1
esxcfg-vmknic -a -i @@VMOTIONIP@@ -n @@VMOTIONNETMASK@@ -p “vMotion Network”
# Assign VLAN to vMotion Network portgroup
# esxcfg-vswitch -v XX -p “vMotion Network” vSwitch0
sleep 10
# Set vMotion to vmk1
vim-cmd hostsvc/vmotion/vnic_set vmk1
# Set security policy to reject on vSwitch0
vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-promisc=0 vSwitch0
vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-macchange=0 vSwitch0
vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-forgedxmit=0 vSwitch0
# Set NIC order policy for portgroups on vSwitch0
vim-cmd hostsvc/net/portgroup_set –nicorderpolicy-active=vmnic0 –nicorderpolicy-standby=vmnic3 vSwitch0 “Management Network”
vim-cmd hostsvc/net/portgroup_set –nicorderpolicy-active=vmnic3 –nicorderpolicy-standby=vmnic0 vSwitch0 “vMotion Network”
# Set failback to No for portgroups on vSwitch0
vim-cmd hostsvc/net/portgroup_set –nicteaming-rollingorder=1 vSwitch0 “Management Network”
vim-cmd hostsvc/net/portgroup_set –nicteaming-rollingorder=1 vSwitch0 “vMotion Network”
############ End : Networking ############
########### Start : Storage ###########
# Configure local datastore with different label
vim-cmd hostsvc/datastore/rename datastore1 “$(hostname -s)-local-storage”
# Set Round Robin (RR) as default PSP for VMW_SATP_SYMM (EMC devices)
esxcli nmp satp setdefaultpsp –satp VMW_SATP_SYMM –psp VMW_PSP_RR
# Set Round Robin (RR) for all EMC devices
EMC_DEVICES=`esxcli nmp device list | grep ‘EMC Fibre Channel Disk’| awk {‘print $NF’}| sed -e ‘s/[()]//g’`
for i in $EMC_DEVICES;do esxcli nmp device setpolicy -d $i -P VMW_PSP_RR; done;
# Set Round Robin (RR) for all HDS devices
HDS_DEVICES=`esxcli nmp device list | grep ‘HDS Fibre Channel Disk’| awk {‘print $NF’}| sed -e ‘s/[()]//g’`
for i in $HDS_DEVICES;do esxcli nmp device setpolicy -d $i -P VMW_PSP_RR; done;
# Determine if this is a cluster with EMC or HDS storage. Cluster with mixed storage not allowed! If mixed EMC settings apply!
if [ -n “$HDS_DEVICES” ]; then
# This is a cluster with HDS storage
# Queue depth for HDS remains default = 32
# If this needs to change unquote following lines. Replace YY with value
# esxcfg-module -s ql2xmaxqdepth=YY qla2xxx
# vim-cmd hostsvc/advopt/update Disk.SchedNumReqOutstanding long YY
# Enable VAAI for HDS
vim-cmd hostsvc/advopt/update DataMover.HardwareAcceleratedMove long 1
vim-cmd hostsvc/advopt/update DataMover.HardwareAcceleratedInit long 1
vim-cmd hostsvc/advopt/update VMFS3.HardwareAcceleratedLocking long 1
else
# This is a cluster with EMC storage
# Set queue depth to 64
esxcfg-module -s ql2xmaxqdepth=64 qla2xxx
vim-cmd hostsvc/advopt/update Disk.SchedNumReqOutstanding long 64
fi
############ End : Enable Management Traffic ############
########### Start : Enable SSH Tech Support Mode ###########
# Only enable this when doing testing with kickstart!
# Default = disable by using # in front of commands
