New Lab Setup

Well, I’ve had the Qnap TS-670 for just about a week now and it has certainly improved things.  I had to rebuild my nest ESXi hosts for some reason because they weren’t able to provision 2012R2 servers, even after starting with a fresh vCenter but that didn’t take long.  I’ve now got the extra NIC card in the TS-670 (4 NICs total) with 1 for mgmt, 2 for iSCSI and one for NAS storage access.  I’ve beefed up the DL380G6 server by doubling the RAM to 64Gb and adding a quad 1Gb NIC card to give 8 NICs overall.  This is now permitting an iSCSI multi-pathing set-up and additional networks to really start playing with things, including hands-on with Cisco IOS!  The aim, to have the following networks routed to the primary LAN where the ADSL router sits:

  • Router LAN
  • Home device LAN
  • Lab LAN
  • iSCSI LAN (not routed)
  • IPStorage LAN (not routed)
  • vMotion LAN (not routed)

I’ve got some network cabling to install at some point because I’m not able to properly set-up IP routing on the Cisco and am moving my study/office/junk room! elsewhere in the house.  I’ll be running a small number of Cat-6 cables from wall sockets to an RJ-45 patch panel and then into the switch.

Since getting the new-lab in place, I’ve been really working on the PowerShell scripts I’ve blogged about previously as the new lab really speeds t hings up, and will be posting another blog on that after this one.

PowerShell Help Needed

I’ve reached a stumbling point in my PowerShell development for automating ESXi/vCenter customisation.  In a bid to leave out any hard-coded localised details from the scripts, by using CSV files, I’m stuck producing a list of clusters for the admin running the script to pick from.  The cluster details are in a CSV and I’ve used a foreach loop to pull out the names of the clusters, presenting them to the admin one by one in a numbered list.  Prior to this, I hard-coded the list and used a switch process to set a variable for the selected cluster.  I’m not sure how I can do this having used a foreach loop to pull CSV content, or if there is a better solution completely!

Could really use some help please!

More PowerCLI Work

Further to my previous post, I’ve been expanding on my PowerCLI scripts to form a more comprehensive automated deployment/configuration of ESXi, vCenter and virtual machines (although I’ve not reached the VM bit yet).  The vCenter and VM deployment/configuration phases will utilise CSV files to hold the relevant information and keep such sensitive details away from the script, which allows the script to be more visible whilst the CSV’s are kept secure.  By having a small set of CSV’s that contain detailed settings I hope to keep management and future changes simple as well as well as provide a controlled means of updating the live environment with changes into those CSV’s without the need to manually connect and apply.  I’m now thinking of taking out any hardening or general configuration actions and putting them into a CSV where they are called and applied, so again, only the CSV needs updating whilst the script remains simple and will perform the same function with whatever data is called.  I don’t have 3rd party deployment/automation tool at my disposal so presently, Excel/esxcli/PowerCLI/Kickstart is all I have available.

I’ve split the phases into separate scripts and used a simple menu script to call the relevant one as chosen by the admin.  The vCenter customisation script manages datacenter creation (if needed), cluster creation or reset to custom defaults, folder creation and vApp/Resource Pool creation/reset.  It will apply in-depth settings for objects such as clusters and vApps once they’ve been agreed.  There are some hardening requirements here also but these will be added later once scripted and probably already deployed into what will be a production vCenter.  The ESXi script I’ve already mentioned in the previous post, whilst the VM deployment script will deploy VM’s using a selection of templates and customisation profiles, into the intended cluster (of 3), custom folders/RP’s/vApps, add disks according to the requirements of the intended guest role (and provision as thin or TEZ) into the appropriate datastore, set CPU/RAM, set IP/Mask/GW/DNS and configure the required number of NICs into the appropriate portgroups.  The 4th script will just contain any PowerCLI cmdlets or ESX CLI commands to configure any VM’s already deployed as necessary, such as templates, which will then be used to build other VM’s and in theory, propagate the settings!

Scripts attached and I’d love PowerCLI guru’s to review and offer advice on how I could streamline or improve the scripts.  I’m a rookie to this so have probably written lines of code to achieve something a wizard could do in one line!ps-customesxi ps-menu ps-vcenter (renamed to .txt extension since WordPress wouldn’t let me attach .ps1 and I can’t be arsed to sort that out!).

vSphere ESXi Unattended Install & Follow-Up PowerCLI Script

This week I’ve been working on a means to automate the installation of ESXi with as much of the configuration completed as possible.  By this I mean joining the host to an AD for local auth, joining the host to a vCenter environment, building out local vSwitches and implementing as much of the hardening guide as possible.  This is the first time I’ve ever used the unattended install kickstart process and it has been slow but pleasing progress.  It would appear that almost all my requirements are possible, but may not be so easy through the kickstart process which is why I’ve also utilised the PowerCLI capabilities of vSphere to capture the impossible/difficult bits as a post deployment script.  All scripts are in early development and continue to progress as other parts of the infrastructure appear and allow additional functions to be possible.

My objective here is to ensure that all hosts are built identically by removing as much user intervention as possible and quick to deploy with little need to work through complex processes.  Yes, you can right a build guide with all the information and steps in minute detail, but steps can be missed for any number of reasons which poses a risk that each host could end up different, or missing a vital security configuration.  A mis-configured script could achieve the same result, but the speed of installation is important to me as it is possible other IT support staff will take over the infrastructure once deployment is complete and I want a good, clean, simple process for them to use should they need to deploy more hosts (which is likely too).

Installing from a USB key onto a server that had a local RAID1 array and USB Flash card, the first challenge was to get ESXi to install to the correct USB device, not onto the HDD and certainly not onto the install media USB (which it did the first time of trying!).  After some investigation into the kickstart script options I figured out how to achieve this objective but also how to prepare the local HDD for a VMFS datastore at the same time.  This was great because I wanted to rename the local datastore through an initial installation process anyway, so sorting out how to build it was taking me in the right direction.

Next came the network info for the management port, and I did have trouble here trying to use the %pre option I’ve seen used in so many online locations….it just complain about “/.pre” and falling over so I left that challenge for later opting for putting the IP details onto the command line.  I’m hoping to use a combination of DHCP/DNS and scripting to grab the correct network details from the network and apply that.  I also configured a number of vSwitch which were needed for the primary objective of this activity and set security etc.   With a few additional %firstboot configuration additions, the install was working nicely but I wanted more!

I don’t yet have a Windows AD, DHCP or DNS so we’re in the early phase of this project task but I’ve seen that it is possible to get a host to join an AD and also a vCenter from the kickstart process.  I’ve also seen people creating a local user account with shell access and setting perms but I decided to leave this out of this initial script for now.

With the ESXi host built and configured with it’s local datastore renamed, vSwitch configured and some hardening achieved, I moved to working on a PowerCLI script to complete some other actions needed for a clean and simple installation.  One big issue I had that I couldn’t find a way of doing during the unattended install, was to rename the default PortGroup used by the Management VMkernel.  I like to rename all PortGroups, which is easily done post-install via the VIClient, but this particular one was trouble for me.  So, one reason for PowerCLI because it is easy to do there!  Another cool task achieved at this stage was adding the host to vCenter and into the right cluster.  There will be several clusters operational so any automated (or semi-automated) process needs to be able to select the correct one without modifying code to do so.  PowerCLI did this with ease too.  While I was adding things to hosts and vCenters I thought why not add the local user and assign it to a role, propagating it!  Again, easy to do………..although I did have problems until I realised I’d spelt propagate wrong!!!

Lots more things to do with these scripts and more scripts to appear as I try to automate, ease the administration of the solution during deployment.  What I’d love to be able to do is automate the activation of a PowerCLI script from the kickstart install so it can be processed without user intervention until 100% completed.  I’d also like to figure out how to get my boot USB to auto-select the ks.cfg file without me have to SHIFT+O all the time.  I’ve tried adding runweasel=ks=usb:/ks.cfg to the BOOT.CFG kernelopts!

Kickstart cfg and PowerShell scripts attached, ps-il3-safetopost ks-il3-safetopost

Further Lab Enhancements

After the initial post regarding my home lab, not much has changed and to be honest, I’ve not done much with it.  Having passed the VCP I’m deciding whether to keep the vSphere 5.5 setup and work towards VCAP or rebuild with vSphere 6.0!  My current contract is based around 5.5 so that’s a tick in the ‘keep it’ box!  At present, the lab is using a single network alongside my general home devices, so it’s not a true reflection of a live environment as I’ve not got VLAN’s and don’t have any devices capable of internal IP routing without setting up another VM/Physical system to handle that.

Needing to learn Cisco’s IOS since that is the most common network vendor used all contracts I’ve applied/interviewed/accepted, I’ve decided to splash some business cash and procure a layer 3 Cisco switch to help further develop my lab and skills……two birds with one stone so to speak.  I wanted 1Gb capable connectivity on all ports (I’m not just learning for CCNA!), and therefore had to spend a bit more to get it.  Ebay had a few 24x 1Gb port switches and I opted for a 3750G-24TS-S which also comes with 4x SFP ports should I ever need to experiment with that!  It was actually cheaper that the base 3750G-24 switches though so I’m pleased with my purchase.  In all, £275, but I do need to get hold of a Cisco serial cable so perhaps my new client will have one spare!

With this new switch, arriving tomorrow, I’ll be placing it directly behind the BT home-hub and running the entire network from it with IP routing enabled.  I’ll VLAN off the general network (using the existing IP range), storage, vSphere Mgmt, vMotion and Lab networks.  It will be especially useful since I’m currently involved in low-level/system-level design with vSphere and Cisco switch/firewalls.

Home Lab Set-Up

My home lab is set up, and ready to provide the hands-on needed for vSphere skills enhancement.  It took a little while and a lot of fiddling to get it working exactly as I wanted and especially so to activate the nested ESXi 64bit guest support.  Although the Xeon E5540 CPU’s support VT-x/EPT, it is necessary to customise the virtual machine settings of the nested ESXi hosts to fully support it.  This creates a chicken/egg situation because with vSphere 5.5 you can only make the final change using the Web Client, which needs vCenter installed, which needs either a completely separate physical machine to run on, or has to be hosted on the main physical ESXi installation.

Having got all the key VM’s needed, FreeNAS, Windows Server 2008 (DNS/AD), Windows Server 2008 (vCenter/SQL Express), running on the primary host, the need for 64bit guest VM’s within the nested environment is not a requirement because 32bit will be enough for the purpose of vSphere study.  However, the challenge was there for me and if it can be done, I want to prove it for myself and get it working in my own lab!

So, I have the above VM’s running on the primary ESXi host and 3x nested ESXi host VM’s.  I have a QNAP offering NAS based NFS storage and a second NAS which is used to provide a VMDK to the FreeNAS VM which is then presented to the nested ESXi hosts as an iSCSI LUN.  The local 683Gb RAID5 is presenting VMDK’s to the nested ESXi hosts to appear as local storage and I’ve built 2x VMDK’s per host; 1x 20Gb & 1x 180Gb.  The 20Gb will be faked to represent an SSD drive and then the two can be utilised to allow VSAN support for all three nested ESXi hosts.  This gives me a chance to play with VSAN whilst providing the three main storage options to my lab, iSCSI SAN, NFS and local.  It’s a bit slow but it’s just for training!

On the nested ESXi hosts themselves I think I’ll only need Windows 7 guests to provide the material for study and I’ve created templates or exported as OVA’s a copy of the ESXi host, Server 2008 and Windows 7 VM’s.  I’m trying to put a Visio diagram together to show how things are set up but can never get it right without making look both rubbish and complicated.

Sites used to assist me:

http://www.v-front.de/2014/01/how-to-provision-nested-esxi-hosts-on.html – by Andreas Peetz

http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html – by William Lam

https://communities.vmware.com/docs/DOC-8970 – from VMware communities

VCP5-DCV (vSphere 5.5)

Having discovered my VCP3 expires completely next March (2015), I thought I’d better do something to upgrade to the latest equivalent.  Giving myself about 4 weeks to study having booked the 50% discounted exam during VMworld USA, I realised how much there was to take in and 4 weeks simply wasn’t enough.  Consequently I failed the exam with a score of 280 needing the 300 to pass.  So, despite using 5.5 every day at work with high levels of exposure to the full vSphere suite of components I have decided to build my own home lab and work through the VMware Press study guide.  Previously I used the Sybex version but decided to switch so I got a different view point and possibly different means of delivering the content.  I also got a good discount at VMworld!

I’ve now booked my retake for the 28 November and bought a refurbished server from Ebay to begin a hands-on home study.  I picked up a DL380 G6 with twin E5540 Xeon’s and a couple of SAS drives with 8Gb RAM for £350.  Having some old SAS disks lying around at work, destined for the shredder and some smaller RAM sticks from the G6 memory upgrades at work, I should be able to boost this to around 32Gb and include 4-6 146Gb SAS disks.  My lab plan will be detailed in a following post once the hardware arrives on Monday 29 September.

Should I fail again I’ll be booking in before the year is out to capitalise on the current 25% discount and then go again asap to ensure I pass before 10 March 2015!!  Fingers, toes, arms and legs all crossed.