VMware Troubleshooting – Time Is On My Side

Lately I’ve been hitting some strange issues in vSphere and vCloud installations. First it was things around SSO not being able to connect and then it was the VMRC console in vCloud that started giving weird “invalid ticket” errors that resulted in vCloud VMRC console being accesible .. or not!

Both issues seemed unrelated, but the solution was the same : incorrect time settings on one of the vSphere / vCloud components.

So from a troubleshooting perspective we can add another check to the default checklist:

1. Check firewall.

2. Check time (NTP) settings!!!

It maybe a simple solution, but something to keep in mind while troubleshooting. It can save you a lot of frustation.

Some resource with regards to time and vSphere / vCloud :

VMware KB 2012069

VMware KB 2033880

Gotcha: NTP Can Affect Load Balanced vCloud VMRC

vCloud Suite 5.1 Licensing Explained

VMware has announced the release of the new vSphere 5.1 solution. Together with this new release, VMware has also announce it’s new VMware vCloud Suite 5.1 licensing model. This model combines multiple components (vSphere Enterprise Plus, vCloud Director, vCloud Networking and Security, etc.) into a single product with a single license. All virtual machines running on a properly licensed vCloud Suite processor can use all components included in that vCloud Suite edition.

Licensing per processor

As mentioned above the licensing unit takes place per-processor. VMware no longer limits it’s customers physical resources and  on the number of virtual machines!!! VMware has listened to the VMware Community and no longer applies the vRAM principle. Or like other call it, the vTax. The VMware vCloud Suite 5.1 is licensed per physical processor. With all physical processors licensed in a server a customer can run all VMware products on top of this server that are licensed within the bundle.

vCloud Suites Editions


There are 3 editions available for the vCloud Suites :

1. VMware vCloud Suite Standard; vSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Standard.

2. VMware vCloud Suite Advanced; vSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Advanced and vCOPs Advanced.

3. VMware vCloud Suite EnterprisevSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Enterprise, vCOPS Enterprise, vFabric Application Director and SRM.

So what’s the deal?

In my opinion VMware tried to simplify the whole licensing part of building a vCloud solution. Most customers that build a private cloud in general want to build such a vCloud solution in an easy manner, but it also needs to be easy to manage, must be monitored and should work in case of a disaster.

All of these components are in the bundle that is licensed with vCloud Suite Enterprise edition. An easy licensing path on the road to your own private vCloud. Most companies already have VMware vSphere licenses and VMware also provide an upgrade path toward the new VMware vCloud Suite licenses. For upgrading VMware has introduced the Fair Value Conversion Program that can be found at http://www.vmware.com/go/ vcloud-suite-licensing.

For more information on the VMware vCloud Suite licensing see the vCloud Suite 5.1 Pricing and Packaging Whitepaper or talk to your own VMware sales representative.


vCenter Single Sign-On Server in vSphere 5.1

VMware introduces vCenter Single Sign-On with vSphere 5.1. This solution creates a new layer between the vSphere solutions and the customers identity sources. The figure below gives a graphical representation where to position vCenter Single Sign-On.

The vCenter Single Sign-On server is the vSphere platform service that will be in between the vSphere solution, such as vSphere Web Client, vCenter, vCloud Director,etc., and the identity sources that are available within the customer infrastructure.

vCenter Single Sign-On has been introduced within the vSphere environment for the following reasons:

·       Provide one single sign-on solution for authentication across all management applications;

·       Support for multiple user identity repository solutions;

·       One central point for authorization and auditing within the vSphere environement;

·       Trust between components using token exchange, in stead of each solution having it’s own identity creation and authorization process;

·       Support for open standard authentication protocols: SAML 2.0 and WS-TRUST.

Besides the improvements mentioned above, vCenter Single Sign-On can now also be setup with a in a more resilient setup. This will result in a high availability level for authentication in the vSphere environment.

For more information about the vCenter Single Sign-On Server look at the “vSphere 5.1 – What’s New vCenter Server”

New storage books for designing cloud infra

When creating a design for your cloud environment you always have to take the physical components, such as compute, network & storage into account. These components are the foundation that your cloud environment will be build on. A good design of these components is crucial for your overall design, the performance and resilience of your solution. Fact remains that you can’t know it all, but when you do want to know it, then the best way is to learn it from the experts.

Now we have the chance to do so. Three experts in the field of storage released two books about storage in relation to virtual cloud environments.

Mostafa Khalil from VMware, released the book “Storage Implementation in vSphere 5.0”

“The more important VMware virtualized infrastructure becomes, the more important virtualization storage becomes. Virtualization storage planning and management is complex, and it’s been almost impossible to find authoritative guidance – until now. In Storage Implementation in vSphere 5.0, one of VMware’s leading experts completely demystifies the “black box” of vSphere storage, and provides illustrated, step-by-step procedures for performing virtually every task associated with it. Mostafa Khalil brings together detailed techniques and guidelines, insights for better architectural design, planning and management best practices, common configuration details, and deep dives into both vSphere and external storage-related technologies. He gives technical professionals the deep understanding they need to make better choices, solve problems, and keep problems from occurring in the first place. This book answers crucial, ground-level questions such as: How do you configure storage array from “Vendor X” to support vSphere “Feature Y”? How do you know you’ve configured it correctly? What happens if you misconfigure it? How can you tell from logs and other tools that you have a problem – and how do you fix it? Most of the author’s troubleshooting techniques are based on his own personal experience as a senior VMware support engineer helping customerstroubleshoot their own vSphere production environments – experience that nobody else has.”

At the same time Vaughn Stewart and Mike Slisinger from NetApp released the book “Virtualization Changes Everything: Storage Strategies for VMware vSphere & Cloud Computing”:

Storage is a foundational component in the support of virtualization and cloud computing – and it is dynamically evolving. It is an aspect of the datacenter that is all-too-often overlooked, but without storage, there is no data, and without data, there is no cloud. Virtualization Changes Everything, by Vaughn Stewart and Mike Slisinger, examines the evolutionary influence of host virtualization and cloud computing in breaking storage deployment out of outdated silo models and into a dynamic, flexible hosting environment. Virtualization Changes Everything reviews common goals and challenges associated with providing storage service with cloud computing, and addresses each through the application of advanced storage technologies designed to scale in order to support the ever-expanding storage needs of the future. The examples within the book are pulled from real-world experience, and often involve the integration of multiple innovative technologies. If you are looking for measured guidance on high availability, efficiency, integration and performance for the storage in your cloud, then this book is for you!”

Both execellent books on the topic of storage and the impact it has on your virtual cloud environment. A must read for everybody that wants to gain more knowledge on this topic and the impact storage has on virtual cloud environments.

vSphere : Storage protocol comparison

This whitepaper by VMware Technical Marketing really is a great comparison between the most common storage protocols that are being used with vSphere Infrastructure. When designing your virtual infrastructure it is important to know what the characteristics are of the various protocols that can be used to connect your ESXi host to a shared storage environment.

This document describes and compares the following storage protocols:

* Fibre Channel
* Fibre Channel over Ethernet (FCoE)

The whitepaper can be found here.

Understanding vSphere stretched clustering and SRM

Stretched clustering is one of the challenging topics I get when meeting with customers. Many customers think that stretched clustering is the ultimate disaster recovery solution and that it makes SRM obsolete. This is due to the fact that people think that HA will solve all their problems when it comes down to DR and that they still have the advantage of vMotion to have workload mobility between two data centers.

This however isn’t always true and there are some catches to the implementation of stretched clusters. In some cases, depending on the customers requirements, it even is better to have an SRM implementation to fulfill the requirements.

The table below give a graphical representation of which solution best suits which requirement.


So in the end its up to the customer to decide which solution best suits its requirement. To help make this decision VMware Tech Marketing created a whitepaper to help make the right choice. A must read for everybody involved with disaster recovery and availability within a vSphere infrastructure.

The whitepaper can be found here.

VMware vSphere Distributed Switch Best Practices

VMware vSphere Distributed Switch (VDS) is a key virtual network platform that is scalable and extensible. It also provides advanced networking features and operational simplicity that helps build a flexible network for the cloud. The advanced networking features such as NIOC, LBT, NetFlow, and Port mirror are available on VDS and not on a virtual standard switch (VSS). This paper walks through the design process of incorporating VDS and its advanced features in different customer deployments. Customers typically have various types of server hardware and network adapter configurations in their environment. Following standard server hardware configurations are considered during the VDS design discussions.

1) Rack Server with Eight 1 Gigabit network adapters
2) Rack Server with two 10 Gigabit network adapters
3) Blade Server with two 10 Gigabit network adapters
4) Blade Server with Hardware assisted multiple Logical network adapters

For all these above hardware configurations different VDS design approaches will be presented along with their pros and cons. Along the way some best practices and important parameters of virtual and physical switches will be highlighted. Finally, some operational aspects of the VDS will also be discussed.

The whitepaper can be downloaded here.

Architecting High Available vSphere Auto Deploy

One of the cool new features of vSphere 5 is Auto Deploy. This tool enables the stateless booting of ESXi hosts. No longer a installation to disks is required to boot the ESXi hypervisor. Instead the hypervisor is booted via PXE boot into the memory. The ESXi image is booted onto a bare-metal server, so in case the server powered off the “installation” and configuration is also lost. Of course the next time the bare-metal server is booted up again, Auto Deploy makes sure that the stateless image is again deployed via PXE into the memory of the server. So far, so good. If of course the Auto Deploy tool is available to provide the deployment of ESXi images.

And this is the question that always pops up when discussing Auto Deploy in a vSphere design: “What if the Auto Deploy tool isn’t available? Then I can’t boot my ESXi hosts anymore!”

That is true, but fortunately there is a way around that. Daniel Hiltgen, Senior Staff Engineer at VMware, explains how to architect your Auto Deploy environment and make it highly available. This also prevents the “chicken-egg” problem. So have a quick look at this short, but interesting video!

Unattended install of VMware Tools

Large environments require different techniques in rolling out software packages. This is also the case for VMware Tools.

You can of course right click on the virtual machine in the vSphere Client and select Guest > Install/Upgrade VMware Tools or even create a PowerCLI script to do the job for you. But sometimes, and common in large environments, you need to comply with the IT infrastructure policy and install your VMware Tools by using a software distribution tool like Microsoft System Center Configuration Manager.

For this you need to know how the VMware Tools MSI package is installed and which options you want to install. For this Valentin Hamburger, Technical Account Manager @ VMware, has written a great PDF document containing the decomposition of the VMware Tools package.

The picture below shows the various installation options from the VMware Tools MSI package and if they are required or not.


You can download the complete PDF over here, but with the following disclaimer : “This documentation is provided “as is”. It is not part of the official VMware product documentation.”