Trust your ESXi hypervisor!

When it comes to security there is always the concerns about the security of the ESXi hypervisor. Its always the hypervisor that is nominated as the layer that can’t be trusted within the IT infrastructure. The whitepaper by Mike Foley tries to give you more insight on how the VMware ESXi hypervisor from a security perspective and what things to look at when securing the hypervisor.

The topics covered in the white paper are:

Secure Virtual Machine Isolation in Virtualization
Network Isolation
Virtualized Storage
Secure Management
Platform Integrity Protection
VMware’s Secure Development Lifecycle

The document can be downloaded here.

VMware Troubleshooting – Time Is On My Side

Lately I’ve been hitting some strange issues in vSphere and vCloud installations. First it was things around SSO not being able to connect and then it was the VMRC console in vCloud that started giving weird “invalid ticket” errors that resulted in vCloud VMRC console being accesible .. or not!

Both issues seemed unrelated, but the solution was the same : incorrect time settings on one of the vSphere / vCloud components.

So from a troubleshooting perspective we can add another check to the default checklist:

1. Check firewall.

2. Check time (NTP) settings!!!

It maybe a simple solution, but something to keep in mind while troubleshooting. It can save you a lot of frustation.

Some resource with regards to time and vSphere / vCloud :

VMware KB 2012069

VMware KB 2033880

Gotcha: NTP Can Affect Load Balanced vCloud VMRC

Performance Best Practices for Hadoop on vSphere 5.1

Apache Hadoop provides a platform for building distributed systems for massive data storage and analysis using a large cluster of standard x86-based servers. It uses data replication across hosts and racks of hosts to protect against individual disk, host, and even rack failures. A job scheduler can be used to run multiple jobs of different sizes simultaneously, which helps to maintain a high level of resource utilization. Given the built-in reliability and workload consolidation features of Hadoop it might appear there is little need to virtualize it.

However there are a lot of benefits on virtualizing the Hadoop workload on top of VMware vSphere. VMware has written a whitepaper with performance best practices for Hadoop on vSphere 5.1. Read the full paper for detailed results and to learn about performance best practices for deploying Hadoop on vSphere.

More information can also be found on the blog by Josh Simons over here.

My road to becoming a VCDX

“Desire is the key to motivation, but it’s determination and commitment to an unrelenting pursuit of your goal – a commitment to excellence – that will enable you to attain the success you seek.” – Mario Andretti

Finally found some time to write down my experience on achieving my VCDX. It’s been a long road to achieving my VMware Certified Design eXpert (VCDX) certification and as the quote above states it takes determination and commitment to achieve this goal. It was a bumpy road, not only during my project, but also when doing the VCDX defenses. I didn’t achieve it the first time, but with determination and commitment I continued the journey. Sure, it was disappointing and it took me some time to get over it and regain confidence, but in the end it pays off when you receive the email stating that you’ve successfully passed for VCDX. That’s a great feeling and proves that with desire, commitment and determination you can come a long way.

Hopefully this post will help others to in achieving it the first time. But even if you didn’t pass the first time, that shouldn’t stop you from trying a second time. It isn’t a nice feeling when being told that you didn’t pass, but don’t look at it as failure, look at it as feedback. Learn from it and use that to your advantage.

Design is an art, it takes time and patience…

For me it al started when doing a VMware vSphere 4.1 project for a customer with a large virtual infrastructure that needed to be upgraded to vSphere 4.1. Not to say that your project needs to be big. This project incorporated 60 clusters and 360 ESXi hosts. That’s what I call a big environment, but isn’t necessary for a VCDX project. What does help is that you choose real-life project.

Tip #1: Choose a real-life project.

Something to get your teeth into and that guides you along the way of design. Design is a step-by-step, iterative process and helps you in determining why certain choices were made in your design. Seek peers to review the design along the way. Ask your customer to be critical and speak to them on the choices you’ve made in the design. Present the results in the end and ask them to challenge what you’ve come up with.

And like I already stated it doesn’t need to be a large, complex environment. As long as you have a project that takes you along the following route:

1. Gather customer requirements and constraints;

2. Create a vSphere logical design that meets the requirements and takes the constraints into account;

3. Translate this logical design into a physical implementation again taking the requirements and constraints into account.

Take into account that during your design you will go up and down this list. In most designs there will be contradictions between the requirements, constraints and the things that are physically possible. This is were your architect skills will come into play and the guidance that you need to provide to your customer and in the end this process will provide you the why for choices you’ve made in your design. Guess what is interesting during your VCDX design defense….

Tip #2 Keep a log of all the design decisions that were made during the design phase

Tip #3 : Use the VCDX blueprint when creating your design

The blueprint has been based on design areas that required in a vSphere design. Therefor it is a very useful document when creating a design. Use this information when going through your design process and try to focus on all the design areas that are mentioned in the blueprint.

Also it is useful to keep in mind that the choices you make for your design should be based on the requirements and constraints that apply to the customers environment. It is your job to explain why certain design choices have been made taking those customer requirements and constraints into account. In other words there is no single perfect design. That is not what the panel is looking for. The panel is looking for the validation of choices you made during your design phase. Show them the thinking process you went through when making a design decision.

Tip #4 : There is no single perfect design. The best design is the design that meets the requirements and constraints of your client.

Keep this tip in mind when designing. There is no perfect design. The best design you can create is the one that you’ve agreed upon with your customer. You need to take all the requirements and constraints into account and create a design that meets those elements. Don’t try to struggle with the fact that is must be a “perfect design”. The only perfect design is the design that takes the customers needs into account. And again you need to be able to recollect why you made certain choices for the customer in you design.

Going doing the certification path.

Trying to achieve your VCDX is a choice. It needs to be your decision and you will need to commit to go along on this journey. It takes time and a lot of effort before you eventually stand in front of the panel. With time and effort comes planning, so…

Tip #5: Make a planning, set a goal for yourself

Do this at the start. Doesn’t matter when. If your a VCAP, VCP or don’t have VMware certification at all, anytime you can decide to go for your VCDX certifcation, but do create a planning and commit to it. Try to analyse what you still need before applying for the VCDX defense. This can be certifications, a vSphere design, more VMware vSpere knowledge, etc. Create a breakdown structure of the things you need to do with a date that you’ve got in mind. Write them all down, put it on a wall and look at it from time. Setting a goal will help you motivate yourself to do the things necessary to achieve VCDX in the end.

Tip #6 : The application is a summary of your design, use it as strategy approach for your defense panel

The panel is to show your design skills to the panel. The application is usefull for strategy approach with the question in mind : What do you want to show the panel? Walk through your requiremetns, constraints and assumptions and see look at the ones that had the biggest impact on your design. Those are the ones that are the most interesting and the most fun to talk about during your defense.

For more information I would like to refer to the VCDX Candidate Tips which is full of useful tips.

# Tip 7 : Seek help from others.

Your not alone. You probably work with a lot of talented people that can challenge and help you to grow. Don’t just use them for a mock defense, but also let them help you achieve your VCDX. Ask for advice, let people read what you are doing and share the experience.

I’m not perfect, but there is no failure. There is only feedback!

Unfortunately I didn’t achieve my VCDX the first time. “We regret to inform you…” were the words that haunted my mind for quite some time. During that time I had a lot of things going through my mind. But when the dust from the “Cloud of Dissapointment” settles, it is time to pick up the pieces. And actually there is a lot of information in the experience you went through. I must admit, it wasn’t a fun experience, but it did have a lot of information in there. Here are some tips to help re-set your goal.

#Tip 8 : Have some time between the VCDX defenses. Don’t take another VCDX defense between 4 -6 months after you didn’t pass.

Instead of looking at it as failure, turn it around, look at it and see it as feedback. It wasn’t that you weren’t good enough. You already made it through the application phase and were allowed to go to the defense panel. You are good enough, but you just need to bring your A game, in stead of the B game that you brought to the defense that didn’t let you pass to become VCDX. And the information to bring your A game is all in the feedback that you got during the defense and in the email with the notice that you didn’t pass. OK, I’ll admit, the feedback in the email isn’t much. It has pointers, but combined with the experience you have, you can probably work out the areas that need your attention for your next try. Go through your design and adres the areas you need to work on using the following tip…

#Tip 9 : Using a real-life project doesn’t mean you can’t tweak things to your “advantage”

Some decisions in you design maybe are hard to explain or aren’t there because the customer didn’t have the information. That shouldn’t stop you from adding and tweaking your design to your “advantage”. Panels only have the information that is presented to them through the application, which basically means that you can create an ideal situation. If there a things in your design that you would like to have tweaked, feel free to do so, but do keep in mind that you need to clarify it with the requirements and constraints that the customers provided to you (or that you also added ) It needs to fit and you need to be able to explain it to the panel. Create your own world, but without losing the grip on reality!

# Tip 10 : Learn for the past, work hard in the present, focus on the future

Please learn from the experiences that you had during the defense you did not pass. This is valuable information before walking into the defense panel again. You already know what is expected of you, you already know what you did wrong the last time, but stop and I repeat STOP going back to “the bad experience”. There is no point in re-living the bad experience over and over again. In stead learn your lessons. Use them to your advantage. Work hard to getting things back on track to your VCDX and focus on the future. Try to imagine what it would be like standing in front of the panel again. You can do this!

To conclude my write-up try to look at gaining your VCDX as a learning experience. There are lots of lessons in there and there are barriers that you need to break before you achieve your goal. If it was that easy to gain your VCDX then there wasn’t any fun in doing it at all. So when you sit down again and think about your bucket list, write this one on it : Achieve VCDX! (I know you want this, you wouldn’t have read my complete article if you didn’t want to do this… Have fun!)

vCloud Suite 5.1 Licensing Explained

VMware has announced the release of the new vSphere 5.1 solution. Together with this new release, VMware has also announce it’s new VMware vCloud Suite 5.1 licensing model. This model combines multiple components (vSphere Enterprise Plus, vCloud Director, vCloud Networking and Security, etc.) into a single product with a single license. All virtual machines running on a properly licensed vCloud Suite processor can use all components included in that vCloud Suite edition.

Licensing per processor

As mentioned above the licensing unit takes place per-processor. VMware no longer limits it’s customers physical resources and on the number of virtual machines!!! VMware has listened to the VMware Community and no longer applies the vRAM principle. Or like other call it, the vTax. The VMware vCloud Suite 5.1 is licensed per physical processor. With all physical processors licensed in a server a customer can run all VMware products on top of this server that are licensed within the bundle.

vCloud Suites Editions

There are 3 editions available for the vCloud Suites :

1. VMware vCloud Suite Standard; vSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Standard.

2. VMware vCloud Suite Advanced; vSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Advanced and vCOPs Advanced.

3. VMware vCloud Suite Enterprise; vSphere Enterprise Plus, vCloud Director, vCloud Connector & VMware vCloud Network and Security Suite Enterprise, vCOPS Enterprise, vFabric Application Director and SRM.

So what’s the deal?

In my opinion VMware tried to simplify the whole licensing part of building a vCloud solution. Most customers that build a private cloud in general want to build such a vCloud solution in an easy manner, but it also needs to be easy to manage, must be monitored and should work in case of a disaster.

All of these components are in the bundle that is licensed with vCloud Suite Enterprise edition. An easy licensing path on the road to your own private vCloud. Most companies already have VMware vSphere licenses and VMware also provide an upgrade path toward the new VMware vCloud Suite licenses. For upgrading VMware has introduced the Fair Value Conversion Program that can be found at http://www.vmware.com/go/ vcloud-suite-licensing.

For more information on the VMware vCloud Suite licensing see the vCloud Suite 5.1 Pricing and Packaging Whitepaper or talk to your own VMware sales representative.

VMware 5.1 release party!

Word is out, vSphere 5.1 and vCloud 5.1 have been released. So what’s new in this release? A lot I can tell you that!

And that’s what the Technical Marketing has been working on the last couple of months.

Here is the list with papers that cover all the new features :

Thanks to Duncan Epping for providing this list on his blog.

Thanks to everybody at Tech Marketing for making this information available to us!!!

New storage books for designing cloud infra

When creating a design for your cloud environment you always have to take the physical components, such as compute, network & storage into account. These components are the foundation that your cloud environment will be build on. A good design of these components is crucial for your overall design, the performance and resilience of your solution. Fact remains that you can’t know it all, but when you do want to know it, then the best way is to learn it from the experts.

Now we have the chance to do so. Three experts in the field of storage released two books about storage in relation to virtual cloud environments.

Mostafa Khalil from VMware, released the book “Storage Implementation in vSphere 5.0”

“The more important VMware virtualized infrastructure becomes, the more important virtualization storage becomes. Virtualization storage planning and management is complex, and it’s been almost impossible to find authoritative guidance – until now. In Storage Implementation in vSphere 5.0, one of VMware’s leading experts completely demystifies the “black box” of vSphere storage, and provides illustrated, step-by-step procedures for performing virtually every task associated with it. Mostafa Khalil brings together detailed techniques and guidelines, insights for better architectural design, planning and management best practices, common configuration details, and deep dives into both vSphere and external storage-related technologies. He gives technical professionals the deep understanding they need to make better choices, solve problems, and keep problems from occurring in the first place. This book answers crucial, ground-level questions such as: How do you configure storage array from “Vendor X” to support vSphere “Feature Y”? How do you know you’ve configured it correctly? What happens if you misconfigure it? How can you tell from logs and other tools that you have a problem – and how do you fix it? Most of the author’s troubleshooting techniques are based on his own personal experience as a senior VMware support engineer helping customerstroubleshoot their own vSphere production environments – experience that nobody else has.”

At the same time Vaughn Stewart and Mike Slisinger from NetApp released the book “Virtualization Changes Everything: Storage Strategies for VMware vSphere & Cloud Computing”:

“Storage is a foundational component in the support of virtualization and cloud computing – and it is dynamically evolving. It is an aspect of the datacenter that is all-too-often overlooked, but without storage, there is no data, and without data, there is no cloud. Virtualization Changes Everything, by Vaughn Stewart and Mike Slisinger, examines the evolutionary influence of host virtualization and cloud computing in breaking storage deployment out of outdated silo models and into a dynamic, flexible hosting environment. Virtualization Changes Everything reviews common goals and challenges associated with providing storage service with cloud computing, and addresses each through the application of advanced storage technologies designed to scale in order to support the ever-expanding storage needs of the future. The examples within the book are pulled from real-world experience, and often involve the integration of multiple innovative technologies. If you are looking for measured guidance on high availability, efficiency, integration and performance for the storage in your cloud, then this book is for you!”

Both execellent books on the topic of storage and the impact it has on your virtual cloud environment. A must read for everybody that wants to gain more knowledge on this topic and the impact storage has on virtual cloud environments.

Resource management in a vSphere vApp

What is a vApp?

A vApp is a container in vSphere. It works the same way as a resource pool, but has extra options that help define a more structured approach to hosting virtual machines. With vApps you can build application stacks of virtual machines that have a relations with one another.

The most common example is always the three tiered app; a webserver, application server and a database server. With a vApp these virtual machines can be grouped together and besides grouping them together you can also control the startup order of the VMs in the vApp and allocate a specific amount of resources to the vApp.

Note : vSphere vApps are not the same as vCloud vApps! Both group workloads together, but they are not the same thing.

Resource allocation

The allocation of resources for a vApp works with the same construct as that of a resource pool. The vApp can be allocated a specific amount of CPU and RAM resources. By default the vApp is set to unlimited and resources are expandable if needed, just like a resource pool. These settings can be changed in the same way as with normal resource pools. Reservations, limits and shares can be defined on a vApp level and can help to allocates resources depending on the requirments of the application stack.

VMs in a vApp share the resources that have been allocated to the vApp only with the other VMs in the vApp. In this way VMs are isolated from other VMs, vApps and resource pools outside of its own vApp. When resource contention takes place all VMs in a vApp will have to compete over the amount om resource that are available to the vApp.

If expendable reservations are configured, the vApp can allocate more resources if the parent resource pool has those available. However if there are no resources available the VMs in the vApp will need to compete over the resource available to the vApp. This is where normal resource mechanisms apply such as shares, limits and reservations.

Lets take the vApp with the three tier vApp (web-app-db) as an example. By default all VMs are equal in a vApp. However the database is the most important VM in this three tiered vApp and needs to be given enough resources when resource contention takes place. To define this one can set the shares for the database VM on High. By default this is set to Normal. This will give the database VM twice as much resource shares as the other two VMs in the vApp. This will elevate its priority within the vApp and provide it with half of the resources when resource contention takes places. In this way one can set a specific priority to VMs within a vApp.

More information about the allocation of resources within vSphere can be found in the book VMware vSphere 5 Clustering Technical Deepdive by Frank Denneman & Duncan Epping.