Tag Archives: esxi

How to monitor ESXi host hardware with SNMP

The Simple Network Management Protocol (SNMP) is a protocol for monitoring and configuration of network-attached devices. SNMP exposes data in the form of variables and values. These variables can then be queried or set. A query retrieves the value of a variable, a set operation assigns a value to a variable. The variables are organized in a hierarchy and each variable is identified by an object identifiers (OID). The management information base (MIB ) describes this hierarchy. MIB files (simple text files) contain metadata for each OID. These are necessary for the translation of a numeric OID into a human-readable format.  SNMP knows two devices types: read more

VMware Update Manager reports “error code 99” during scan operation

After updating my lab to VMware vSphere 6.0 U2, one of my hosts continuously thrown an error during an update scan.

The first thing I’ve checked was the esxupdate.log on the affected ESXi host. This is the output, that was logged during a scan operation.

You might notice the “Unrecognized file vendor-index.xml in Metadata file” error. I also found this error message on the other hosts, so I excluded it from further research. It was unlikely, that this error was related to the observed problem. I started searching differences between the hosts and found out, that the output of “esxcli software vib list” was different on the faulty host. read more

HP ProLiant BL460c Gen9: MicroSD card missing during ESXi 5.5 setup

Today, I was at a customer to prepare a two node vSphere cluster for some MS SQL server tests. Nothing fancy, just two HP ProLiant BL460c Gen9 blades and two virtual volumes from a HP 3PAR. Each blade had two 400 GB SSDs, two 64 GB M.2 SSDs and a 1 GB MicroSD card. Usually, I install ESXi to a SD card. In this case, a MicroSD card. The SSDs were dedicated for PernixData FVP. Although I saw the MicroSD card in the boot menu, ESXi doesn’t showed it as a installation target.

bl460_no_sdcard

I’ve read a lot about similar observations of other users, but no solution seemed to be plausible. It’s not a solution to switch from UEFI to legacy BIOS or play with power management settings. BIOS and ILO firmware were up to date. But disabling USB3 support seemed to be a possible solution. read more

Storage vMotion stuck at 100% – cleaning up migration state

Moving VMs from an old cluster with old ESXi hosts to a new cluster with new hosts can be so easy, even if the clusters doesn’t share any storage. A PowerCLI one-liner or the Web Client allow you to migrate VMs between hosts and datastores, while the VMs are running. This enhancement was added with vSphere 5.1. I’m often suprised how many customers doesn’t know this feature, just because they are still using the old vSphere C# client.

Some days ago, I had to move some VMs to a new cluster and I used this well known feature to move the running VMs to a new hosts. Everything was fine, until I realized, that the vMotion process stopped at 100%. Usually, the cleanup is finishes very quickly. But this time, the vMotion stopped. read more

ESXi 5.5 U3b and later are no longer manageable without vCenter 5.5 U3b

On December 8, 2015, VMware released VMware ESXi 5.5 patch ESXi550-201512001 (2135410). This patch is known as U3b and it contains general and security fixes, nothing special. Usually, you would install this update without notice. But this time, you should better take a look into the release notes of ESXi 5.5 U3b, before you install this update. This is taken from the release notes:

Note: In your vSphere environment, you need to update vCenter Server to vCenter Server 5.5 Update 3b before updating ESXi to ESXi 5.5 Update 3b. vCenter Server will not be able to manage ESXi 5.5 Update 3b, if you update ESXi before updating vCenter Server to version 5.5 Update 3b. For more information about the sequence in which vSphere environments need to be updated, refer, KB 2057795. read more

Reset the HP iLO Administrator password with hponcfg on ESXi

Sometimes you need to reset the ILO Administrator password. Sure, you can reboot the server, press F8 and then reset the Administrator password. If you have installed a HP customized ESXi image, then there is a much better way to reset the password: HPONCFG.

Check the /opt/hp/tools directory. You will find a binary called hponcfg.

All you need is a simple XML file. You can use the VI editor or you can copy the necessary file with WinSCP to the root home directory on your ESXi host. I prefer VI. read more

VMware publishes patch for ESXi 6.0 CBT bug (KB2114076)

Today, VMware has published the long-awaited patch for the ESXi 6.0 CBT bug. This patch is the result of a problem, which is described in KB2114076 (Backing up a virtual machine with Change Block Tracking (CBT) enabled fails after upgrading to or installing VMware ESXi 6.0). All customers that upgraded to ESXi 6.0 or installed ESXi 6.0 were affected.

Symptoms of this bug were:

  • Powering on virtual machines fails
  • Expanding the size of a virtual disk fails
  • Taking virtual machine quiesced snapshots fails
  • Error messages like “An error occurred while taking a snapshot: msg.snapshot.error-QUIESCINGERROR” (vSphere Client), “WARNING: CBT: 191: No memory available! Called from 0x4180219af50e” (vmkernel.log) or “Creating cbt node 92b78c-cbt failed with error Cannot allocate memory (0xbad0014, Out of memory)” in the vmware.log of the affected virtual machine

A workaround was to disable CBT, which resulted in longer running backups.

You can download the patch (ESXi600-201505401-BG) from VMware to manually update your ESXi hosts. Otherwise you can use VMware Update Manager to patch your hosts. A reboot is necessary! read more

What to consider when implementing HP 3PAR with iSCSI in VMware environments

Some days ago a colleague and I implemented a small 3-node VMware vSphere Essentials Plus cluster with a HP 3PAR StoreServ 7200c. Costs are always a sore point in SMB environments, so it should not surprise that we used iSCSI in this design. I had some doubt about using iSCSI with a HP 3PAR StoreServ, mostly because of the performance and complexity. IMHO iSCSI is more complex to implement then Fibre Channel (FC). But in this case I had to deal with it.

iSCSI options for HP 3PAR StoreServ 7000 read more

Add a new version of HP Agentless Management Service to a customized ESXi 5.5.0 ISO

While preparing for a VMware vSphere 5.5 update at a customer of mine, I stumbled over VMware KB2085618 (ESXi host cannot initiate vMotion or enable services and reports the error: Heap globalCartel-1 already at its maximum size.Cannot expand.). I checked the HP AMS version in the latest HP custom ESXi image and found out, that version hp-ams-esx-550.10.0.0-18.1198610 is included (source). Unfortunately the bug is not fixed in 10.0.0, but it’s fixed in 10.0.1 (source).

hp_ams_10_0_1

According to the VMware KB article only the HP AMS versions hp-ams 500.9.6.0-12.434156 and hp-ams-550.9.6.0-12.1198610 should be affected. But since I do not like surprises, I decided to update the HP AMS version in the latest HP custom ESXi image from 10.0.0 to 10.0.1. read more

Memory management: VMware ESXi vs. Microsoft Hyper-V

Virtualization is an awesome technology. Last weeks I visited a customer and we took a walk through their data centers. While standing in one of their data centers I thought: Imagine that all server, that they are currently run as VMs, would be physical?. I’m still impressed about the influence of virtualization. The idea is so simple You share the resources of the physical hardware between multiple virtual instances. I/O, network bandwidth, CPU cycles and memory. After nearly 10 years of experience with server virtualization I can tell that especially the memory resources is one of the weak points. When a customer experiences performance problems, they were mostly caused by a  lack of storage I/O or memory. read more