Tag Archives: vExpert

VMware ESXi 6.7 memory health warnings after ProLiant SPP

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

During the deployment of a vSAN cluster consisting of multiple HPE ProLiant DL380 Gen10 hosts, I noticed a memory health warning after updating the firmware using the Support Pack for ProLiant. The error was definitely not shown before the update, so it was clear, that this was not a real issue with the hardware. Furthermore: All hosts showed this error.

Memory health status after SPP

The same day, a customer called me and asked me about a strange memory health error after he has updated all of his hosts with the latest SPP…

My first guess, that this was not caused by a HW malfunction was correct. HPE published a advisory about this issue:

The Memory Sensor Status Reported in the vSphere Web Client Is Not Accurate For HPE ProLiant Gen10 and Gen10 Plus Servers Running VMware ESXi 6.5/6.7/7.0 With HPE Integrated Lights-Out 5 (iLO 5) Firmware Version 2.30

To fix this issue, you have to update the ILO5 firmware to version 2.31. You can do this manually using the ILO5 interface, or you can add the file to the SPP. I’ve added the BIN file to the USB stick with the latest SPP.

If you want to update the firmware manually, simply upload the BIN file using the built-in firmware update function.

  1. Navigate to Firmware & OS Software in the navigation tree, and then click Update Firmware
  2. Select the Local file option and browse to the BIN file
  3. To save a copy of the component to the iLO Repository, select the Also store in iLO Repository check box
  4. To start the update process, click Flash

You can download the latest ILO5 2.31from HPE using this link. After the FW update, the error will resolve itself.

Only ESXi 6.7 is affected, and only ESXi 6.7 running on HPE ProLiant hosts, regardless if ML, DL or BL series.

Setup the View Agent Direct-Connection (VADC) Plug-In

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

The View Agent Direct-Connection (VADC) Plug-In was designed as an extension to the Horizon Agent, which allows a Horizon Client to directly connect to a VM or physical machine withtout using a Horizon Connection Server.

The VADC is nothing new, it is part of the Horizon View eco system for a couple of years now. Meanwhile, the VADC supports the Blast Exteme protocol, which makes it pretty interesting for remote access to lab environment or home office equipment.

There are a couple of requirements which I want to highlight:

VADC Plug-In has the following additional requirements:

  • The VM or or physical machine must have a minimum of 128 MB of video RAM
  • For a virtual machine, you must install VMware Tools before you install Horizon Agent
  • A physical machine supports Windows 10 Enterprise version 1803 or version 1809, newer releases tend to work flawless
  • A VM supports Blast and PCoIP protocols
  • A physical machine supports Blast only

The installation of the VADC is divieded into two steps:

  • Installation of the View Agent
  • Installation of the VADC

The View Agent has to be installed silently, because you are unable to add it to a Connection Server. The silent installation allows you to skip this step.

I used this command line to install the View Agent:

VMware-Horizon-Agent-x86_64-8.0.0-16530789.exe /v VDM_SKIP_BROKER_REGISTRATION=1 RDP_CHOICE=1 ADDLOCAL=Core,ClientDriveRedirection,VmwVaudio,PrintRedir,USB,RTAV

The second step is to install the VADC. This is pretty easy: Setup > Next, next, next. :)

Finally, you can start the View Client on another machine and add a Connection Server with the IP or FQDN of you newly installed VADC machine.

This is the output of netstat on my X250 after connecting using the VADC:

TCP 192.168.20.52:443 t480s:50996 ESTABLISHED
TCP 192.168.20.52:443 t480s:50997 ESTABLISHED
TCP 192.168.20.52:443 t480s:50998 ESTABLISHED
TCP 192.168.20.52:22443 t480s:51014 ESTABLISHED
TCP 192.168.20.52:32111 t480s:51027 ESTABLISHED

You might notice the typical Horion View Ports 22443 for Blast Extreme and 32111 for USB redirection.

Installation von Horizon View Agent fails continuously with “The System must be rebooted before installation can continue”

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

This issue was a bit annoying. I faced this issue not in a customer environment, rather then on my second Lenovo laptop, an X250 with Windows 10 20H2. My intention was to use it headless in a docking station. So how should I access it? RDP? TeamViewer? Why not use the Horizon Direct Connection Plug-in?

The Horizon Direct Connection Plug-in is not a new feature and you can think of it as a View Agent without a Connection Server. You can access it using the View Client, but you don’t have to run the connection through a Connection Server. For pretty small environments or direct access a perfect fit!

Error message View Agent setup

In order to use the Horizon Direct Connection Plug-in, you have to install the View Agent. So I downloaded the latest View 2006 Agent (VMware-Horizon-Agent-x86_64-8.0.0-16530789.exe) and started the setup.

It fails right at the beginning. Okay, I just installed Windows Updates, so I rebooted my laptop. But the setup fails again. Next reboot. And it fails, and fails, and fails.

There are some registry keys you can check of you get such an error. “PendingFileRenameOperations” is one of the common issues when you face this problem. I found a script, but there was no reboot pending.

I finally found it: RunOnce.

HKLM RunOnce

There was an entry under HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce. After deleting this entry, the setup went through.

Fun Fact: There is an MSI Property for this, when you want to Silently install the View Agent : SUPPRESS_RUNONCE_CHECK.

Update Manager fails with unknown error during host remediation

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

During an vSphere 6.5 > 6.7 update a was host failing continously at the remediation with an “unknown error”. The host was updated from ESXI 6.5 to 6.7 using an upgrade baseline. Other hosts were updated to 6.7 and with the latest patches without any issues. Something strange was going on…

The esxupdate.log and the vua.log on the host itself showed nothing special. So I checked the vmware-vum-server-log4cpp.log which was much more informative!

[2020-07-19 13:03:25:217 'SingleHostScanTask.SingleHostScanTask{262}' 139762329831168 ERROR] [singleHostScanTask, 693] caught an odbc error: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vci_scanresults"; Error while executing the query" is returned when executing SQL statement "INSERT INTO VCI_SCANRESULTS(endtime, id, scan_status, scan_type, starttime, target_component, target_uid) VALUES (?, ?, ?, ?, ?, ?, ?)"
[2020-07-19 13:03:25:219 'SingleHostScanTask.SingleHostScanTask{262}' 139762329831168 ERROR] [singleHostScanTask, 404] SingleHostScan caught exception: caught an odbc error: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vci_scanresults"; Error while executing the query" is returned when executing SQL statement "INSERT INTO VCI_SCANRESULTS(endtime, id, scan_status, scan_type, starttime, target_component, target_uid) VALUES (?, ?, ?, ?, ?, ?, ?)" with code: -1
[2020-07-19 13:03:25:223 'SingleHostScanTask.SingleHostScanTask{262}' 139762329831168 ERROR] [vciTaskBase, 568] Task execution has failed: caught an odbc error: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vci_scanresults"; Error while executing the query" is returned when executing SQL statement "INSERT INTO VCI_SCANRESULTS(endtime, id, scan_status, scan_type, starttime, target_component, target_uid) VALUES (?

Well… ERROR: duplicate key value violates unique constraint “pk_vci_scanresults” is not what I expected, but it is an error, and it occured everytime I tried to remediate the host.

Google found nothing about this error, so I decided to reset the VUM database. Please don’t try this at your customer! Log a call at VMware.

To reset the VUM database:

  1. Connect to vCenter Server Appliance via SSH
  2. Switch to the BASH 
  3. Stop the VMware Update Manager Service with this command

    service-control –stop vmware-updatemgr
     
  4. To reset the VMware Update Manager Database (applies only to VCSA 6.7 and 7.0!)

    /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db
  1. Delete the contents of the VMware Update Manager Patch Store

    rm -rf /storage/updatemgr/patch-store/*
     
  2. Start the VMware Update Manager Service again

    service-control –start vmware-updatemgr

You will lose all your baselines, so you have to configure them again. And you need to download all patches again.

For vSAN environments this procedure will also remove the vSAN default baselines, but they will recreated automatically when there is a configuration change to vSAN or an update to the HCL DB. Again: Don’t do this at home!

VCAP6.5-DCV Design – Objective 2.4 Build manageability requirements into a vSphere 6.x logical design

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

This seems to be my last blog post for 2019 and it covers covers objective 2.4 (Build manageability requirements into a vSphere 6.x logical design) of the VCAP6.5-DCV Design exam. It is based on the VMware Certified Advanced Professional 6.5 in Data Center Virtualization Design (3V0-624) Exam Preparation Guide (last update August 2017).

The necessary skills and abilities are documented in the exam prep guide for the older VCAP6-DCV Design exam (3V0-622). I think they also apply to the current version of the exam:

  • Evaluate which management services can be used with a given vSphere Solution
  • Differentiate infrastructure qualities related to management
  • Differentiate available command line-based management tools (PowerCLI, vMA etc.)
  • Evaluate VMware Management solutions based on customer requirements
  • Build interfaces into the logical design for existing operations practices
  • Address identified operational readiness deficiencies
  • Define Event, Incident and Problem Management practices
  • Analyze Release Management practices
  • Determine request fulfillment and release management processes
  • Determine requirements for Configuration Management
  • Define change management processes based on business requirements
  • Based on customer requirements, identify required reporting assets and processes

While the last blog post has covered the availability requirements, this blog posts focuses on the manageability requirements of a logical design. It’s all about how to manage the proposed solution.

Evaluate which management services can be used with a given vSphere Solution

You can use different “services” to manage a vSphere environment.

  • vCenter and vMA

Both appliances offer you different services to connect to in order to manage your environment, like

  • vSphere Client (Web Client, C# Client)
  • SSH
  • APIs
  • PowerCLI

The different tools help you to manage the different vSphere components, like

  • HA
  • DRS
  • Networking (vDS, vSS)
  • Auto Deploy
  • Host Profiles
  • etc.

Differentiate infrastructure qualities related to management

The different infrastructure qualities are

  • Availability
  • Manageability
  • Performance
  • Recoverability
  • Security

Depending on which infrastructure quality you consider, it affects the manageability of the proposed solution. For example: A single vCenter might not offer the required availability. Or a single datastore might not meet the required performance. But a highly-available vCenter or a SDRS cluster affects the way how you management the solution.

Differentiate available command line-based management tools (PowerCLI, vMA etc.)

You should be able to differentiate between PowerCLI (PowerShell) and vMA (Appliance) or vCLI (command-line tools for ESXi).

Evaluate VMware Management solutions based on customer requirements

Depending on the customers requirements, some solutions might be out of scope. If the customer doesn’t have a vSphere Enterprise Plus license, there’s no way to use Storage DRS.

Build interfaces into the logical design for existing operations practices

This topic is about what existing interfaces (in terms of systems) the customer already using and how to build them into the design. Think about Syslog servers, Active Directory for authentication (infrastructure quality design), Public Key Infrastructure (PKI) for certificates etc.

Address identified operational readiness deficiencies

Operational Readiness (OR) is the capability of an organization to (efficiently) deploy, operate, and maintain a system and/ or its processes. Before the proposed solution is going to production, any deficits in regard of OR has to be identified and addresses.

Define Event, Incident and Problem Management practices

This sounds like ITIL, and I would assume that the definition of event, incident and problem of ITIL is meant. ITIL defines

  • Event: An event can be defined as any detectable or discernible occurrence that has significance for the management of the IT Infrastructure or the delivery of IT service and evaluation of the impact a deviation might cause to the services. Events are typically notifications created by an IT service, Configuration Item (CI) or monitoring tool. (Wikipedia)
  • Incident: An incident is an event that could lead to loss of, or disruption to, an organization’s operations, services or functions. (Wikipedia)
  • Problem: The Information Technology Infrastructure Library defines a problem as the cause of one or more incidents. (Wikipedia)

The design should include practices for event, incident and problem management. Most customers will already have practices for this, but they might be adjusted for the proposed solution.

Analyze Release Management practices

Release management is the process of managing, planning, scheduling and controlling the deployment of new or modified services. This topic covers the currently deployed Release Management processes of the customers.

Determine request fulfillment and release management processes

This topic is related to the prior topic. You should determine if the customers has already deployed request fulfillment and release management processes, and if they are already deployed, you should check if they are suitable for the proposed solution.

The request fulfillment will allow users to request and receive standardized services. Think about the automated deployment of VMs after requesting a new VM using a portal web site.

Determine requirements for Configuration Management

Changes to the proposed solution will be required over time. Configuration Management covers the management of all Configuration Items (CI). Event if it’s not mentioned in this topic, Configuration Management is related to Change Management, because all changes to CIs has to be documented.

Define change management processes based on business requirements

The objective of change management in this context is to ensure that standardized methods and procedures are used for efficient and prompt handling of all changes to control IT infrastructure, in order to minimize the number and impact of any related incidents upon service. (Wikipedia)

If a customer already has ITSM processes in place, they most likely will have a change management process. This process has to be defined to fulfill the requirements of the proposed solution.

Based on customer requirements, identify required reporting assets and processes

Especially when it comes down to security, it’s important to talk about monitoring and logging. This topic is about

  • What CIs have to be monitored?
  • What events have to be logged/ tracked?
  • How to keep track of changes to configuration items?
  • How keep documentation up-to-date?

Summary

This objective is full of ITSM/ ITIL. It’s pretty helpful if you were familiar with the concepts of ITSM/ ITIL. You should have a good understanding of the different management tools and management solutions and services of a vSphere design.

VCAP6.5-DCV Design – Objective 2.3 Build availability requirements into a vSphere 6.x logical design

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

This blog post covers objective 2.3 (Build availability requirements into a vSphere 6.x logical design) of the VCAP6.5-DCV Design exam. It is based on the VMware Certified Advanced Professional 6.5 in Data Center Virtualization Design (3V0-624) Exam Preparation Guide (last update August 2017).

The necessary skills and abilities are documented in the exam prep guide for the older VCAP6-DCV Design exam (3V0-622). I think they also apply to the current version of the exam:

  • Evaluate which logical availability services can be used with a given vSphere solution
  • Differentiate infrastructure qualities related to availability
  • Describe the concept of redundancy and the risks associated with single points of failure
  • Explain class of nines methodology
  • Determine availability component of service level agreements (SLAs) and service level management processes
  • Determine potential availability solutions for a logical design based on customer requirements
  • Create an availability plan, including maintenance processes
  • Balance availability requirements with other infrastructure qualities
  • Analyze a vSphere design and determine possible single points of failure

Let’s start with…

Evaluate which logical availability services can be used with a given vSphere solution

VMware vSphere offers a broad band of features that allows you to create highly available solutions. When we take a look at the infrastructure, feature like VMware HA, FT, or even multiple NICs at a distributed vSwitch allow to increase availablility. When we look at the application layer, other techniques, like DRS can help us to increase availability to use DRS to place VMs on different hosts (anti-affinity rules) etc.

Differentiate infrastructure qualities related to availability

The infrastructure qualities are:

  • Availability
  • Manageability
  • Performance
  • Recoverability
  • Security

Availability and Recoverability are tight together. René van den Bedem has written an very good blog post about how recoverability affectes availability.

Describe the concept of redundancy and the risks associated with single points of failure

This topic is pretty clear and should be easy to explain. You should be able to identify what a single point of failure is, and how you can avoid them. Examples for a single point of failure are:

  • only a single-port HBA in a server
  • only one network uplink from a Top-of-Rack switch to a Core-Switch
  • using of RAID 0

Explain class of nines methodology

This is also easy:

  • Two Nines- 99% – 3.65 days downtime per year
  • Three Nines- 99,9% – 8.76 hours downtime per year
  • Four Nines- 99,99% – 52.6 minutes downtime per year
  • Five Nines – 99,999% – 5.26 minutes downtime per year
  • Six Nines – 99,9999% – 31.56 seconds downtime per year

Important note: “Downtime” means “unplanned downtime”, not planned downtime, like in maintenance windows.

Determine availability component of service level agreements (SLAs) and service level management processes

An Service Level Agreement (SLA) is a contact between two parties, usually a supplier and a customer. The SLA describes targets that should be met. This can be an availability expressed using the “class of nines methodology”. If this target is missed,the supplier ofthen has to pay a penalty to the customer.

So it is pretty important to build a design that can fulfill the availability requirements. Depending on the requirements you may have to use VMware FT. If the availability requirements are lower, VMware HA may be sufficient. It is important that you can choose the best technique for the given SLA.

Determine potential availability solutions for a logical design based on customer requirements

Now it’s time to put things together. You know the different techniques that are offered by VMware vSphere, and you know the customer requirements. This allows you to determine the potential availability solutions for a logical design.

Create an availability plan, including maintenance processes

Again, I’d like to recommend the blog post of René van den Bedem. It’s all about RPO, RTO, MTD and how much does an unplanned downtime costs (result of a Business Impact Analysis).

Balance availability requirements with other infrastructure qualities

At some point of your design you need to holistically look at your design and you have to ensure that a decision, that was made, doesn not impact other requirements or other decision.

Analyze a vSphere design and determine possible single points of failure

This is pretty self-explanatory and can be done together with the preceding step.

Summary

Availability is the main theme of this objective. Do not lose sight of the customer’s requirements. Increasing availability is often associated with immense additional costs.

Read the mentioned blog post from René and I rellay recommend this vBrownBag video with Rebecca Fitzhugh.

Why we need a vSAN licensing for SMB customers

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Not every customer is running a full-blown vSphere Enterprise Plus licensing. To be honest, when I look at the number of sold licenses, most of my customers are running vSphere Essentials Plus. Not Essentials, nor Standard or Enterprise (Plus), but two or three hosts with Essentials Plus. And that’s perfectly fine!

Two or three hosts with 10 GbE and pretty often 12G SAS. Some of them with Fibre-Channel, nearly no one with iSCSI. My colleagues and I developed a pretty rock solid setup over the last years, which we sell like some kind of building block: HPE ProLiant, HPE MSA, Aruba Switches, vSphere Essentials Plus. A perfect setup for most of our customers, which run something between 10 and 30 VMs on it. Some of them also add Horizon View (Add-On) to it.

But requirements change. More customers ask for more hosts. When customers break out of the Essentials Plus licensing, then often because of the host limitation. Less of them do this because they need DRS or even Storage vMotion.

Some of my customers have heard about vSAN and they like the idea behind it. Especially when you take into account, that hardware costs decrease and flash storage is getting cheaper. But when you discuss the idea of combining vSAN and Essentials licensing, you will hit the host limitation early.

VMware itself states in the vSAN licensing guide:

The 2-node vSAN deployment model is not restricted to a specific vSAN license edition. In other words, any of the licensing editions can be used with a 2-host configuration. vSphere Essentials Kit or vSphere Essentials Plus Kit licensing limits the number of hosts managed by
vCenter Server Essentials to three. The vSAN witness host – virtual appliance or physical – is considered a host in these Essentials licensing bundles.

Source: VMware vSAN Licensing Guide

When you take a look at the Horizon Desktop licensing, or at the RoBo licensing, you will see another kind of limitation: Limiting the number of VMs, not the number of hosts. This is pretty interesting when you think about combining vSAN and Essentials licensing.

Why not offering a “HCI Essentials Kit” limitied to 25 VMs, and the features offered by Essentials Plus and vSAN Standard? This would allow customers to run four or five hosts with vSAN. By limiting the number of VMs, customers can scale-out their infrastructure in terms of capacity.

Hey VMware, you might think about this over the Christmas holiday. ;) There is a customer segment that is not yet sufficiently addressed by your sales team. This is a chance for more YoY growth. ;)

VMware ESXi 6.7: Recurring host hardware sensor state alarm

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

If you found this blog post because you are searchting for a solution for a FAN FAILURE on your ProLiant Gen10 HW after applying the latest ESXi 6.7 patches, then use this shortcut for the workaround: Fan health sensors report false alarms on HPE Gen10 Servers with ESXi 6.7


I had a really annoying problem at one of my customers. After deploying new VMware ESXi hosts (HPE ProLiant DL380 Gen10) along with an upgrade of the vCenter Server Appliance to 6.7 U2, the customer reported recurring host hardware sensor state alarm messages in the vCenter for all hosts.

After acknowledging the alarm, it recurred after a couple of minutes or hours. The hardware was finde, no errors or warnings were noticed in the ILO Management Log. But the vCenter reported periodically a Sensor -1 type error in the Events window. The /var/log/syslog.log contained messages like this:

2019-11-29T04:39:48Z sfcb-vmw_ipmi[4263212]: IpmiIfcSelGetInfo: IPMI_CMD_GET_SEL_INFO cc=0xc1
 2019-11-29T04:39:49Z sfcb-vmw_ipmi[4263212]: IpmiIfcSelGetInfo: IPMI_CMD_GET_SEL_INFO cc=0xc1
 2019-11-29T04:39:50Z sfcb-vmw_ipmi[4263212]: IpmiIfcSelGetInfo: IPMI_CMD_GET_SEL_INFO cc=0xc1
 2019-11-29T04:39:51Z sfcb-vmw_ipmi[4263212]: IpmiIfcSelGetInfo: IPMI_CMD_GET_SEL_INFO cc=0xc1
 2019-11-29T04:39:52Z sfcb-vmw_ipmi[4263212]: IpmiIfcSelGetInfo: IPMI_CMD_GET_SEL_INFO cc=0xc1

Sure, you can ignore this. But you shouldn’t ignore this, because these events can result in the vCenter database increasing in size. vCenter can crash once the SEAT partition size goes above the 95% threshold. So you better fix this!

Long story short: This bug is fixed with the latest November updates for ESXi 6.7 U3. A workaround is to disable the WBEM service. The WBEM service might be enabled after a reboot. In this case you have to disable the sfcbd-watchdog service.

But the best way to solve this is to install the latest patches (VMware ESXi 6.7, Patch Release ESXi670-201911001)

VCAP6.5-DCV Design – Objective 2.2 Map service dependencies

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

This blog post covers objective 2.2 (Map service dependencies) of the VCAP6.5-DCV Design exam. It is based on the VMware Certified Advanced Professional 6.5 in Data Center Virtualization Design (3V0-624) Exam Preparation Guide (last update August 2017).

The necessary skills and abilities are documented in the exam prep guide for the older VCAP6-DCV Design exam (3V0-622). I think they also apply to the current version of the exam:

  • Evaluate dependencies for infrastructure and application services that will be included in a vSphere design
  • Create Entity Relationship Diagrams that map service relationships and dependencies
  • Analyze interfaces to be used with new and existing business processes
  • Determine service dependencies for logical components
  • Include service dependencies in a vSphere 6.x Logical Design
  • Analyze services to identify upstream and downstream service dependencies
  • Navigate logical components and their interdependencies and make decisions based upon all service relationships

Let’s start with the second topic of this objective.

Evaluate dependencies for infrastructure and application services that will be included in a vSphere design

This topic covers two different parts of our vSphere design:

  • infrastructure, and
  • application services

You should clarify what components of your design depend on each other, or if they depend on components, that are not part of your design. VMware HA needs a shared Storage, or VMware ESXi needs NTP and DNS to work properly.

The same applies to the application services (or applications) that are part of your design. What dependencies do they have. Imagine a three-tier application with database, application logic and web frontend.

You must be able to identify and describe these dependencies.

Create Entity Relationship Diagrams that map service relationships and dependencies

If you are able to identify and describe the dependencies, you also must be able to create a Entity Relationship Diagrams (ER-Diagram) to visualize these dependencies.

Do your homework and try to identify these dependencies at the beginning. Tools like the vRealize Infrastructure Navigator can help you to identify them.

Analyze interfaces to be used with new and existing business processes

It is pretty important to understand how systems interact. To gain this knowledge, you have to analyze the interfaces of business processes. This doesn’t mean that you have to click through ERP applications, but you should get familiar with how processes are tight together.

Determine service dependencies for logical components

You also have to identify the service dependencies for the logical components in your design. You can use tools like vRealize Operations Manager or the Infrastructure Navigator to get the necessary information.

Include service dependencies in a vSphere 6.x Logical Design

The identified service dependencies have to be included into the logical design. This is a pretty important step and you should pay it the necessary attention. Tables and ER diagrams will help you at this step.

Analyze services to identify upstream and downstream service dependencies

An upstream service is a service, which is mandatory for another service, because it relies on it. Downstream services need upstream services to work properly. For example: DNS is an upstream service for Active Directory.

The understanding of up- and downstream services is important for things like startup/ shutdown plans.

Navigate logical components and their interdependencies and make decisions based upon all service relationships

You should visualize the service dependencies. This will help you to evaluate the impact if a service fails or how service are interact with each other.

Summary

Most of the topics in this objective overlap. Quite basic everything is about the understanding how things are connected and interact. This will help you to get a better understanding of dependencies and what services are crucial for the business or your solution.

Think again on DNS. No one of us will ever build a solution with a single DNS server, because nearly everything will melt down if DNS is not available. DNS is a perfect example for an upstream service.

vCenter Migration from 6.0 to 6.7 fails due to missing user role

This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Actually, yesterday should be the day at which I migrate one of the last physical Windows vCenter servers installed in my customer base. Actually… the migration failed twice. And each time I had to rollback, power-on the old physical server, reset the computer account etc.

The update was from VMware vCenter Server 6.0 Update 3d (7462484) on a Windows 2012 R2 server to vCenter Server 6.7 Update 3 (Appliance). The migration failed at 62% with the following message:

Traceback (most recent call last):
  File "/usr/lib/vmware-content-library/firstboot/content-library-firstboot.py", line 219, in Main
    vdc_fb.register_cis()
  File "/usr/lib/vmware-content-library/firstboot/content-library-firstboot.py", line 77, in register_cis
    self._reg_info.registerAll(self.get_soluser_id(), self.get_soluser_ownerId())
  File "/usr/lib/vmware-content-library/install_lib/cis_register.py", line 368, in registerAll
    self.registerUserAndService(user_name, user_id, service)
  File "/usr/lib/vmware-content-library/install_lib/cis_register.py", line 395, in registerUserAndService
    add_vmtx_privileges(self.vdc_cfg_dir)
  File "/usr/lib/vmware-content-library/install_lib/add_vmtx_privileges_after_fb.py", line 105, in add_vmtx_privileges
    log("Adding privileges [%s] to role %s" % (' '.join(VMTX_SYNC_PRIVILEGES), cls_admin_role.name))
AttributeError: 'NoneType' object has no attribute 'name'

I found the same error in the content-library-firstboot.py_9150_stderr.log file of the downloaded log bundle.

Okay, that’s a pretty long error message and I had no idea where I should start searching. But it seems related to the Content Library of the vCenter. And it looks like it is related to the privileges.

log("Adding privileges [%s] to role %s" % (' '.join(VMTX_SYNC_PRIVILEGES), cls_admin_role.name))

A forum post led me to the content library administrator role. The author had to deal with a failed migration (6.5 to 6.7), but his conten administrator role was missing. In my case, the role was existent.

Sorry for the german translation. As you can see, the role was existent… Obviously. I tried to add a new role with the name com.vmware.Content.Admin, as mentioned in the forum post, and… a new role appeared.

You might notice the “Beispiel” or “Example”. That’s the difference. Whatever the other role is or what its look like, it is definitely not the original content library administrator role.

And to make a long story short: The migration was successful after this small change.