Category Archives: Server

HP Service Pack for ProLiant 2014.06

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I’m a bit late, but HP released a new version of their HP Service Pack for ProLiant in June 2014. This version of the SPP supersedes the version 2014.02.0(B). This release adds support for HPs new 20 GbE adapter

and contains new firmware (v4.20b) for HP BladeSystem c-Class Virtual Connect, 4/8Gb 20-port and 8Gb 24-port FC components. HP also added the following firmware and software components to this release:

  • HP ProLiant Converged Network Utility for Windows Server 2008
  • HP ProLiant Converged Network Utility for Windows Server x64 Editions
  • Online ROM Flash Component for Windows – HP ProLiant XL220a Gen8 v2 (P94) Servers
  • HP ProLiant Converged Network Utility for Linux x86_64
  • HP ProLiant Converged Network Utility for Linux x86
  • Online ROM Flash Component for Linux – HP ProLiant XL220a Gen8 v2 (P94) Servers
  • Online ROM Flash Component for VMware ESXi – HP ProLiant XL220a Gen8 v2 (P94) Servers
  • HP Firmware Flash for Emulex Fibre Channel Host Bus and Converged Network Adapters – VMware 5.0/5.1
  • HP Firmware Flash for Emulex Fibre Channel Host Bus and Converged Network Adapters for VMware vSphere 5.5
  • HP Firmware Flash for QLogic Fibre Channel Host Bus Adapters – Vmware
  • HP Firmware Flash for QLogic Fibre Channel Host Bus Adapters for VMware vSphere 5.5

This release of the SPP is the last release, that will support ProLiant G5 (and earlier) models, as also the last release that will contain support for Red Hat Enterprise Linux 5 (RHEL5). Needless to say that HP fixed CVE-2014-0224 with this release. The SPP 2014.06 includes HP SUM 6.4.1.

HP marked the following updates as critical updates. An update is strongly recommended:

  • HP BladeSystem c-Class Virtual Connect Firmware, Ethernet plus 4/8Gb 20-port and 8Gb 24-port FC Edition Component for Windows
  • HP ProLiant Dynamic Smart Array RAID Controller Driver for Windows Server 2008
  • HP ProLiant Dynamic Smart Array RAID Controller Driver for Windows 2008 x64 Editions
  • HP ProLiant Dynamic Smart Array RAID Controller Driver for Microsoft Windows Server 2012 and Microsoft Windows 2012 R2 x64 Editions

Please take a look into the release notes. You can download the ISO image here. A HP Passport login is required.

Replace HP iLO security certificates

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

When you access the HP iLO webinterface, you will be redirected to a HTTPS website. This connection is usually secured by a self-signed SSL certificate. To replace this certificate with a certificate that was issued by your own CA, you have to complete several steps. I will guide you to the steps. I focused on HP ilO 2, but the steps are similar for iLO 3 or iLO 4.

The requirements

We need:

  • an iLO interface that is connected to the network and that has an ip address assigned
  • access to this iLO interface
  • a CA and access to it
  • a web browser

Create the Certificate Signing Request (CSR)

Before we can issue the certificate, we need to create a certificate signing request. This request is used by the CA to create the digital certificate. The CSR contains information to identifying the applicant. This is e.g. the distinguished name (DN), which is the FQDN for a webserver. To create a CSR we have to login into the iLO webinterface.

Create the CSR, issue and install the certificate

I use a Microsoft Windows Server 2012 R2 CA in my lab. This CA is integrated into my Active Directory and I use it to issue certificates for my lab infrastructure. Because it’s my lab, I don’t use a two-tier CA with an offline root CA. ;) But if you are interested in how to setup this, I recommend this two excellent articles written by Derek Seaman and posted on his blog: Windows Server 2012 R2 Two-Tier PKI CA Pt. 1 & Windows Server 2012 R2 Two-Tier PKI CA Pt. 2.

To create a CSR we have to login into iLO and access the “Administration” tab. Then select “Security” from the left menu.

ilo2_ssl_cert_1

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Usually the lower fields are greyed out, so you have to enable “Customized CSR”. Then you can fill the lower, now enabled fields, with values. Don’t forget to hit apply.

A little further down the page, you can create a certificate request.

ilo2_ssl_cert_2

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Click the “Create Certificate Request” button. The certificate request will be generated and you will forwarded to the next page. Now you have to copy the request into a text file or you can past it directly into you CA. I use a W2K12 R2 CA which is running on another host. So I copied the text into a file and saved the file as ilo-esx1.csr.

ilo2_ssl_cert_3

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

No it’s time to issue the certificate. I copied the CSR to my CA into a temp directory. Open an elevated CMD, switch to the directory with the CSR and run the following command:

certreq.exe - submit - attrib "CertificateTemplate:WebServer" ilo-esx1.csr ilo-esx1.pem

A windows will pop up where you have to chose the CA. Because I only have on CA, I can’t choose much… Select you CA and click “OK”. Copy the pem file to you client (or whereever you have the browser with the iLO open), click “Next Step” and then paste the content of the pem file into the text field.

ilo2_ssl_cert_4

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Click “Install Certificate”.

ilo2_ssl_cert_5

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

If you click “Restart” a counter will appear. After 60 seconds you will be redirected to the login page. Please note, that you have to access the login page via the FQDN. Otherwise you will get a certificate error.

Summary

Essentially there is nothing special. It’s much more easier as to do this for a VMware environment… It’s a simple three-step plan: 1. Create the CSR, 2. issue a certificate by using the CSR and 3. install the certificate. Don’t forget to import the CA certificate into you browser. Otherwise you will furthermore get this nasty security warning…

Trouble with Broadcom NetXtreme II and VMware ESXi

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I faced today a really nasty problem. I have four HP ProLiant DL360 G6 in my lab. This server type has two 1 GbE NICs with the Broadcom NetXtreme II BCM5709 chip onboard, which are usually claimed by the bnx2 driver. While applying a host profile to three of the hosts, one hosts reported an error. Supposedly the host hasn’t a vmnic0 and because of this the host profile couldn’t be applied. Okay, quick check in the vSphere Web Client: Only three NICs. C# client showed the same result. Now it was interesting:

<pre>/var/log # esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic1  0000:02:00.01 bnx2        Up   1000Mbps  Full   00:26:55:7c:da:82 1500   Broadcom Corporation NC382i Integrated Multi Port PCI Express Gigabit Server Adapter
vmnic2  0000:04:00.00 bnx2        Up   1000Mbps  Full   68:b5:99:bc:6a:8c 1500   Broadcom Corporation NC382T PCI Express Dual Port Multifunction Gigabit Server Adapter
vmnic3  0000:04:00.01 bnx2        Up   1000Mbps  Full   68:b5:99:bc:6a:8e 1500   Broadcom Corporation NC382T PCI Express Dual Port Multifunction Gigabit Server Adapter
/var/log # lspci | grep vmnic
0000:02:00.0 Network controller: Broadcom Corporation NC382i Integrated Multi Port PCI Express Gigabit Server Adapter [vmnic0]
0000:02:00.1 Network controller: Broadcom Corporation NC382i Integrated Multi Port PCI Express Gigabit Server Adapter [vmnic1]
0000:04:00.0 Network controller: Broadcom Corporation NC382T PCI Express Dual Port Multifunction Gigabit Server Adapter [vmnic2]
0000:04:00.1 Network controller: Broadcom Corporation NC382T PCI Express Dual Port Multifunction Gigabit Server Adapter [vmnic3]

Okay… lspci shows four NICs, esxcfg-nics only three.

/var/log # ethtool -i vmnic0
driver: bnx2
version: 2.2.4f.v55.3
firmware-version: bc 5.2.3 NCSI 2.0.12
bus-info: 0000:02:00.0

Okay, vmnic0 is claimed by a driver. Quick check with another DL360 G6. Same firmware and driver. Lets dig deeper.

/var/log # grep vmnic0 *
shell.log:2014-04-22T12:17:25Z shell[35761]: [root]: grep vmnic0 *
vmkdevmgr.log:2014-04-22T12:14:28Z vmkdevmgr: AddAlias: Not commiting alias vmnic0 for busAddress p0000:02:00.0
vmkdevmgr.log:2014-04-22T12:14:28Z vmkdevmgr: AddAlias: skipping matching alias vmnic0 for pci device p0000:02:00.0 with assigned alias vmnic0
vmkernel.log:2014-04-22T12:14:28.981Z cpu0:33378)PCI: 1095: 0000:02:00.0 named 'vmnic0' (was '')
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)&lt;6&gt;bnx2 0000:02:00.0: vmnic0: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem f2000000, IRQ 17, node addr 00:26:55:7c:da:80
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)&lt;6&gt;bnx2 0000:02:00.0: vmnic0: NetQueue Ops registered [0]
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)VMK_PCI: 395: Device 0000:02:00.0 name: vmnic0
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)Uplink: 6511: Device vmnic0 not yet opened
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)DMA: 612: DMA Engine 'vmnic0' created using mapper 'DMANull'.
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 8230: Opening device vmnic0
vmkernel.log:2014-04-22T12:14:33.431Z cpu3:32836)IRQ: 540: 0x39 &lt;vmnic0-0&gt; exclusive, flags 0x10
vmkernel.log:2014-04-22T12:14:33.431Z cpu3:32836)Uplink: 8111: Network device open handler failed for 'vmnic0': Failure
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 8260: Device vmnic0 failed to open
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 6807: Device vmnic0 not yet opened
vmkernel.log:2014-04-22T12:14:33.773Z cpu12:33407)&lt;6&gt;bnx2x: Added CNIC device: vmnic0
vmkernel.log:2014-04-22T12:14:33.773Z cpu12:33407)&lt;3&gt;bnx2x: vmnic0 - Num 1G iSCSI licenses = 65535

Ah, okay. That looks interesting:

vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)VMK_PCI: 395: Device 0000:02:00.0 name: vmnic0
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)Uplink: 6511: Device vmnic0 not yet opened
vmkernel.log:2014-04-22T12:14:33.429Z cpu12:33406)DMA: 612: DMA Engine 'vmnic0' created using mapper 'DMANull'.
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 8230: Opening device vmnic0
vmkernel.log:2014-04-22T12:14:33.431Z cpu3:32836)IRQ: 540: 0x39 &lt;vmnic0-0&gt; exclusive, flags 0x10
vmkernel.log:2014-04-22T12:14:33.431Z cpu3:32836)Uplink: 8111: Network device open handler failed for 'vmnic0': Failure
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 8260: Device vmnic0 failed to open
vmkernel.log:2014-04-22T12:14:33.431Z cpu12:33406)Uplink: 6807: Device vmnic0 not yet opened

At this point I asked Google and found a discussion in the VMTN, at which @VirtuallyMikeB had participated. Unfortunately the posted solution (power off the server and pull the power cables) didn’t helped (would have surprised me…). This solution was found in this blog article. Although this was not the solution, but it prompted me to start another attempt: A firmware update, because this may reset the NIC as well. I started the server from a USB stick with the current SPP 2014.02. The automatic firmware update updated the BIOS, the ILO board, NICs, the Smart Array controller, the whole damn server, every part of it. Okay, the server was a “bit” outdated… To make a long story short: The firmware update did the trick.

EDIT: And it seems that I’m not the only one…

A word of warning: Julian Wood wrote a blog article about a firmware update that kills Broadcom NICs in HP ProLiant G2 up to G7 servers. He also links to a customer advisory from HP. Following NICs are affected:

  • HP NC373T PCIe Multifunction Gig Server Adapter
  • HP NC373F PCIe Multifunction Gig Server Adapter
  • HP NC373i Multifunction Gigabit Server Adapter
  • HP NC374m PCIe Multifunction Adapter
  • HP NC373m Multifunction Gigabit Server Adapter
  • HP NC324i PCIe Dual Port Gigabit Server Adapter
  • HP NC326i PCIe Dual Port Gigabit Server Adapter
  • HP NC326m PCI Express Dual Port Gigabit Server Adapter
  • HP NC325m PCIe Quad Port Gigabit Server Adapter
  • HP NC320i PCIe Gigabit Server Adapter
  • HP NC320m PCI Express Gigabit Server Adapter
  • HP NC382i DP Multifunction Gigabit Server Adapter
  • HP NC382T PCIe DP Multifunction Gigabit Server Adapter
  • HP NC382m DP 1GbE Multifunction BL-c Adapter
  • HP NC105i PCIe Gigabit Server Adapter

Don’t update the affected NICs with the HP Smart Update Manager (HP SUM) or the HP Service Pack for ProLiant (HP SPP) 2014.2.0. If you update one of the affected NICs with the firmware smart component be sure to avoid updating the Comprehensive Configuration Management (CCM) firmware to version 7.8.21.

EDIT: Hewlett-Packard published HP Service Pack for ProLiant (SPP) Version 2014.02.0(B), which addresses several issues, not only the Issue with Broadcom NICs. This is taken from the HP website:

This updated version of the SPP was released to address the OpenSSL issue.  See HPN Customer Notice: OpenSSL HeartBleed Vulnerability.  Additionally for Red Hat Enterprise Linux 6 customers, please reference the Red Hat knowledge base article, OpenSSL CVE-2014-0160.  Products affected:

  • HP Onboard Administrator for Windows and Linux version 4.12 replaced 4.11
  • HP System Management Homepage for Windows and Linux version 7.3.2 replaced 7.3.1.4
  • HP Integrated Lights-Out 2 for Windows and Linux version 2.25 replaced 2.23
  • HP BladeSystem c-Class Virtual Connect Firmware, Ethernet plus 4/8Gb 20-port and 8Gb 24-port FC Edition Component for Windows and Linux version 4.10(b) replaced 4.10
  • HP Smart Update Manager version 6.3.1 replaced 6.2.0

This release also resolves the Broadcom Comprehensive Configuration Management Firmware issue with version 7.8.21 found in the Service Pack for ProLiant 2014.02.0.  See Customer Advisory c04258304 for additional information.

Thanks to Rotem Agmon, who has posted a comment with this information.

HP Service Pack for ProLiant 2014.02

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

After nearly 5 months released HP a new version of the HP Service Pack for ProLiant (SPP). The latest release is now 2014.02.

What is the HP Service Pack for ProLiant?

Back in the days there were two software products to update a ProLiant server with the latest firmware, drivers & agents.

  • HP Smart Update Firmware DVD
  • HP ProLiant Support Pack

the first one was bootable for offline firmware, and also contained Online ROM flash components for online firmware updates. The second was to install/ update the latest drivers and agents. The HP Service Pack for ProLiants (SPP) replaces both. The SPP is a comprehensive software and it’s delivered as a ISO. It can be burned to DVD, installed on a USB stick or extracted and run from the directory. The SPP used the HP Smart Update Manager (SUM) as deployment tool. The HP SUM can be used as a standalone product.

What’s new in SPP 2014.02?

  • Added new support for the HP ProLiant DL580 Gen8 Server
  • Enhanced support (processor update) for the following HP ProLiant servers:
    • HP DL380e Gen8 Server
    • HP DL360e Gen8 Server
    • HP BL420 Gen8 Server Blade
    • HP SL4540/SL4545 Gen8 Server
    • HP ML350e Gen8 V2 Server
  • Added operating system support for:
    • Microsoft Windows Server 2012 R2
    • Red Hat Enterprise Linux 5.10
    • Red Hat Enterprise Linux 6.5
    • VMware ESXi 4.1 U2
    • VMware ESXi 5.0 U3
    • VMware vSphere 5.1 U2
    • VMware vSphere 5.5
  • Contains HP Smart Update Manager v6.2.0
  • Added Simplified Chinese language support

For more information take a look into the Release Notes. The supported server models can be found in the HP Service Pack for ProLiant Server Support Guide.

Changes regarding HP ProLiant Server firmware access

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
Note: I work for a HP partner and a HP fanboy for about 15 years.

It’s only a small note in the HP support portal, but this small note has a large impact.

Starting February 2014, an active warranty or contract is required to access HP ProLiant Server firmware updates. View your existing contracts & warranties or get help linking contracts or warranties to your HP Support Center user profile. To obtain additional support coverage, please contact your local HP office, HP representative, or visit Contact HP. Click here for more information.

What does this mean? If you want a firmware update, you need a system that is under warrenty, has a active CarePack or belongs to a service agreement. Otherwise you will not be able to download new firmware releases. Why is this a problem? Who’s affected by this? In my opinion there are three affected groups:

  1. Customers
  2. Reseller of refurbished hardware
  3. Provider of 3rd party support contracts

Customers

Customers paid a lot for a high-quality server product and they want to use it for a some time. Some customers sort out servers after three or four years. They don’t buy a support extension. They just buy new servers. Other customers use a server as long as they can, regardless if it’s under support or not. But what happens to a server that is only three or four years old? Especially if a customer leases server, the hardware is returned after the end of the leasing contract. There are some companies that buy, refurbish and sells those kind of hardware. This leads us to group 2.

Reseller of refurbished hardware

These resellers can beat every price, because used hardware is much cheaper. Their offers are mainly addressed due to price-sensitive customers. This hits HP and their partners. Both want to sell new instead of used hardware. Often the reseller can offer hardware support. So if the hardware fails, the reseller can replace defective parts. There is no need to offer more support, because driver and firmware can be downloaded from the HP website.

Provider of 3rd party support contracts

Some customers want to use the hardware for more then three or four years. That’s not problem, because HP offers support contracts with a terms up to five years, or support extensions to extend the warranty. Some customers is this to expensive. They buy hardware support from 3rd party providers. Those providers can act much cheaper than HP. They only have to cover defective hardware. Firmware and drivers can still be downloaded from the HP website.

Conclusion

HP has no interest to annoy customers. But they want resellers and 3rd party support meet heavy. If a customer wants to buy used hardware, he can do that. But he has to purchase a support contract from HP. Or he has to live with the risk not to get new firmware. If the customer buy a HP ProLiant G6 or G7 this isn’t a problem. But what about c-Class blades and Virtual Connect modules? HP wants that customers buy new hardware or that they buy support for used hardware. This allows HP to reduce value of used hardware and still make money with used hardware.

My opinion

Other vendors will follow. IBM/ Lenovo is doing the same for some time. DELL offers free access to firmware, but for how long? Take a look into the software business? There’s a lot of software for which you have to buy support in order to get updates. Or think about Cisco and IOS. No support > no IOS updates. It’s a trend and it will spread. To be honest: I ignored a fourth affected group: The lab users. There are many people that run server hardware in their lab. Maybe it would be a good idea to open firmware access for this group of users. In keeping with: Do good and talk about it.

I’d like to link to Lindsay Hills blog. He wrote a great article about the restricted access to the server firmware.