Tag Archives: power cli

PowerShell ISE on steroids

I’m not a developer. I deal mainly with infrastructe, things like virtualization, storage & backup, networking etc. Sometimes I had to write scripts, primarily PowerShell, batch or Bash. Many years back, I also wrote Csh and Ksh scripts. In the past years, automation was one of the rising trends in the infrastructure segment. And with automation, new challenges came up. Today I have to work with Windows PowerShell, in case of VMware with PowerCLI (which bases on Windows PowerShell), and sometimes I have use with REST APIs. I’m still not a developer. Due to this fact, I need tools that help me getting my work done.

So I was searching for a tool, mainly for PowerShell development, and I’ve tried some tools. Microsoft Visual Studio was to complex. Microsoft Visual Studio Code was light, but offered not the features I needed. The Windows integrated PowerShell ISE was nice, but it also lacked some features. So I asked on Twitter:

The answer was simple: ISESteroids.

What is ISESteroids?

ISESteroids is not a standalone product. It’s a PowerShell module that extents the built-in PowerShell ISE. That’s nice, because you don’t have to install anything. Simply extract it. You don’t need any special privileges to install it. Load the PowerShell module, done.

ISESteroids offers a broad feature set and transforms the PowerShell ISE into a full-featured PowerShell IDE. Visit the ISESteroids homepage for a full feature list. Nothing I want to copy & paste here.

Why is ISESteroids helpful for me?

As already mentioned: I’m not a developer. Therefore, I’m thankful for all hints and tips to make my scripts better. One of the features that I noticed immediately was the light bulb on the left side of the scripting area. The icon indicates that there is an automatic fix. In my case, this is usually converting double into single quotes.

ISESteroids_01

Another often mentioned fix is the replacement of aliases with the full command names. Another feature I really like is the risk analyzer. Sometimes you use commands and functions, that might not work with future releases, or which involve other risks. The risk analyzer is an easy way to highlight these risky commands and functions.

ISESteroids_02

Green indicates: Everything’s fine. If something risky is found, you will get a explanation why this was marked as a risky element. If you still want to use it, you can add the marked element to a whitelist. Some risks, are not a risk at all. The risk analyzer will mark the usage of the cmdlet Move-VM as a risk. This is because cmdlets with the verb “Move” will move things. IN case of Move-VM, this is intended. That’s something you can certainly whitelists.

ISESteroids_03

One of my most used cmdlets is Get-Help. Intellisense is nice, but sometimes I have to look up the correct syntax or similar. ISESteroids offers a context sensitive help. Click on the icon with the question mark,

ISESteroids_04

and you will see a new add-on tab on the left. Very handy. Click on a command, and the help will appear help add-on tab.

ISESteroids_05

You might notice another add-on tab in the picture above: Variables. This tab belongs to the Variables monitor, which can be useful to watch the content of variables. I use it frequently in conjunction with the debugging function.

ISESteroids_07

You can set breakpoints, add variables to the monitor and then watch the content of the variable.

ISESteroids_08

But you can also take a look at the current content of variables, in this case $VMhostScsiLunPaths.

ISESteroids_09

The last feature I’d like to show, is the AutoRefactor. Usually, I tend to follow best practices (mostly my own…) to make my scripts more “readable”. The AutoRefactor feature of ISESteroids helps me to make my scripts cleaner and more readable. It’s customizable, so I can tweak it where necessary. You can enable the refactor add-on tab by clicking the small icon with the check mark.

ISESteroids_06

Write down the code and click “Fix Script Now”. Then watch the magic. ;)

Why didn’t I highlight the other cool features, like code signing, file version control, keyboard shortcuts or test arguments? Because I’m still not a developer. The features I mentioned in this blog post are worthy enough to buy a PowerShell ISE license. Check the full feature list, download and install the trial version. I really recommend to take a look at the trial version! I was sceptical until I worked with ISESteroids. It was a great recommendation!

Licensing

ISESteroids is available in two commercial licenses:

  • Professional
  • Enterprise

The Professional license is available for 99 €, the Enterprise license costs 249 €. Latter offers more features. For individuals (natural persons), a discounted Enterprise license (99 €) is available. Startups, MVPs, trainers etc. can request a discounted license. Check the order website for more details.

PowerCLI: Get-LunPathState

Careful preparation is a key element to success. If you restart a storage controller, or even the whole storage, you should be very sure that all ESXi hosts have enough paths to every datstore. Sure, you can use the VMware vSphere C# client or the Web Client to check every host and every datastore. But if you have a large cluster with a dozen datastores and some Raw Device Mappings (RDMs), this can take a looooong time. Checking the path state of each LUN is a task, which can be perfectly automated. Get a list of all hosts, loop through every host and every LUN, output a list of all hosts with all LUNs and all paths for each LUN. Sounds easy, right?

For a long time, I used this PowerCLI script for checking the LUN path state. But now I decided to give something back and I tweaked it a bit for my needs.

Feel free to use and/ or modify it.

Automating ESXi configuration for DataCore SANsymphony-V

DataCore describes in their Host Configuration Guide for VMware ESXi some settings that must be adjusted before storage from DataCore SANsymphony-V storage servers will be assigned to the ESXi hosts. Today, for ESXi 5.x and 6.0, you have to add a custom rule and adjust the advanced setting DiskMaxIOSize. For ESX(i) 4 more parameters had to be adjusted. But I will focus on  ESXi 5.x and 6.0. You need to adjust these settings for each host that should get storage mapped from a DataCore storage server. If you have more then one host, you may have the wish to automate the necessary steps. The check the current value of DiskMaxIOSize, you can use this lines of PowerCLI code.

So set DiskMaxIOSize to the recommended value of 512 and to create the custom SATP rule, use this lines:

Please make sure that you adjust the $esxCluster variable.  Nothing fancy, but it can save some time – even if you only have two hosts. ;)

 

How to shrink thin-provisioned disks

Get the latest version: Reclaim-ThinVMDK.ps1

Disk space is rare. I only have about 1 TB of SSD storage in my lab and I don’t like to waste too much of it. My hosts use NFS to connect to my Synology NAS, and even if I use the VAAI-NAS plugin, I use thin-provisioned disks only. Thin-provisioned disks tend to grow over time. If you copy a 1 GB file into a VM and you delete this file immediately, you will find that the VMDK is increased by 1 GB. This is caused by the guest filesystem. It marks the blocks of deleted files as free, even if it only deletes metadata and not the data itself. Later, the data is overwritten with new data, since the blocks are marked as free and the new data is written in there. VMware ESXi doesn’t know that the guest has marked blocks as free. So ESXi can’t shrink the thin-provisioned VMDK.

You can observe a similar behavior in case of VMFS and underlying thin-provisioned LUNs: If a VMDK is removed from a VMFS datastore, the underlying  thin-provisioned LUN doesn’t show more free space. In this case, the VAAI UNMAP primitive can be used to tell the storage system which blocks are free and can be reclaimed. Some storage system that doesn’t support VAAI UNMAP use contiguous regions filled with zeros to identify reclaimable storage space. Before free space can be reclaimed, the VMFS has to be filled with zeros. A similar technique can be used to shrink thin-provisioned guest hard disks. Please note that I don’t want to focus on reclaiming space from underlaying LUNs. I’m only talking about shrinking thin-provisioned disks!

To shrink a thin-provisioned VMDK the guest filesystem has to be zeroed out. If you use Windows, you can use SDelete. In case of a unixoide OS (Linux, FreeBSD, Solaris…), use dd. After you have zeroed out the guest file system, you have to move the VM with Storage vMotion to another datastore. Now it’s getting complicated: You have to make sure that the legacy datamover (fsdm) is used for the Storage vMotion. There are three different datamovers:

  • fsdm
  • fs3dm, and
  • fs3dm – hardware offload

The fsdm is the oldest and slowest datamover. The fs3dm and fs3dm with HW offload are newer. In case of the latter, the process is offloaded to the hardware using VAAI (Full Copy primitive). At this point, I’d like to refer to a blog post of Duncan Epping (Blocksize impact?) , who has highlighted the differences between the datamovers more detailed. The point is, that the fsdm doesn’t copy blocks that are filled with zeros. But how can I make sure, that the fsdm is used?

  • Move the VM to a datastore with another blocksize

This can be difficult, because VMFS5 datastores have a block size of 1 MB, except they were upgraded from VMFS3. Simply create a new VMFS3 datastore and use it as destination.

  • Move the VM from VMFS to NFS, from NFS to VMFS or from NFS to NFS

In this case fsdm will be used. Please note that fsdm will not be used if you move a VM from a VMFS5 to a VMFS5 datastore! In this case the fs3dm is used. This wouldn’t shrink the thin-provisioned VMDK. On the downside the fsdm is slow. Really slow. If you have a monster VM, a vMotion can take a looooong time (worth reading: “VMware Storage vMotion, Data Movers, Thin Provisioning, Barriers to Monster VM’s” by Michael Webster).

I wrote a PowerShell script that uses PowerShell remoting and VMwares PowerCLI cmdlets to do the following tasks:

  • get a list of all local disks using Get-WmiObject
  • zero-out filesystem on those disks
  • move the VM to a destination datastore
  • move the VM back to its source host and source datastore

For the moment, the script only works with Windows VMs. SDelete must be available in the VM. Make sure that you use the latest release of SDelete (currently 1.61). PowerShell remoting has to be enabled on the VMs. Feel free to use and/ or edit my script. To get this script working, please change the content of the variables for

  • $PathToSDelete
  • $VIServer
  • $CredFile
  • $Username
  • $DstDS
  • $DstDSHost and
  • $ClusterName

according to your environment. The script skips VMs with active snapshots and VMs that have one or more ZeroedThick or EagerZeroedThick disks attached. Because the script use all local disks, it will also zero-out disks that were attached using in-guest iSCSI. So please be test the script in your lab until you try it in production.

This is an example for the output of the script:

VMDK-thin-reclaim_script_output

 

In this picture you can see, that the script processes one disk after another:

VMDK-thin-reclaim_performance_zero_out

This script is provided “AS IS” with no warranty expressed or implied. Run at your own risk. Please test the script in your lab.

Add a new version of HP Agentless Management Service to a customized ESXi 5.5.0 ISO

While preparing for a VMware vSphere 5.5 update at a customer of mine, I stumbled over VMware KB2085618 (ESXi host cannot initiate vMotion or enable services and reports the error: Heap globalCartel-1 already at its maximum size.Cannot expand.). I checked the HP AMS version in the latest HP custom ESXi image and found out, that version hp-ams-esx-550.10.0.0-18.1198610 is included (source). Unfortunately the bug is not fixed in 10.0.0, but it’s fixed in 10.0.1 (source).

hp_ams_10_0_1

According to the VMware KB article only the HP AMS versions hp-ams 500.9.6.0-12.434156 and hp-ams-550.9.6.0-12.1198610 should be affected. But since I do not like surprises, I decided to update the HP AMS version in the latest HP custom ESXi image from 10.0.0 to 10.0.1.

Prerequisites

Before you can start building a new customized ESXi image, you have to fulfill some prerequisites.

  1. Latest version of the HP customized ESXi Image. Make sure that you download the ZIP and not the ISO file! Download
  2. Latest version of the HP Agentless Management Service Offline Bundle for VMware vSphere 5.5. Download
  3. VMware Power CLI installed on your computer. Download

Updating HP AMS

Copy both downloaded files into a temporary folder. Then import both depot files. You can proof the success with Get-EsxImageProfile which should show you the just imported ESXi image file version.

The next step is to clone the image profile. This cloned image profile will be the target for our software package update. You can check the success again with Get-EsxImageProfile. At this point you should get two image profiles listed.

Now you can update the HP AMS package. The update is done using the Add-EsxSoftwarePackage commandlet.

When you compare the original and the clones profile, you should get this result. Note the UpgradeFromRef entry.

The last step is to export the clones and updates image profile to a ZIP or, this is our case, to a ISO file. This ISO file can be used to upgrade hosts using VMware Update Manager.

That’s it. Now you can update you hosts with this ISO and you have automatically updates the HP Agentless Management Services.

EDIT

Ivo Beerens wrote a nice script to check the installed version of the HP AMS version. Checkout his blog post about this topic.

EDIT 2

I discovered today that HP has published a new version of their customized HP ESXi release (vSphere 5.5 U2 Nov 2014). This release includes the latest version of the HP Agentless Management Service Offline Bundle for VMware vSphere 5.5.

Install VMware Tools from VMware repository

Today I stumbled over a nice workaround. While installing a CentOS 6 VM, I needed to install the VMware Tools. I don’t know why, but I got an error message, regarding a non accessible VMware Tools ISO.

vmware_tools_iso_not_accessable

I remembered a blog post I read a few months ago, about a VMware online repository, from which VMware tools can be installed. You can download the repository information here. The RPM for RHEL can also be used for CentOS. Simply download and install the RPM:

Now you can use the repository information to install the VMware Tools.

That’s it.

centos_with_vmwaretools

Load VMware PowerCLI snap-in automatically in PowerShell ISE

The PowerShell Integrated Scripting Environment (ISE) is a very handy application when dealing with the PowerShell. And because of this, the ISE is also a very handy application when dealing with VMware PowerCLI. When I write a script or a one-liner, one of the first things I do is to load the necessary snap-ins. And because I’m lazy, I’m trying to automate everything, what I have to do more than once. So how can I load the necessary snap-ins automatically when starting PowerShell ISE? The Windows PowerShell profile will help you. This is a simple text file, or to be more precise, a PowerShell script. Because of this, you can write everything (cmdlets, scripts, functions etc.) in this script file, and it will be executed when you start the PowerShell or the PowerShell ISE. Please note, that there are two profile files: One for the PowerShell and one for the PowerShell ISE. But where can you find the Windows PowerShell profile files? The path to the PowerShell profile is returned by the built-in variable $profile.

Open a PowerShell windows:

When you try the same in the ISE, you will get this output:

You see the difference? Depending on your PowerShell environment, the PowerShell reads a different profile file on startup. Usually the files doesn’t exist, except you created them. Check if the files exist. If not, this command will create an empty profile file. Depending on if the command is executed in a PowerShell windows or in the PowerShell ISE, a profile file for PowerShell or PowerShell ISE is created.

Close the PowerShell ISE. Now you can open the file with your favorite editor and add the command Add-PsSnapin.

Save the file and open the PowerShell ISE. A Get-PSSnapin should return, that the VMware.VimAutomation.Core module was loaded. You should also notice that you can now use PowerCLI cmdlets. Everytime you open the PowerShell ISE, the VMware.VimAutomation.Core snap-in is automatically loaded.

Change network adapter type with PowerCLI

Today I found this neat PowerCLI One-liner in my Twitter timeline:

A nice side effect of this one-liner is, that the mac-address doesn’t change, as you can see in the screenshots.

change_adapter_type_vmxnet3 change_adapter_type_e1000

If you have ever changed the adapter type of a vNIC you will know, that this leads to a changed mac-address and a new adapter in the OS. Windows will show a “Local Area Connection 2”, Linux will show a eth1 instead of eth0. So you need to lend a hand. If you use Linux and you’ve changed the adapter type using this one-liner, everythings fine. eth0 will stay eth0, but the kernel loads another driver. No need to modify or delete the 70-persistent-net.rule file under /etc/udev/rules.d. But how does Windows handle it? Unfortunately Windows doesn’t handle it. Windows detects a new device, because the hardware ID changed.

change_adapter_type_e1000_hw_id change_adapter_type_vmxnet3_hw_id

The only way to fix this, is to delete the old, non-present adapter. In order to do that, open a CMD and enter the following command:

Then open the Device Manager, choose “View” and click “Show hidden devices”. Then delete all non-present network adapters and the newly detected VMXNET3 adapter.

change_adapter_type_remove_e1000_devmgmt

After that, right click the computer icon (in my case LABLAB-V4143BPG) and select “Scan for hardware changes”. Windows will detect a new VMXNET3 adapter, which you can finally configure with the hopefully documented IP settings.

My most frequently used PowerCLI One-liner

Over the last months I wrote different PowerCLI One-liners who I want to share. Nothing fancy and one or two are ugly. But they worked for me. :)

Changing the multipathing policy for all hosts and datastores in a cluster

Get a list of all VMs in a cluster and the datastore in which the VMs resides

Get a list of all VMs, their mac-address and the connected port groups

vMotion of a VM between hosts without a shared storage (not really a One-liner…)

Enable SSH on all hosts

Check on which hosts SSH is enabled

Get a list of hosts and the numer of VM that are running on these hosts

If you’re searching for more advanced PowerCLI stuff visit the blogs of Alan Renouf and Luc Dekens.