Tag Archives: devops

Creating console screenshots with Get-ScreenshotFromVM.ps1

Today, I had a very interesting discussion. As part of an ongoing troubleshooting process, console screenshots of virtual machines should be created.

The colleagues, who were working on the problem, already found a PowerCLI script that was able to create screenshots using the Managed Object Reference (MoRef). But unfortunately all they got were black screens and/ or login prompts. Latter were the reason why they were unable to run the script unattended. They used the Get-VMScreenshot script, which was written by Martin Pugh.

I had some time to take a look at his script and I created my own script, which is based on his idea and some parts of his code.

This file is also available on GitHub.

One important note: If you want to take console screenshots of VMs, please make sure that display power saving settings are disabled! Windows VMs are showing a black screen after some minutes. Please disable this using the energy options, or better using a GPO. Otherwise you will capture a black screen!

Monitoring hardware status with Python and vSphere API calls

Apparently it’s “how to monitor hardware status” week on vcloudnine.de. Some days ago, I wrote an article about using SNMP for hardware monitoring. You can also use the vSphere Web Client to get the status of the host hardware. A third way is through the vSphere API. I just want to share a short example how to use vSphere API calls and pyVmomi. pyVmomi is the Python SDK for the VMware vSphere API.

Get hardware status with vSphere API calls

I just want to share a small example, that shows the basic principle. The script gathers the temperature sensor data of a ProLiant DL360 G7 running ESXi 6.0 U2 using vSphere API calls.

The output of the script looks like this:

Nothing fancy. You can easily loop through numericSensorInfo to gather data from other sensors. Use the Managed Object Browser (MOB) to navigate through the API. This is handy if you search for specific sensors. If you need accurate data, the vSphere API is the way to go. If you focus on something lightweight, try SNMP.

HPE StoreVirtual REST API

Representational State Transfer (REST) APIs are all the rage. REST was defined by Roy Thomas Fielding in his PhD dissertation “Architectural Styles and the Design of Network-based Software Architectures“. The architectural style of REST describes six constraints:

  • Uniform interface
  • Stateless
  • Cacheable
  • Client – Server communication
  • Layered system
  • Code on demand

RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.) to send data to, or retrieve data from remote systems. To do so, REST APIs use Uniform Resource Identifiers (URIs) to interact with remote systems. Thus, a client can interact with a remote system over a REST API using standard HTTP URIs and HTTP verbs. For the data transfer, common internet media types, like JSON or XML are used. It’s important to understand that REST is not a standard per se. But most implementations make use of standards such as HTTP, URI, JSON or XML.

Because of the uniform interface, you have different choices in view of a client. I will use PowerShell and the Invoke-RestMethod cmdlet in my examples.

HPE StoreVirtual REST API

With the release of LeftHand OS 11.5 (the latest release is 12.6), HPE added a REST API for management and storage provisioning. Due to a re-engineered management stack, the REST API is significantly faster than the same task processed on the CLI or using the  Centralized Management Console (CMC). It’s perfect for automation and scripting. It allows customers to achieve a higher level of automation and operational simplicity. The StoreVirtual REST API is using JavaScript Object Notation (JSON) for data transfer between client and the StoreVirtual management group. With the REST API, you can

  • Read, create, and modify volumes
  • Create and delete snapshots
  • Create, modify, and delete servers
  • Grant and revoke access of servers to volumes

I use two StoreVirtal VSA (LeftHand OS 12.6) in my lab. Everything I show in this blog post is based on LeftHand OS 12.6.

The REST API in LeftHand OS 12.6 uses:

  • HTTPS 1.1
  • media types application/JSON
  • Internet media types application/schema+JSON
  • UTF-8 character encoding

RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.). I case of the StoreVirtual REST API:

  • GET is used to retrieve an object. No body is necessary.
  • PUT is used to update an object. The information to update the object is sent within the body.
  • POST is used to create of an object, or to invoke an action or event. The necessary information are sent within the body.
  • DELETE is used to delete an object.

Entry point for all REST API calls is /lhos, starting from a node, eg.

Subsequent resources are relative to this base URI. Resources are:

Resource pathDescription
/lhos/managementGroupManagement group entity
/lhos/clustersCluster collection
/lhos/cluster/<id>Cluster entity
/lhos/credentialsCredentials collection
/lhos/credentials/<session token>Credentials entity
/lhos/serversServer collection
/lhos/servers/<id>Server entity
/lhos/snapshotsSnapshot collection
/lhos/snapshots/<id>Snapshot entity
/lhos/volumesVolume collection
/lhos/volumes/<id> Volume entity

The object model of the StoreVirtual REST API uses

  • Collections, and
  • Entities

to address resources. An entity is used to address individual resources, whereas a collection is a group of individual resources. Resources can be addressed by using a URI.

Exploring the API

First of all, we need to authenticate us. Without a valid authentication token, no REST API queries can be made. To create a credential entity, we have to use the POST method.

$cred is a hash table which includes the username and the password. This hash table is converted to the JSON format with the ConvertTo-Json cmdlet. The JSON data will be used as body for our query. The result is an authentication token.

This authentication token must be used for all subsequent API queries. This query retrieves a collection of all valid sessions.

The GET method is used, and the authentication token is sent with the header of the request.

To retrieve an individual credential entity, the URI of the entity must be used.

The result of this query is the individual credential entity

It’s important to know, that if a session has not been used for 15 minutes, it is automatically removed. The same applies to constantly active sessions after 24 hours. After 24 hours, the credential entity will be automatically removed.

Let’s try to create a volume. The information about this new volume has to be sent within the body of our request. We use again the ConvertTo-Json cmdlet to convert a hash table with the necessary information to the JSON format.

The size must be specified in bytes. As a result, Invoke-RestMethod will output this:

Using the CMC, we can confirm that the volume was successfully created.

storevirtual_rest_api_vol_1

Since we have a volume, we can create a snapshot. To create a snapshot, we need to invoke an action on the volume entity. We have to use the POST method and the URI of our newly created volume.

In case of a successful query, Invoke-RestMethod will give us this output.

Again, we can use the CMC to confirm the success of our operation.

storevirtual_rest_api_vol_2

To delete the snapshot, the DELETE method and the URI of the snapshot entity must be used.

To confirm the successful deletion of the snapshot, the GET method can be used. The GET method will retrieve a collection of all snapshot entities.

The result will show no members inside of the snapshot collection.

At the end of the day, we remove our credential entity, because it’s not longer used. To delete the credential entity, we use the DELETE method with the URI of our credential entity.

The next query should fail, because the credential entity is no longer valid.

HTTPS workaround

The StoreVirtual API is only accessable over HTTPS. By default, the StoreVirtual nodes use an untrusted HTTPS certifificate. This will cause Invoke-RestMethod to fail.

After a little research, I found a workaround. This workaround uses the System.Security.Cryptography.X509Certificates namespace. You can use this snippet to build a function or add it to a try-catch block.

Final words

The StoreVirtual REST API is really handy. It can be used to perform all important tasks. It’s perfect for automation and it’s faster than the CLI. I’ve used PowerShell in my examples, but I’ve successfully tested it with Python. Make sure to take a look in to the HPE StoreVirtual REST API Reference Guide.

First steps with Python and pyVmomi (vSphere SDK for Python)

In December 2013, VMware made an christmas gift to the community by releasing pyVmomi. pyVmomi is a SDK that allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Nearly 18 months are past since then and pyVmomi has developed over time.

I’ve started to play around with Python, and I’ve written about the reasons in one of my last blog posts (Hey infrastructure guy, you should learn Python!).

How to get pyVmomi?

You can install the official release of pyVmomi using pip (pip installs packages, a recursive acronym).

The latest version is available on GitHub. To get the latest version, use

or

That you can fetch the latest version from GitHub is pretty cool and shows a big benefit: The community can contribute to pyVmomi and it’s more frequently updated. A huge benefit in regard of code quality and features.

What Python releases are support?

The latest information about supported Python releases can be found on the GitHub page of the project.

  • pyVmomi 6.0.0.2016.4 and later support 2.7, 3.3 and 3.4
  • pyVmomi 6.0.0 and later support 2.7, 3.3 and 3.4
  • pyVmomi 5.5.0-2014.1 and 5.5.0-2014.1.1 support Python 2.6, 2.7, 3.3 and 3.4
  • pyVmomi 5.5.0 and below support Python 2.6 and 2.7

Interesting fact: pyVmomi version numbers correlate with vSphere releases. pyVmomi 6.0.0 was released with the GA of VMware vSphere 6. pyVmomi supports the corresponding vSphere release and the previous four vSphere releases.

I’m using Python 3 for my examples. I wouldn’t recommend to start with Python 2 these days.

First steps

pyVmomi allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Because of this, the VMware vSphere API Reference Documentation will be your best friend.

First of all, you need a connection to the API. To connect to the vSphere API, we have to import and use the module pyVim, more precise, the pyVim.connect module and the SmartConnect function. pyVim.connect is used for the connection handling (creation, deletion…) to the Virtualization Management Object Management Infrastructure (VMOMI). pyVim is part of pyVmomi and it’s installed automatically.

SmartConnect accepts various parameters, but for the beginning it’s sufficient to use three of them: host, user and pwd. You can use “help(SmartConnect)” to get information about the SmartConnect function. “c” is the object (pyVmomi.VmomiSupport.vim.ServiceInstance) which we will use later.

A connection itself is useless. But how can we explore the API? Python doesn’t support typing, so it can be difficult to “explore” an API. That’s why the VMware vSphere API Reference Documentation and the Managed Object Browser (MOB) will be your best friends. The MOB is a web-based interface and represents the vSphere API. It allows you to navigate through the API. Any changes you make through the MOB, by invoking methods, take effect and change the config or will give you an output.

Important note: If you are using VMware vSphere 6 (ESXi 6.0 and vCenter 6.0), you have to enable the MOB. The MOB is disabled by default. Check VMware KB2108405 (The Managed Object Browser is disabled by default in vSphere 6.0) for more details.

Open a browser and open https://ip-or-fqdn/mob. You can use the IP address or the FQDN of an ESXi host or a vCenter Server (Appliance). I use a standalone ESXi 5.5 host in this example.

python_esxi_mob_1

Our first code

Let’s try something easy. I’ve framed a method in the screenshot above. We will use this method now.

This code will connect to the vSphere API, invoke the method “CurrentTime()” and prints the result. What happens if we execute our first lines of Python code? We will get an error…

Python checks SSL certificates in strict mode. Because of this, untrusted certificates will cause trouble. This applies to Python 3, as well as to Python >= 2.7.9 (PEP 0466). Most people use untrusted certificates. To deal with this, we have to create a context for our HTTP connection. This context can be used by the SmartConnect function. To create a context, we have to import the ssl module of Python.

“s” is the new object (ssl.SSLContext) we will use and the parameter “sslContext=s” will told SmartConnect to use this object.

Save this code into a file (I called it pyvmomitest.py in my example). Navigate to the folder, open a Python REPL and import the file you’ve saved (module) a moment ago.

Hurray! We used the vSphere API to get the current date and time (CET).

But what if we have deployed valid certificates? And what about housekeeping? We have connected, but we haven’t disconnected from the API? We can use a try-except block to handle this. And because we are nice, we import also the function “Disconnect” from pyVim.connect to disconnect from the vSphere API at the end.

With this code, we should get the following output.

Okay, the vSphere API wasn’t designed to retrieve the current date and time. Let’s look at something more useful. This script will give us the names of all VMs in the datacenter.

Let’s take this statement and look at everything after the “c”. We will use the MOB to navigate through the API. This will help you to understand, how the Python code and the structure of the vSphere API correlate.

Open the MOB. You will easily find  the property “content”.

python_esxi_mob_2

Click on “content” and search for the property “rootFolder”.

python_esxi_mob_3

Click on the value “ha-folder-root.” The property “childEntity” is an ManagedObjectReference (MOR) and references to all datacenters (the counting starts at 0) known to the ESXi or vCenter. The value “childEntity[0]” will give us the first datacenter.

python_esxi_mob_4

If we have the datacenter, the way to get the names of the VMs is the same. You can use the MOB, to verify this.

Click on the value “ha-datacenter”. At the bottom of the list, you will find the property “vmFolder”.

python_esxi_mob_5

Click on the value “ha-folder-vm”.

python_esxi_mob_6

The MOR “childEntity” references to two VMs. Click on one of the IDs.

python_esxi_mob_7

The property “name” includes the name of the VM. Because of this, we can use a simple

to get the name for each VM.

Summary

This was only a short introduction into pyVmomi. You should be now able to install pyVmomi, make a connection to the vSphere API and retrieve some basic stuff.

Every day I discover something new. It’s important to understand how the vSphere API works. Play with pyVmomi and with the vSphere API. It looks harder as it is.

btw: There is a Hands-on-Lab available “HOL-SDC-1622 VMware Development Tools and SDKs“. Check it out!

Hey infrastructure guy, you should learn Python!

I’m not a developer. I’m an infrastructure guy. All I ever needed was to write some scripts. Therefore, I never needed more than DOS batches, BASH/ CSH/ KSH, Visual Basic Script and nowadays PowerShell. So why should I learn another programming language?

One to rule them all?

I don’t think that there is a single programming language that is perfect for all use cases. The spread and acceptance of a language shows a positive correlation with the number of available frameworks, tools and libraries. That’s why I love the Microsoft PowerShell. Nearly all vendors offer a PowerShell module for their products (think about VMware PowerCLI, Rubrik, Veeam, DataCore and much more). The downside: The PowerShell code has to run on a Windows box. I think the time of writing DOS batches is over. UNIX shell scripts are still awesome, but focused on UNIX.

Different problems require different tools. I think it’s better to know a few, general-purpose tools well, as every conceivable special tool. Don’t get me wrong: PowerShell is awesome powerful! It’s quite easy to learn and you will have quick success.

Why Python?

Python is easy to learn (I can confirm this, at least for what I’ve seen). Python was developed from scratch by Guido van Rossum in the early 1990s. Python is an interpreted and dynamic programming language, which supports multiple paradigms, like the object-oriented or the functional programming. Python features a dynamic type system and automatic memory management. It uses only 35 keywords, what makes it easy to lern. It’s underlying philosophy is The Zen of Python.

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.

These rules lead to code with a high legibility, and it is possible to solve problems with fewer lines of code. Python is highly extensible. It comes with a large standard library and you can choose from 72.000 packages, that are available using the official 3rd party repository.

For me, as an infrastructure guy, the VMware vSphere API Python Bindings, the 3PAR Python client or the module for the Alcatel-Lucent Enterprise OmniSwitch RESTful API are reasons enough to start with Python. It’s the extensibility and platform independence of Python, what makes it so interesting. Like PowerShell, Python is an awesome language to automate things.

First steps with Python

Currently, the stabled releases are 2.7 and 3.5. I recommend to start with the 3.5 release. You can get the latest release from python.org. They offer packages for Windows, MacOS X and Linux/ UNIX. Python comes with an IDE called IDLE (Integrated Development and Learning Environment). Make sure that you take a look into the official documentation! If you want something more comfortable, try JetBrain PyCharm. JetBrains offer a free community edition for Windows, MacOS X and Linux. But it’s not the worst idea to start with IDLE. I use both IDEs, IDLE and PyCharm.

Where can you get help? YouTube is full of videos about Python. If you have a Pluralsight subscription, checkout the courses on Pluralsight. There are many good books out there, as well as some good howtos. Just use Google. It depends on what type of learner you are.

Learn the basics and try to strengthen them during a small project. Buy a Raspberry Pi. Raspberry Pi and Python are the biggest friends. If you are focused on VMware vSphere, take a closer look at the VMware vSphere API Python Bindings. Create yourself a project to learn.

I just started to learn Python, but I think that this wasn’t the worst idea in my life.