This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
Today, I had a very interesting discussion. As part of an ongoing troubleshooting process, console screenshots of virtual machines should be created.
The colleagues, who were working on the problem, already found a PowerCLI script that was able to create screenshots using the Managed Object Reference (MoRef). But unfortunately all they got were black screens and/ or login prompts. Latter were the reason why they were unable to run the script unattended. They used the Get-VMScreenshot script, which was written by Martin Pugh.
I had some time to take a look at his script and I created my own script, which is based on his idea and some parts of his code.
One important note: If you want to take console screenshots of VMs, please make sure that display power saving settings are disabled! Windows VMs are showing a black screen after some minutes. Please disable this using the energy options, or better using a GPO. Otherwise you will capture a black screen!
This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
Apparently it’s “how to monitor hardware status” week on vcloudnine.de. Some days ago, I wrote an article about using SNMP for hardware monitoring. You can also use the vSphere Web Client to get the status of the host hardware. A third way is through the vSphere API. I just want to share a short example how to use vSphere API calls and pyVmomi. pyVmomi is the Python SDK for the VMware vSphere API.
Get hardware status with vSphere API calls
I just want to share a small example, that shows the basic principle. The script gathers the temperature sensor data of a ProLiant DL360 G7 running ESXi 6.0 U2 using vSphere API calls.
The output of the script looks like this:
Valid certificate
Hostname: esx1.lab.local
Type: ProLiant DL360 G7
Getting temperature sensor data...
Other 1 Temp 28 --- Normal 65.0 Degrees C
Drive Backplane 1 Temp 27 --- Normal 35.0 Degrees C
Peripheral Bay 8 Temp 26 --- Normal 42.0 Degrees C
Peripheral Bay 7 Temp 25 --- Normal 38.0 Degrees C
Peripheral Bay 6 Temp 24 --- Normal 45.0 Degrees C
Peripheral Bay 5 Temp 23 --- Normal 39.0 Degrees C
Peripheral Bay 4 Temp 22 --- Normal 46.0 Degrees C
Peripheral Bay 3 Temp 21 --- Normal 44.0 Degrees C
Peripheral Bay 2 Temp 20 --- Normal 39.0 Degrees C
Peripheral Bay 1 Temp 19 --- Normal 37.0 Degrees C
Other 5 Temp 18 --- Normal 39.0 Degrees C
Memory Module 10 Temp 17 --- Normal 36.0 Degrees C
Other 4 Temp 16 --- Normal 34.0 Degrees C
Other 3 Temp 15 --- Normal 35.0 Degrees C
Memory Module 9 Temp 14 --- Normal 34.0 Degrees C
Power Supply 5 Temp 13 --- Normal 45.0 Degrees C
Power Supply 4 Temp 12 --- Normal 36.0 Degrees C
Memory Module 8 Temp 11 --- Normal 36.0 Degrees C
Memory Module 7 Temp 10 --- Normal 38.0 Degrees C
Memory Module 6 Temp 9 --- Normal 36.0 Degrees C
Memory Module 5 Temp 8 --- Normal 39.0 Degrees C
Memory Module 4 Temp 7 --- Normal 35.0 Degrees C
Memory Module 3 Temp 6 --- Normal 37.0 Degrees C
Memory Module 2 Temp 5 --- Normal 36.0 Degrees C
Memory Module 1 Temp 4 --- Normal 38.0 Degrees C
Other 2 Temp 3 --- Normal 40.0 Degrees C
Other 1 Temp 2 --- Normal 40.0 Degrees C
Other 1 Temp 1 --- Normal 27.0 Degrees C
>>>
Nothing fancy. You can easily loop through numericSensorInfo to gather data from other sensors. Use the Managed Object Browser (MOB) to navigate through the API. This is handy if you search for specific sensors. If you need accurate data, the vSphere API is the way to go. If you focus on something lightweight, try SNMP.
RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.) to send data to, or retrieve data from remote systems. To do so, REST APIs use Uniform Resource Identifiers (URIs) to interact with remote systems. Thus, a client can interact with a remote system over a REST API using standard HTTP URIs and HTTP verbs. For the data transfer, common internet media types, like JSON or XML are used. It’s important to understand that REST is not a standard per se. But most implementations make use of standards such as HTTP, URI, JSON or XML.
Because of the uniform interface, you have different choices in view of a client. I will use PowerShell and the Invoke-RestMethod cmdlet in my examples.
HPE StoreVirtual REST API
With the release of LeftHand OS 11.5 (the latest release is 12.6), HPE added a REST API for management and storage provisioning. Due to a re-engineered management stack, the REST API is significantly faster than the same task processed on the CLI or using the Centralized Management Console (CMC). It’s perfect for automation and scripting. It allows customers to achieve a higher level of automation and operational simplicity. The StoreVirtual REST API is using JavaScript Object Notation (JSON) for data transfer between client and the StoreVirtual management group. With the REST API, you can
Read, create, and modify volumes
Create and delete snapshots
Create, modify, and delete servers
Grant and revoke access of servers to volumes
I use two StoreVirtal VSA (LeftHand OS 12.6) in my lab. Everything I show in this blog post is based on LeftHand OS 12.6.
The REST API in LeftHand OS 12.6 uses:
HTTPS 1.1
media types application/JSON
Internet media types application/schema+JSON
UTF-8 character encoding
RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.). I case of the StoreVirtual REST API:
GET is used to retrieve an object. No body is necessary.
PUT is used to update an object. The information to update the object is sent within the body.
POST is used to create of an object, or to invoke an action or event. The necessary information are sent within the body.
DELETE is used to delete an object.
Entry point for all REST API calls is /lhos, starting from a node, eg.
https://fqdn-or-ip:8081/lhos/
Subsequent resources are relative to this base URI. Resources are:
Resource path
Description
/lhos/managementGroup
Management group entity
/lhos/clusters
Cluster collection
/lhos/cluster/<id>
Cluster entity
/lhos/credentials
Credentials collection
/lhos/credentials/<session token>
Credentials entity
/lhos/servers
Server collection
/lhos/servers/<id>
Server entity
/lhos/snapshots
Snapshot collection
/lhos/snapshots/<id>
Snapshot entity
/lhos/volumes
Volume collection
/lhos/volumes/<id>
Volume entity
The object model of the StoreVirtual REST API uses
Collections, and
Entities
to address resources. An entity is used to address individual resources, whereas a collection is a group of individual resources. Resources can be addressed by using a URI.
Exploring the API
First of all, we need to authenticate us. Without a valid authentication token, no REST API queries can be made. To create a credential entity, we have to use the POST method.
$cred is a hash table which includes the username and the password. This hash table is converted to the JSON format with the ConvertTo-Json cmdlet. The JSON data will be used as body for our query. The result is an authentication token.
This authentication token must be used for all subsequent API queries. This query retrieves a collection of all valid sessions.
$b = Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials -Method Get -Headers @{'Authorization'=$a.authToken}
The GET method is used, and the authentication token is sent with the header of the request.
PS C:\Users\p.terlisten> $b
name : REST Sessions Collection
description : Collection of authentication sessions used by the REST server.
type : RESTSession
uri : /lhos/credentials
total : 1
members : {@{name=fa0a7b56-0134-400f-9d62-79b3071c950a; description=REST Session; type=RESTSession; id=0; uri=/lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a;
created=2016-06-07T08:38:06.426241Z; modified=2016-06-07T08:44:28.255283Z; userName=admin; clientIP=192.168.200.90}}
To retrieve an individual credential entity, the URI of the entity must be used.
$b = Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a -Method Get -Headers @{'Authorization'=$a.authToken}
The result of this query is the individual credential entity
PS C:\Users\p.terlisten> $b
name : fa0a7b56-0134-400f-9d62-79b3071c950a
description : REST Session
type : RESTSession
id : 0
uri : /lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a
created : 2016-06-07T08:38:06.426241Z
modified : 2016-06-07T08:51:56.358096Z
userName : admin
clientIP : 192.168.200.90
It’s important to know, that if a session has not been used for 15 minutes, it is automatically removed. The same applies to constantly active sessions after 24 hours. After 24 hours, the credential entity will be automatically removed.
Let’s try to create a volume. The information about this new volume has to be sent within the body of our request. We use again the ConvertTo-Json cmdlet to convert a hash table with the necessary information to the JSON format.
$vol = @{
name='api-vol';
description='Volume created via REST API';
size=1073741824;
clusterId=28;
isThinProvisioned=$true;
dataProtectionLevel=2
}
$body = $vol | ConvertTo-Json
Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/volumes -Method Post -Headers @{'Authorization'=$a.authToken} -Body $body -ContentType 'application/JSON'
The size must be specified in bytes. As a result, Invoke-RestMethod will output this:
Using the CMC, we can confirm that the volume was successfully created.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Since we have a volume, we can create a snapshot. To create a snapshot, we need to invoke an action on the volume entity. We have to use the POST method and the URI of our newly created volume.
To confirm the successful deletion of the snapshot, the GET method can be used. The GET method will retrieve a collection of all snapshot entities.
Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/snapshots -Method Get -Headers @{'Authorization'=$a.authToken}
The result will show no members inside of the snapshot collection.
name : Snapshots Collection
description : Collection of Snapshot objects
type : snapshot
uri : /lhos/snapshots
total : 0
members : {}
At the end of the day, we remove our credential entity, because it’s not longer used. To delete the credential entity, we use the DELETE method with the URI of our credential entity.
The next query should fail, because the credential entity is no longer valid.
PS C:\Users\p.terlisten> Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials -Method Get -Headers @{'Authorization'=$a.authToken}
The remote server returned an error: (401) Unauthorized. (raised by: Invoke-RestMethod)
HTTPS workaround
The StoreVirtual API is only accessable over HTTPS. By default, the StoreVirtual nodes use an untrusted HTTPS certifificate. This will cause Invoke-RestMethod to fail.
[10,6: Invoke-RestMethod] The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
After a little research, I found a workaround. This workaround uses the System.Security.Cryptography.X509Certificates namespace. You can use this snippet to build a function or add it to a try-catch block.
add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
Final words
The StoreVirtual REST API is really handy. It can be used to perform all important tasks. It’s perfect for automation and it’s faster than the CLI. I’ve used PowerShell in my examples, but I’ve successfully tested it with Python. Make sure to take a look in to the HPE StoreVirtual REST API Reference Guide.
This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
In December 2013, VMware made an christmas gift to the community by releasing pyVmomi. pyVmomi is a SDK that allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Nearly 18 months are past since then and pyVmomi has developed over time.
You can install the official release of pyVmomi using pip (pip installs packages, a recursive acronym).
pip3 install --upgrade pyvmomi
The latest version is available on GitHub. To get the latest version, use
python setup.py develop
or
python setup.py install
That you can fetch the latest version from GitHub is pretty cool and shows a big benefit: The community can contribute to pyVmomi and it’s more frequently updated. A huge benefit in regard of code quality and features.
pyVmomi 6.0.0.2016.4 and later support 2.7, 3.3 and 3.4
pyVmomi 6.0.0 and later support 2.7, 3.3 and 3.4
pyVmomi 5.5.0-2014.1 and 5.5.0-2014.1.1 support Python 2.6, 2.7, 3.3 and 3.4
pyVmomi 5.5.0 and below support Python 2.6 and 2.7
Interesting fact: pyVmomi version numbers correlate with vSphere releases. pyVmomi 6.0.0 was released with the GA of VMware vSphere 6. pyVmomi supports the corresponding vSphere release and the previous four vSphere releases.
I’m using Python 3 for my examples. I wouldn’t recommend to start with Python 2 these days.
First steps
pyVmomi allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Because of this, the VMware vSphere API Reference Documentation will be your best friend.
First of all, you need a connection to the API. To connect to the vSphere API, we have to import and use the module pyVim, more precise, the pyVim.connect module and the SmartConnect function. pyVim.connect is used for the connection handling (creation, deletion…) to the Virtualization Management Object Management Infrastructure (VMOMI). pyVim is part of pyVmomi and it’s installed automatically.
from pyVim.connect import SmartConnect
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
SmartConnect accepts various parameters, but for the beginning it’s sufficient to use three of them: host, user and pwd. You can use “help(SmartConnect)” to get information about the SmartConnect function. “c” is the object (pyVmomi.VmomiSupport.vim.ServiceInstance) which we will use later.
A connection itself is useless. But how can we explore the API? Python doesn’t support typing, so it can be difficult to “explore” an API. That’s why the VMware vSphere API Reference Documentation and the Managed Object Browser (MOB) will be your best friends. The MOB is a web-based interface and represents the vSphere API. It allows you to navigate through the API. Any changes you make through the MOB, by invoking methods, take effect and change the config or will give you an output.
Important note: If you are using VMware vSphere 6 (ESXi 6.0 and vCenter 6.0), you have to enable the MOB. The MOB is disabled by default. Check VMware KB2108405 (The Managed Object Browser is disabled by default in vSphere 6.0) for more details.
Open a browser and open https://ip-or-fqdn/mob. You can use the IP address or the FQDN of an ESXi host or a vCenter Server (Appliance). I use a standalone ESXi 5.5 host in this example.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Our first code
Let’s try something easy. I’ve framed a method in the screenshot above. We will use this method now.
from pyVim.connect import SmartConnect
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
print(c.CurrentTime())
This code will connect to the vSphere API, invoke the method “CurrentTime()” and prints the result. What happens if we execute our first lines of Python code? We will get an error…
Python checks SSL certificates in strict mode. Because of this, untrusted certificates will cause trouble. This applies to Python 3, as well as to Python >= 2.7.9 (PEP 0466). Most people use untrusted certificates. To deal with this, we have to create a context for our HTTP connection. This context can be used by the SmartConnect function. To create a context, we have to import the ssl module of Python.
from pyVim.connect import SmartConnect
import ssl
s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)
print(c.CurrentTime())
“s” is the new object (ssl.SSLContext) we will use and the parameter “sslContext=s” will told SmartConnect to use this object.
Save this code into a file (I called it pyvmomitest.py in my example). Navigate to the folder, open a Python REPL and import the file you’ve saved (module) a moment ago.
C:\Users\p.terlisten\Documents\Development\Python\Playground>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvmomitest
2016-05-20 12:29:23.837049+00:00
>>>
Hurray! We used the vSphere API to get the current date and time (CET).
But what if we have deployed valid certificates? And what about housekeeping? We have connected, but we haven’t disconnected from the API? We can use a try-except block to handle this. And because we are nice, we import also the function “Disconnect” from pyVim.connect to disconnect from the vSphere API at the end.
from pyVim.connect import SmartConnect, Disconnect
import ssl
s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE
try:
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
print('Valid certificate')
except:
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)
print('Invalid or untrusted certificate')
print(c.CurrentTime())
Disconnect(c)
With this code, we should get the following output.
C:\Users\p.terlisten\Documents\Development\Python\Playground>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvmomitest
Invalid or untrusted certificate
2016-05-20 12:36:28.707876+00:00
>>>
Okay, the vSphere API wasn’t designed to retrieve the current date and time. Let’s look at something more useful. This script will give us the names of all VMs in the datacenter.
from pyVim.connect import SmartConnect, Disconnect
import ssl
s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE
try:
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
print('Valid certificate')
except:
c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)
print('Invalid or untrusted certificate')
datacenter = c.content.rootFolder.childEntity[0]
vms = datacenter.vmFolder.childEntity
for i in vms:
print(i.name)
Disconnect(c)
Let’s take this statement and look at everything after the “c”. We will use the MOB to navigate through the API. This will help you to understand, how the Python code and the structure of the vSphere API correlate.
datacenter = c.content.rootFolder.childEntity[0]
Open the MOB. You will easily find the property “content”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click on “content” and search for the property “rootFolder”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click on the value “ha-folder-root.” The property “childEntity” is an ManagedObjectReference (MOR) and references to all datacenters (the counting starts at 0) known to the ESXi or vCenter. The value “childEntity[0]” will give us the first datacenter.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
If we have the datacenter, the way to get the names of the VMs is the same. You can use the MOB, to verify this.
vms = datacenter.vmFolder.childEntity
Click on the value “ha-datacenter”. At the bottom of the list, you will find the property “vmFolder”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click on the value “ha-folder-vm”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
The MOR “childEntity” references to two VMs. Click on one of the IDs.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
The property “name” includes the name of the VM. Because of this, we can use a simple
for i in vms:
print(i.name)
to get the name for each VM.
Summary
This was only a short introduction into pyVmomi. You should be now able to install pyVmomi, make a connection to the vSphere API and retrieve some basic stuff.
Every day I discover something new. It’s important to understand how the vSphere API works. Play with pyVmomi and with the vSphere API. It looks harder as it is.
This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
I’m not a developer. I’m an infrastructure guy. All I ever needed was to write some scripts. Therefore, I never needed more than DOS batches, BASH/ CSH/ KSH, Visual Basic Script and nowadays PowerShell. So why should I learn another programming language?
One to rule them all?
I don’t think that there is a single programming language that is perfect for all use cases. The spread and acceptance of a language shows a positive correlation with the number of available frameworks, tools and libraries. That’s why I love the Microsoft PowerShell. Nearly all vendors offer a PowerShell module for their products (think about VMware PowerCLI, Rubrik, Veeam, DataCore and much more). The downside: The PowerShell code has to run on a Windows box. I think the time of writing DOS batches is over. UNIX shell scripts are still awesome, but focused on UNIX.
Different problems require different tools. I think it’s better to know a few, general-purpose tools well, as every conceivable special tool. Don’t get me wrong: PowerShell is awesome powerful! It’s quite easy to learn and you will have quick success.
Why Python?
Python is easy to learn (I can confirm this, at least for what I’ve seen). Python was developed from scratch by Guido van Rossum in the early 1990s. Python is an interpreted and dynamic programming language, which supports multiple paradigms, like the object-oriented or the functional programming. Python features a dynamic type system and automatic memory management. It uses only 35 keywords, what makes it easy to lern. It’s underlying philosophy is The Zen of Python.
Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. …
These rules lead to code with a high legibility, and it is possible to solve problems with fewer lines of code. Python is highly extensible. It comes with a large standard library and you can choose from 72.000 packages, that are available using the official 3rd party repository.
Currently, the stabled releases are 2.7 and 3.5. I recommend to start with the 3.5 release. You can get the latest release from python.org. They offer packages for Windows, MacOS X and Linux/ UNIX. Python comes with an IDE called IDLE (Integrated Development and Learning Environment). Make sure that you take a look into the official documentation! If you want something more comfortable, try JetBrain PyCharm. JetBrains offer a free community edition for Windows, MacOS X and Linux. But it’s not the worst idea to start with IDLE. I use both IDEs, IDLE and PyCharm.
Where can you get help? YouTube is full of videos about Python. If you have a Pluralsight subscription, checkout the courses on Pluralsight. There are many good books out there, as well as some good howtos. Just use Google. It depends on what type of learner you are.
Learn the basics and try to strengthen them during a small project. Buy a Raspberry Pi. Raspberry Pi and Python are the biggest friends. If you are focused on VMware vSphere, take a closer look at the VMware vSphere API Python Bindings. Create yourself a project to learn.
I just started to learn Python, but I think that this wasn’t the worst idea in my life.
To change your privacy setting, e.g. granting or withdrawing consent, click here:
Settings
