Category Archives: Development

Getting started with the Microsoft Graph PowerShell SDK

This posting is ~1 year years old. You should keep this in mind. IT is a short living business. This information might be outdated.

There is a new API in town… naa, not really new, but the Microsoft Graph API will replace most, if not all, other Azure AD/ Microsoft 365 APIs. Actually, Microsoft has planned to retire Azure AD Graph API and ADAL in Juni 2022. Now they have postponed this date to somewhere after December 2022. This will give you some extra time to refactor your PowerShell scrips and move them to use the PowerShell SDK for Graph.

What is Microsoft Graph? Microsoft Graph is the spider web that connects everything in Microsoft 365. One part of Graph is a single and unified API endpoint, which allows you to access and manage Azure AD and Microsoft 365 services, such as Teams and Exchange Online, Intune etc.

Source: Microsoft

Instead of different endpoints, the Graph API uses a single endpoint (https://graph.microsoft.com/v1.0), which is the pretty nice thing if it comes down to Firewall rules and traffic management. This single endpoint, together with a single access token, allows you to manage all M365 services using REST API calls. So you don’t have to use the PowerShell SDK, you can also use Python or curl. Furthermore, the Microsoft Graph PowerShell SDK is Open Source, it offers cross-platform support (Linux, MacOS, Windows), and its available on PowerShell 5.1 nd above. This is a pretty important thing for me. Now I can use a single PowerShell Module to manage all my M365 services. This was PITA in the past. Depending on the service, I had to use different PowerShell modules, and I had to switch between PowerShell and PoSh Core.

Install the Microsoft Graph PowerShell SDK

The installation is pretty simple. All you need is

  • at least PowerShell 5.1 or later,
  • .NET 4.7.2 or later, and
  • the PowerShell script execution policy must be set to remote signed or a less restrictive execution policy

The installation is done by calling Install-Module.

Install-Module Microsoft.Graph -Scope CurrentUser

Sure, you can also use -Scope AllUsers, but I prefer to install PowerShell modules in my CurrentUser context. That’s it! Now let’s authenticate to the Graph API.

Authentication & Authorization

First of all: The Graph API knows two types of authentication:

  • delegated access, and
  • app-only access

Delegates access allows an application to act as a specific user, where as application access allows an application to act as its own entity. If you want that an API call is executed with the users permissions, then you should use delegated access. If you want to run scripts, or a service, then you want to use app-only access. Some API call are only possible with delegated access, some calls can only be made with app-only access, and some calls can be made with either of the two methods.

As part of the login process, you must define the requested permission scope. This is to ensure that the least possible privileges are used. For example:

Connect-MgGraph -Scopes "User.Read.All","Group.ReadWrite.All"

As you can see, this command will open a connection to the Graph API and you must consent the requested permissions after entering valid user credentials. Side note: Graph offers all the modern authentication stuff that we want to use today.

The tenant is specified by the singed-in user. If you want to connect to a specific tenant (to avoid usage of a cached access token), then you can add –TenantId <TenantId> to the command above.

As you can see, I now have a working connection using the Graph API using delegated access and with the requested permission scopes.

This map of Azure AD and MSOnline cmdlets was pretty helpful when playing around with Graph.

If you want to run a script, you might want to register an app in your tenant. This allows you to define the required permission scopes as properties of the app itself. I will show this in a separate blog post. This blog post will also cover the certificate-based authentication, as well as the authentication using a client secret.

Mail notification for specific Active Directory security events

This posting is ~2 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

A customer used PRTG Network Monitor to notify him in case of account lockouts. This worked quite well until we implemented Admin Tiering. In order to get a mail notification in case of an account lockout, or other security-relevant events in Active Directory, I customized some scripts from my PowerShell dump.

The solution is pretty simple: I used the Task Planner to run a PowerShell script if a specific event id occurs. The events are generated in case of a various number of Active Directory events. You have to enable audit policy to get the needed events in the security event log. Take a look at Microsoft audit policy recommendations and enable what you need. I recommend to enable the stronger settings.

Image by Vitor Dutra Kaosnoff from Pixabay

I implemented five scripts:

  • Account lockout
  • New account in Active Directory
  • New member in domain-local group
  • New member in domain-global group
  • New member in universal group

The implementation is pretty easy. Create a basic task and execute this task if a specific event occurs.

The action is “Start a program”, like in the following screenshot.

Save the task and then change the user, which is used to run this task, to SYSTEM.

Please note, that you run these scripts with SYSTEM privileges. So make sure that NO ONE can easily edit these scripts! Best way is to restrict it to specific domain admins, restrict access to your domain controllers etc!

Repeat these steps for each script and implement them on each domain controller.

Please leave a comment with feedback :)

How to install Visual Studio Code on Linux Mint 18

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I have wrote about the installation of PowerShell Core in Linux Mint 18 yesterday. Today, I want to show you, how to install Visual Studio Code on Linux Mint 18. The installation is really easy:

  1. Download the deb package
  2. Install the deb package
  3. Run Visual Studio Code

You can download the latest packages for Windows, Linux (deb and rpm, if you want even a tar ball), and Mac on the Visual Studio Code download page. Download the deb file. To install the package, open a Terminal window and run dpkg .

patrick@nb-patrick ~/Downloads
 % sudo dpkg -i code_1.17.1-1507645403_amd64.deb 
[sudo] password for patrick: 
Selecting previously unselected package code.
(Reading database ... 236413 files and directories currently installed.)
Preparing to unpack code_1.17.1-1507645403_amd64.deb ...
Unpacking code (1.17.1-1507645403) ...
Setting up code (1.17.1-1507645403) ...
Processing triggers for gnome-menus (3.13.3-6ubuntu3.1) ...
Processing triggers for desktop-file-utils (0.22+linuxmint1) ...
Processing triggers for mime-support (3.59ubuntu1) ...
patrick@nb-patrick ~/Downloads
 %

Visual Studio Code on Linux

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

sudo  might ask you for a password. That’s it! Now you can simply start VS Code.After you have installed your favorite extensions, VS Code is ready to code.

How to install PowerShell Core on Linux Mint 18

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Beside my Lenovo X250, which is my primary working machine, I’m using a HP ProBook 6450b. This was my primary working machine from 2010 until 2013. With a 128 GB SSD, 8 GB RAM and the Intel i5 M 450 CPU, it is still a pretty usable machine. I used it mainly during projects, when I needed a second laptop (or the PC Express card with the serial port…). It was running Windows 10, until I decided to try Linux MInt. I used Linux as my primary desktop OS more than a decade ago. It was quite productive, but especially with laptops, there were many things that does not worked out of the box.

Because I use PowerShell quite often, and PowerShell is available for Windows, MacOS and Linux, the installation of PowerShell on this Linux laptop is a must.

How to install PowerShell?

Linux Mint is a based on Ubuntu, and I’m currently using Linux Mint 18.2. Microsoft offers different pre-compiled packages on the PowerShell GitHub repo. For Linux Mint 18, you have to download the Ubuntu 16.04 package. For Linux Mint 17, you will need the 14.04 package. Because you need the shell to install the packages, you can download the deb package from the shell as well. I used wget to download the deb package.

patrick@nb-patrick ~/Downloads
 % wget https://github.com/PowerShell/PowerShell/releases/download/v6.0.0-beta.8/powershell_6.0.0-beta.8-1.ubuntu.16.04_amd64.deb

The next step is to install the deb package, and to fix broken dependencies. Make sure that you run dpkg  with sudo .

patrick@nb-patrick ~/Downloads
 % sudo dpkg -i powershell_6.0.0-beta.8-1.ubuntu.16.04_amd64.deb 
Selecting previously unselected package powershell.
(Reading database ... 235671 files and directories currently installed.)
Preparing to unpack powershell_6.0.0-beta.8-1.ubuntu.16.04_amd64.deb ...
Unpacking powershell (6.0.0-beta.8-1.ubuntu.16.04) ...
dpkg: dependency problems prevent configuration of powershell:
 powershell depends on liblttng-ust0; however:
  Package liblttng-ust0 is not installed.

dpkg: error processing package powershell (--install):
 dependency problems - leaving unconfigured
Processing triggers for man-db (2.7.5-1) ...
Errors were encountered while processing:
 powershell

Looks like it failed, because of broken dependencies. But this can be easily fixed. To fix the broken dependencies, run apt-get -f install . Make sure that you run it with sudo !

patrick@nb-patrick ~/Downloads
 % sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  liblttng-ust-ctl2 liblttng-ust0 liburcu4
The following NEW packages will be installed:
  liblttng-ust-ctl2 liblttng-ust0 liburcu4
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 247 kB of archives.
After this operation, 1.127 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://mirror.netcologne.de/ubuntu xenial/universe amd64 liburcu4 amd64 0.9.1-3 [47,3 kB]
Get:2 http://mirror.netcologne.de/ubuntu xenial/universe amd64 liblttng-ust-ctl2 amd64 2.7.1-1 [72,2 kB]
Get:3 http://mirror.netcologne.de/ubuntu xenial/universe amd64 liblttng-ust0 amd64 2.7.1-1 [127 kB]
Fetched 247 kB in 0s (841 kB/s)        
Selecting previously unselected package liburcu4:amd64.
(Reading database ... 236372 files and directories currently installed.)
Preparing to unpack .../liburcu4_0.9.1-3_amd64.deb ...
Unpacking liburcu4:amd64 (0.9.1-3) ...
Selecting previously unselected package liblttng-ust-ctl2:amd64.
Preparing to unpack .../liblttng-ust-ctl2_2.7.1-1_amd64.deb ...
Unpacking liblttng-ust-ctl2:amd64 (2.7.1-1) ...
Selecting previously unselected package liblttng-ust0:amd64.
Preparing to unpack .../liblttng-ust0_2.7.1-1_amd64.deb ...
Unpacking liblttng-ust0:amd64 (2.7.1-1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Setting up liburcu4:amd64 (0.9.1-3) ...
Setting up liblttng-ust-ctl2:amd64 (2.7.1-1) ...
Setting up liblttng-ust0:amd64 (2.7.1-1) ...
Setting up powershell (6.0.0-beta.8-1.ubuntu.16.04) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...

That’s it! PowerShell is now installed.

patrick@nb-patrick ~/Downloads
 % powershell
PowerShell v6.0.0-beta.8
Copyright (C) Microsoft Corporation. All rights reserved.

PS /home/patrick/Downloads>  Get-ChildItem /home/patrick                                                                                                                                              


    Directory: /home/patrick


Mode                LastWriteTime         Length Name                                                                                                                                                
----                -------------         ------ ----                                                                                                                                                
d-----         10/10/17  10:26 PM                Desktop                                                                                                                                             
d-----         10/14/17   8:45 AM                Documents                                                                                                                                           
d-----         10/14/17   8:41 AM                Downloads                                                                                                                                           
d-----         10/10/17  10:26 PM                Music                                                                                                                                               
d-----         10/14/17   8:37 AM                Pictures                                                                                                                                            
d-----         10/10/17  10:26 PM                Public                                                                                                                                              
d-----         10/10/17  10:26 PM                Templates                                                                                                                                           
d-----         10/10/17  10:26 PM                Videos                                                                                                                                              


PS /home/patrick/Downloads> exit                                                                                                                                                                      
patrick@nb-patrick ~/Downloads
 %

Yep, looks like a PowerShell prompt…on Linux. Thank you, Microsoft! :)

Monitoring hardware status with Python and vSphere API calls

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Apparently it’s “how to monitor hardware status” week on vcloudnine.de. Some days ago, I wrote an article about using SNMP for hardware monitoring. You can also use the vSphere Web Client to get the status of the host hardware. A third way is through the vSphere API. I just want to share a short example how to use vSphere API calls and pyVmomi. pyVmomi is the Python SDK for the VMware vSphere API.

Get hardware status with vSphere API calls

I just want to share a small example, that shows the basic principle. The script gathers the temperature sensor data of a ProLiant DL360 G7 running ESXi 6.0 U2 using vSphere API calls.

The output of the script looks like this:

Valid certificate

Hostname: esx1.lab.local
Type: ProLiant DL360 G7
Getting temperature sensor data...

Other 1 Temp 28 --- Normal 65.0 Degrees C
Drive Backplane 1 Temp 27 --- Normal 35.0 Degrees C
Peripheral Bay 8 Temp 26 --- Normal 42.0 Degrees C
Peripheral Bay 7 Temp 25 --- Normal 38.0 Degrees C
Peripheral Bay 6 Temp 24 --- Normal 45.0 Degrees C
Peripheral Bay 5 Temp 23 --- Normal 39.0 Degrees C
Peripheral Bay 4 Temp 22 --- Normal 46.0 Degrees C
Peripheral Bay 3 Temp 21 --- Normal 44.0 Degrees C
Peripheral Bay 2 Temp 20 --- Normal 39.0 Degrees C
Peripheral Bay 1 Temp 19 --- Normal 37.0 Degrees C
Other 5 Temp 18 --- Normal 39.0 Degrees C
Memory Module 10 Temp 17 --- Normal 36.0 Degrees C
Other 4 Temp 16 --- Normal 34.0 Degrees C
Other 3 Temp 15 --- Normal 35.0 Degrees C
Memory Module 9 Temp 14 --- Normal 34.0 Degrees C
Power Supply 5 Temp 13 --- Normal 45.0 Degrees C
Power Supply 4 Temp 12 --- Normal 36.0 Degrees C
Memory Module 8 Temp 11 --- Normal 36.0 Degrees C
Memory Module 7 Temp 10 --- Normal 38.0 Degrees C
Memory Module 6 Temp 9 --- Normal 36.0 Degrees C
Memory Module 5 Temp 8 --- Normal 39.0 Degrees C
Memory Module 4 Temp 7 --- Normal 35.0 Degrees C
Memory Module 3 Temp 6 --- Normal 37.0 Degrees C
Memory Module 2 Temp 5 --- Normal 36.0 Degrees C
Memory Module 1 Temp 4 --- Normal 38.0 Degrees C
Other 2 Temp 3 --- Normal 40.0 Degrees C
Other 1 Temp 2 --- Normal 40.0 Degrees C
Other 1 Temp 1 --- Normal 27.0 Degrees C
>>>

Nothing fancy. You can easily loop through numericSensorInfo to gather data from other sensors. Use the Managed Object Browser (MOB) to navigate through the API. This is handy if you search for specific sensors. If you need accurate data, the vSphere API is the way to go. If you focus on something lightweight, try SNMP.

HPE StoreVirtual REST API

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Representational State Transfer (REST) APIs are all the rage. REST was defined by Roy Thomas Fielding in his PhD dissertation “Architectural Styles and the Design of Network-based Software Architectures“. The architectural style of REST describes six constraints:

  • Uniform interface
  • Stateless
  • Cacheable
  • Client – Server communication
  • Layered system
  • Code on demand

RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.) to send data to, or retrieve data from remote systems. To do so, REST APIs use Uniform Resource Identifiers (URIs) to interact with remote systems. Thus, a client can interact with a remote system over a REST API using standard HTTP URIs and HTTP verbs. For the data transfer, common internet media types, like JSON or XML are used. It’s important to understand that REST is not a standard per se. But most implementations make use of standards such as HTTP, URI, JSON or XML.

Because of the uniform interface, you have different choices in view of a client. I will use PowerShell and the Invoke-RestMethod cmdlet in my examples.

HPE StoreVirtual REST API

With the release of LeftHand OS 11.5 (the latest release is 12.6), HPE added a REST API for management and storage provisioning. Due to a re-engineered management stack, the REST API is significantly faster than the same task processed on the CLI or using the  Centralized Management Console (CMC). It’s perfect for automation and scripting. It allows customers to achieve a higher level of automation and operational simplicity. The StoreVirtual REST API is using JavaScript Object Notation (JSON) for data transfer between client and the StoreVirtual management group. With the REST API, you can

  • Read, create, and modify volumes
  • Create and delete snapshots
  • Create, modify, and delete servers
  • Grant and revoke access of servers to volumes

I use two StoreVirtal VSA (LeftHand OS 12.6) in my lab. Everything I show in this blog post is based on LeftHand OS 12.6.

The REST API in LeftHand OS 12.6 uses:

  • HTTPS 1.1
  • media types application/JSON
  • Internet media types application/schema+JSON
  • UTF-8 character encoding

RESTful APIs typically use HTTP and HTTP verbs (GET, POST, PUT, DELETE, etc.). I case of the StoreVirtual REST API:

  • GET is used to retrieve an object. No body is necessary.
  • PUT is used to update an object. The information to update the object is sent within the body.
  • POST is used to create of an object, or to invoke an action or event. The necessary information are sent within the body.
  • DELETE is used to delete an object.

Entry point for all REST API calls is /lhos, starting from a node, eg.

https://fqdn-or-ip:8081/lhos/

Subsequent resources are relative to this base URI. Resources are:

Resource pathDescription
/lhos/managementGroupManagement group entity
/lhos/clustersCluster collection
/lhos/cluster/<id>Cluster entity
/lhos/credentialsCredentials collection
/lhos/credentials/<session token>Credentials entity
/lhos/serversServer collection
/lhos/servers/<id>Server entity
/lhos/snapshotsSnapshot collection
/lhos/snapshots/<id>Snapshot entity
/lhos/volumesVolume collection
/lhos/volumes/<id> Volume entity

The object model of the StoreVirtual REST API uses

  • Collections, and
  • Entities

to address resources. An entity is used to address individual resources, whereas a collection is a group of individual resources. Resources can be addressed by using a URI.

Exploring the API

First of all, we need to authenticate us. Without a valid authentication token, no REST API queries can be made. To create a credential entity, we have to use the POST method.

$cred = @{
            user='admin';
            password='Passw0rd'
}

$body = $cred | ConvertTo-Json

$a = Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials -Method Post -Body $body -ContentType 'application/JSON'

$cred is a hash table which includes the username and the password. This hash table is converted to the JSON format with the ConvertTo-Json cmdlet. The JSON data will be used as body for our query. The result is an authentication token.

PS C:\Users\p.terlisten> $a

authToken                                                                                                                                                                                     
---------                                                                                                                                                                                     
fa0a7b56-0134-400f-9d62-79b3071c950a

This authentication token must be used for all subsequent API queries. This query retrieves a collection of all valid sessions.

$b = Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials -Method Get -Headers @{'Authorization'=$a.authToken}

The GET method is used, and the authentication token is sent with the header of the request.

PS C:\Users\p.terlisten> $b


name        : REST Sessions Collection
description : Collection of authentication sessions used by the REST server.
type        : RESTSession
uri         : /lhos/credentials
total       : 1
members     : {@{name=fa0a7b56-0134-400f-9d62-79b3071c950a; description=REST Session; type=RESTSession; id=0; uri=/lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a; 
              created=2016-06-07T08:38:06.426241Z; modified=2016-06-07T08:44:28.255283Z; userName=admin; clientIP=192.168.200.90}}

To retrieve an individual credential entity, the URI of the entity must be used.

$b = Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a -Method Get -Headers @{'Authorization'=$a.authToken}

The result of this query is the individual credential entity

PS C:\Users\p.terlisten> $b


name        : fa0a7b56-0134-400f-9d62-79b3071c950a
description : REST Session
type        : RESTSession
id          : 0
uri         : /lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a
created     : 2016-06-07T08:38:06.426241Z
modified    : 2016-06-07T08:51:56.358096Z
userName    : admin
clientIP    : 192.168.200.90

It’s important to know, that if a session has not been used for 15 minutes, it is automatically removed. The same applies to constantly active sessions after 24 hours. After 24 hours, the credential entity will be automatically removed.

Let’s try to create a volume. The information about this new volume has to be sent within the body of our request. We use again the ConvertTo-Json cmdlet to convert a hash table with the necessary information to the JSON format.

$vol = @{
            name='api-vol';
            description='Volume created via REST API';
            size=1073741824;
            clusterId=28;
            isThinProvisioned=$true;
            dataProtectionLevel=2
}

$body = $vol | ConvertTo-Json

Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/volumes -Method Post -Headers @{'Authorization'=$a.authToken} -Body $body -ContentType 'application/JSON'

The size must be specified in bytes. As a result, Invoke-RestMethod will output this:

name                          : api-vol
description                   : Volume created via REST API
type                          : volume
id                            : 3058
uri                           : /lhos/volumes/3058
created                       : 2016-06-07T08:55:41Z
modified                      : 
friendlyName                  : 
transport                     : 0
isThinProvisioned             : True
size                          : 1073741824
serialNumber                  : f9df3e8bb0a160f269027ecc0371884e0000000000000bf2
provisionedSpace              : 1073741824
numberOfReplicas              : 2
dataProtectionLevel           : 2
iscsiIqn                      : iqn.2003-10.com.lefthandnetworks:mgmt:3058:api-vol
isPrimary                     : True
isDeleting                    : False
bytesWritten                  : 0
isAvailable                   : True
clusterName                   : CLUSTER
clusterId                     : 28
isVIPRebalancing              : False
isAdaptiveOptimizationEnabled : True
isMigrating                   : False
scsiLUNStatus                 : available
hasUnrecoverableIOErrors      : False
restripePendingStatus         : none
replicationStatus             : normal
resynchronizationStatus       : none
migrationStatus               : none
isLicensed                    : True
transportServerId             : 0
fcTransportStatus             : 0
createdBy                     : Unknown
iscsiSessions                 : 
fibreChannelPaths             : 
snapshots                     : @{name=snapshots; type=snapshot; uri=/snapshots?volumeName=api-vol; resource=}

Using the CMC, we can confirm that the volume was successfully created.

storevirtual_rest_api_vol_1

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Since we have a volume, we can create a snapshot. To create a snapshot, we need to invoke an action on the volume entity. We have to use the POST method and the URI of our newly created volume.

$snapshot = @{
                action='createSnapshot';
                parameters = @{
                                name='vol-api-snap';
                                description='Volume snapshot created via REST API';
}
}

$body = $snapshot | ConvertTo-Json

Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/volumes/3058 -Method Post -Headers @{'Authorization'=$a.authToken} -Body $body -ContentType 'application/JSON'

In case of a successful query, Invoke-RestMethod will give us this output.

name                     : vol-api-snap
description              : Volume snapshot created via REST API
type                     : snapshot
id                       : 3060
uri                      : /lhos/snapshots/3060
created                  : 2016-06-07T09:01:12Z
modified                 : 
friendlyName             : 
transport                : 0
isThinProvisioned        : True
size                     : 1073741824
serialNumber             : f9df3e8bb0a160f269027ecc0371884e0000000000000bf4
provisionedSpace         : 8388608
iscsiIqn                 : iqn.2003-10.com.lefthandnetworks:mgmt:3060:vol-api-snap
isPrimary                : True
isDeleting               : False
bytesWritten             : 0
isAvailable              : True
clusterName              : CLUSTER
clusterId                : 28
iscsiSessions            : 
fibreChannelPaths        : 
snapshotACL              : 
writableSpaceUsed        : 0
managedBy                : 0
isAutomatic              : False
isMigrating              : False
scsiLUNStatus            : available
hasUnrecoverableIOErrors : False
restripePendingStatus    : none
replicationStatus        : normal
resynchronizationStatus  : none
migrationStatus          : none
isLicensed               : True
transportServerId        : 0
fcTransportStatus        : 0
createdBy                : Unknown

Again, we can use the CMC to confirm the success of our operation.

storevirtual_rest_api_vol_2

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

To delete the snapshot, the DELETE method and the URI of the snapshot entity must be used.

Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/snapshots/3060 -Method Delete -Headers @{'Authorization'=$a.authToken}

To confirm the successful deletion of the snapshot, the GET method can be used. The GET method will retrieve a collection of all snapshot entities.

Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/snapshots -Method Get -Headers @{'Authorization'=$a.authToken}

The result will show no members inside of the snapshot collection.

name        : Snapshots Collection
description : Collection of Snapshot objects
type        : snapshot
uri         : /lhos/snapshots
total       : 0
members     : {}

At the end of the day, we remove our credential entity, because it’s not longer used. To delete the credential entity, we use the DELETE method with the URI of our credential entity.

Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials/fa0a7b56-0134-400f-9d62-79b3071c950a -Method Delete -Headers @{'Authorization'=$a.authToken}

The next query should fail, because the credential entity is no longer valid.

PS C:\Users\p.terlisten> Invoke-RestMethod -Uri https://vsa1.lab.local:8081/lhos/credentials -Method Get -Headers @{'Authorization'=$a.authToken}
The remote server returned an error: (401) Unauthorized. (raised by: Invoke-RestMethod)

HTTPS workaround

The StoreVirtual API is only accessable over HTTPS. By default, the StoreVirtual nodes use an untrusted HTTPS certifificate. This will cause Invoke-RestMethod to fail.

[10,6: Invoke-RestMethod] The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

After a little research, I found a workaround. This workaround uses the System.Security.Cryptography.X509Certificates namespace. You can use this snippet to build a function or add it to a try-catch block.

add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@

[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

Final words

The StoreVirtual REST API is really handy. It can be used to perform all important tasks. It’s perfect for automation and it’s faster than the CLI. I’ve used PowerShell in my examples, but I’ve successfully tested it with Python. Make sure to take a look in to the HPE StoreVirtual REST API Reference Guide.

First steps with Python and pyVmomi (vSphere SDK for Python)

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

In December 2013, VMware made an christmas gift to the community by releasing pyVmomi. pyVmomi is a SDK that allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Nearly 18 months are past since then and pyVmomi has developed over time.

I’ve started to play around with Python, and I’ve written about the reasons in one of my last blog posts (Hey infrastructure guy, you should learn Python!).

How to get pyVmomi?

You can install the official release of pyVmomi using pip (pip installs packages, a recursive acronym).

pip3 install --upgrade pyvmomi

The latest version is available on GitHub. To get the latest version, use

python setup.py develop

or

python setup.py install

That you can fetch the latest version from GitHub is pretty cool and shows a big benefit: The community can contribute to pyVmomi and it’s more frequently updated. A huge benefit in regard of code quality and features.

What Python releases are support?

The latest information about supported Python releases can be found on the GitHub page of the project.

  • pyVmomi 6.0.0.2016.4 and later support 2.7, 3.3 and 3.4
  • pyVmomi 6.0.0 and later support 2.7, 3.3 and 3.4
  • pyVmomi 5.5.0-2014.1 and 5.5.0-2014.1.1 support Python 2.6, 2.7, 3.3 and 3.4
  • pyVmomi 5.5.0 and below support Python 2.6 and 2.7

Interesting fact: pyVmomi version numbers correlate with vSphere releases. pyVmomi 6.0.0 was released with the GA of VMware vSphere 6. pyVmomi supports the corresponding vSphere release and the previous four vSphere releases.

I’m using Python 3 for my examples. I wouldn’t recommend to start with Python 2 these days.

First steps

pyVmomi allows you to manage VMware ESXi and vCenter using Python and the VMware vSphere API. Because of this, the VMware vSphere API Reference Documentation will be your best friend.

First of all, you need a connection to the API. To connect to the vSphere API, we have to import and use the module pyVim, more precise, the pyVim.connect module and the SmartConnect function. pyVim.connect is used for the connection handling (creation, deletion…) to the Virtualization Management Object Management Infrastructure (VMOMI). pyVim is part of pyVmomi and it’s installed automatically.

from pyVim.connect import SmartConnect


c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')

SmartConnect accepts various parameters, but for the beginning it’s sufficient to use three of them: host, user and pwd. You can use “help(SmartConnect)” to get information about the SmartConnect function. “c” is the object (pyVmomi.VmomiSupport.vim.ServiceInstance) which we will use later.

A connection itself is useless. But how can we explore the API? Python doesn’t support typing, so it can be difficult to “explore” an API. That’s why the VMware vSphere API Reference Documentation and the Managed Object Browser (MOB) will be your best friends. The MOB is a web-based interface and represents the vSphere API. It allows you to navigate through the API. Any changes you make through the MOB, by invoking methods, take effect and change the config or will give you an output.

Important note: If you are using VMware vSphere 6 (ESXi 6.0 and vCenter 6.0), you have to enable the MOB. The MOB is disabled by default. Check VMware KB2108405 (The Managed Object Browser is disabled by default in vSphere 6.0) for more details.

Open a browser and open https://ip-or-fqdn/mob. You can use the IP address or the FQDN of an ESXi host or a vCenter Server (Appliance). I use a standalone ESXi 5.5 host in this example.

python_esxi_mob_1

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Our first code

Let’s try something easy. I’ve framed a method in the screenshot above. We will use this method now.

from pyVim.connect import SmartConnect


c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')

print(c.CurrentTime())

This code will connect to the vSphere API, invoke the method “CurrentTime()” and prints the result. What happens if we execute our first lines of Python code? We will get an error…

self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

Python checks SSL certificates in strict mode. Because of this, untrusted certificates will cause trouble. This applies to Python 3, as well as to Python >= 2.7.9 (PEP 0466). Most people use untrusted certificates. To deal with this, we have to create a context for our HTTP connection. This context can be used by the SmartConnect function. To create a context, we have to import the ssl module of Python.

from pyVim.connect import SmartConnect
import ssl


s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE

c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)

print(c.CurrentTime())

“s” is the new object (ssl.SSLContext) we will use and the parameter “sslContext=s” will told SmartConnect to use this object.

Save this code into a file (I called it pyvmomitest.py in my example). Navigate to the folder, open a Python REPL and import the file you’ve saved (module) a moment ago.

C:\Users\p.terlisten\Documents\Development\Python\Playground>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvmomitest
2016-05-20 12:29:23.837049+00:00
>>>

Hurray! We used the vSphere API to get the current date and time (CET).

But what if we have deployed valid certificates? And what about housekeeping? We have connected, but we haven’t disconnected from the API? We can use a try-except block to handle this. And because we are nice, we import also the function “Disconnect” from pyVim.connect to disconnect from the vSphere API at the end.

from pyVim.connect import SmartConnect, Disconnect
import ssl

s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE

try:
    c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
    print('Valid certificate')
except:
    c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)
    print('Invalid or untrusted certificate')

print(c.CurrentTime())

Disconnect(c)

With this code, we should get the following output.

C:\Users\p.terlisten\Documents\Development\Python\Playground>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvmomitest
Invalid or untrusted certificate
2016-05-20 12:36:28.707876+00:00
>>>

Okay, the vSphere API wasn’t designed to retrieve the current date and time. Let’s look at something more useful. This script will give us the names of all VMs in the datacenter.

from pyVim.connect import SmartConnect, Disconnect
import ssl

s = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode = ssl.CERT_NONE

try:
    c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd')
    print('Valid certificate')
except:
    c = SmartConnect(host="192.168.20.1", user="root", pwd='Passw0rd', sslContext=s)
    print('Invalid or untrusted certificate')

datacenter = c.content.rootFolder.childEntity[0]
vms = datacenter.vmFolder.childEntity

for i in vms:
    print(i.name)

Disconnect(c)

Let’s take this statement and look at everything after the “c”. We will use the MOB to navigate through the API. This will help you to understand, how the Python code and the structure of the vSphere API correlate.

datacenter = c.content.rootFolder.childEntity[0]

Open the MOB. You will easily find  the property “content”.

python_esxi_mob_2

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Click on “content” and search for the property “rootFolder”.

python_esxi_mob_3

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Click on the value “ha-folder-root.” The property “childEntity” is an ManagedObjectReference (MOR) and references to all datacenters (the counting starts at 0) known to the ESXi or vCenter. The value “childEntity[0]” will give us the first datacenter.

python_esxi_mob_4

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

If we have the datacenter, the way to get the names of the VMs is the same. You can use the MOB, to verify this.

vms = datacenter.vmFolder.childEntity

Click on the value “ha-datacenter”. At the bottom of the list, you will find the property “vmFolder”.

python_esxi_mob_5

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Click on the value “ha-folder-vm”.

python_esxi_mob_6

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

The MOR “childEntity” references to two VMs. Click on one of the IDs.

python_esxi_mob_7

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

The property “name” includes the name of the VM. Because of this, we can use a simple

for i in vms:
    print(i.name)

to get the name for each VM.

Summary

This was only a short introduction into pyVmomi. You should be now able to install pyVmomi, make a connection to the vSphere API and retrieve some basic stuff.

Every day I discover something new. It’s important to understand how the vSphere API works. Play with pyVmomi and with the vSphere API. It looks harder as it is.

btw: There is a Hands-on-Lab available “HOL-SDC-1622 VMware Development Tools and SDKs“. Check it out!

Hey infrastructure guy, you should learn Python!

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

I’m not a developer. I’m an infrastructure guy. All I ever needed was to write some scripts. Therefore, I never needed more than DOS batches, BASH/ CSH/ KSH, Visual Basic Script and nowadays PowerShell. So why should I learn another programming language?

One to rule them all?

I don’t think that there is a single programming language that is perfect for all use cases. The spread and acceptance of a language shows a positive correlation with the number of available frameworks, tools and libraries. That’s why I love the Microsoft PowerShell. Nearly all vendors offer a PowerShell module for their products (think about VMware PowerCLI, Rubrik, Veeam, DataCore and much more). The downside: The PowerShell code has to run on a Windows box. I think the time of writing DOS batches is over. UNIX shell scripts are still awesome, but focused on UNIX.

Different problems require different tools. I think it’s better to know a few, general-purpose tools well, as every conceivable special tool. Don’t get me wrong: PowerShell is awesome powerful! It’s quite easy to learn and you will have quick success.

Why Python?

Python is easy to learn (I can confirm this, at least for what I’ve seen). Python was developed from scratch by Guido van Rossum in the early 1990s. Python is an interpreted and dynamic programming language, which supports multiple paradigms, like the object-oriented or the functional programming. Python features a dynamic type system and automatic memory management. It uses only 35 keywords, what makes it easy to lern. It’s underlying philosophy is The Zen of Python.

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.

These rules lead to code with a high legibility, and it is possible to solve problems with fewer lines of code. Python is highly extensible. It comes with a large standard library and you can choose from 72.000 packages, that are available using the official 3rd party repository.

For me, as an infrastructure guy, the VMware vSphere API Python Bindings, the 3PAR Python client or the module for the Alcatel-Lucent Enterprise OmniSwitch RESTful API are reasons enough to start with Python. It’s the extensibility and platform independence of Python, what makes it so interesting. Like PowerShell, Python is an awesome language to automate things.

First steps with Python

Currently, the stabled releases are 2.7 and 3.5. I recommend to start with the 3.5 release. You can get the latest release from python.org. They offer packages for Windows, MacOS X and Linux/ UNIX. Python comes with an IDE called IDLE (Integrated Development and Learning Environment). Make sure that you take a look into the official documentation! If you want something more comfortable, try JetBrain PyCharm. JetBrains offer a free community edition for Windows, MacOS X and Linux. But it’s not the worst idea to start with IDLE. I use both IDEs, IDLE and PyCharm.

Where can you get help? YouTube is full of videos about Python. If you have a Pluralsight subscription, checkout the courses on Pluralsight. There are many good books out there, as well as some good howtos. Just use Google. It depends on what type of learner you are.

Learn the basics and try to strengthen them during a small project. Buy a Raspberry Pi. Raspberry Pi and Python are the biggest friends. If you are focused on VMware vSphere, take a closer look at the VMware vSphere API Python Bindings. Create yourself a project to learn.

I just started to learn Python, but I think that this wasn’t the worst idea in my life.