Tag Archives: software

Meltdown & Spectre: What about Microsoft Exchange?

On January 18, 2018, Microsoft has published KB4074871 which has the title “Exchange Server guidance to protect against speculative execution side-channel vulnerabilities”. As you might guess, Exchange is affected by Meltdown & Spectre – like any other software. Microsoft explains in KB4074871:

Because these are hardware-level attacks that target x64-based and x86-based processor systems, all supported versions of Microsoft Exchange Server are affected by this issue.

Like Citrix, Microsoft does not offer any updates to address this issue, because there is nothing to fix in Microsoft Exchange. Instead of this, Microsoft recommends to run the lates Exchange Server cumulative update and any required security updates. On top, Microsoft recommends to check software before it is deployed into production. If Exchange is running in a VM, Microsoft recommends to follow the instructions offered by the cloud or hypervisor vendor.

My tool chain for 2018

Each of us has his or her personal tool chain. Depending on your job role, the tool chain will look different. My personal tool chain does not have changed much over the last few years, but if I added or removed a tool to my tool chain, this change was often influenced by other peoples tool chain.

Rainer Sturm/ pixelio.de

My primary work device is a Lenovo ThinkPad X250 (Intel i5 5200U, 8 GB RAM, 250 GB SSD) with Windows 10. I’ve added a 6 cell battery, so I have ~ 95 Wh of battery capacity. This gives me ~ 16h of battery lifetime with my common workload. The 12,5″ screen seems to be small, but it’s okay as I have two 24″ displays at the office. It’s small, lightweight, long battery life and powerful. I awaited the new Lenovo Thinkpads, that were presented some day ago on the CES. But Lenovo removed the Ethernet port on the X280. So this is not longer an option. Maybe the T480 with an additional 72 Wh battery… Devices and accessories are safely stowed in an Eastpack Floid Ash Blend2. It’s a great backpack, light and not too big.

Browser and Office

I have used Google Chrome for years, but with the latest Firefox release I switched back to Firefox and disabled all Google services I used before. I even try to avoid using google.com and use duckduckgo.com instead. Microsoft Office 2013 is corporate standard, so nothing much to say about it.

Knowledge Management

I primarily use two tools to dump my brain onto my hard disk. One is Microsoft OneNote, the other one is XMind 8 Pro. I’m using OneNote to store snippets, meeting protocols, summaries etc. in two notebooks. One notebook is for customer related stuff, the other notebook is for knowhow and snippets. A third notebook is shared between colleagues and me. I often use the web version of OneNote, available on onenote.com.

Tools

Royal TS is an awesome remote management solution, helping me to keep track of all those RDP and VNC sessions. And it can do much more. I switches from PuTTY to KiTTY last year. KiTTY is a PuTTY fork with some nice additions, like folders or scripting. My VMware application stack consists of the good, old VMware vSphere C# Client (don’t judge me…), PowerCLI and the VMware vSphere Remote Console. The web-based Clients are onboard as well. Filezilla is something that I’m using for years. FTP, S/FTP oder SCP are common protocols, most times used to upload firmware, or download config files from network devices. Wireshark is another veteran in my tool chain. Nothing much to say. The army knife in case of network troubleshooting. Authy is pretty new in my tool chain. I discovered it some weeks ago as an alternative to the Google Authenticator app in my iPhone. A pretty cool app. I can have the same accounts on my smartphone and in a desktop app. No need to grab my phone if I need 2FA at my laptop. And, IMHO a big benefit, an encrypted backup of my 2FA accounts. But 2FA or MFA is only one factor. The other factor is the password and I’m forcing me to use different passwords for different services. I’m getting older, so I use Keepass to store my usernames and passwords in a safe, password protected and encrypted database.

Development

I’m not a developer. But sometimes I have to write scripts in PowerShell or Python, transform data etc. My developer tool chain is full of well known tools. Notepad++ is my favorite text editor for years. ISE Steroids is still my favorite PowerShell IDE, even if I have Visual Studio Code installed. But this is mainly used for Python. ISE Steroids variable monitoring function is superior. Currently, I don’t get my mind wrapped around the VS Code debugging mode. But I swear that I will try it in 2018! GitHub Desktop is mandatory, not only for PowerShell and Python snippets, but also for my scripts and dot files (VIM, ZSH etc.).

Other stuff

Sometimes I like to hear music during work. I love Spotify. I don’t have to run VMs on my laptop, but when I have to, VMware Workstation Pro is my desktop virtualizer of choice. For reading PDFs I switched from Adobe Reader to Google Chrome, and after removing Chrome, to Foxit Reader.

HPE Data Protector 9.08 is available

3 days ago, on 13th October 2016, HPE has released patch bundle 9,08 for Data Protector 9. A patch bundle isn’t a directly installable version, instead it’s a bundle of patches and enhancements for a specific version of Data Protector, in this case Data Protector 9.

Beside fixes for discovered problems, a patch bundle includes also enhancements. There are some enhancements in this patch bundle, that have caught my attention particularly.

QCCR2A64053: Support for object copy of file system data to Microsoft Azure. Data Protector now supports the creation of a special backup device, which can be used together with Data Protector object copies, to copy Data Protector file system backups to Azure Backup Vaults. This is an easy way to create copies of important data on Microsoft Azure.

Contemporaneous with the announcement of Data Protector 9.08, I got an e-mail of HPE with the information, that one of my change request has made it into the latest patch bundle:

QCCR2A68100: VMWARE GRE stays in debug mode. I have observed this behaviour in different Data Protector installations: If debugging isn’t explicitly disabled (OB2DBG=0 in the omnirc), the VMware GRE always writes debug logs. Regardless if debugging is enabled or disabled in the GRE configuration.

Because of some security related changes and fixes in Data Protector 9.08, HPE has marked this patch bundle as critical.

Download Data Protector patch bundle 9.08:

Data Protector 9.08 for Windows

Data Protector 9.08 for HP-UX/IA

Data Protector 9.08 for Linux/64

Get progress of ‘Shrink Database’ task on a Microsoft SQL Server

Shrinking a big database on a Microsoft SQL Server can take some time. And it’s one of those tasks, where you wont get a status until it’s finished. I really hate this… But this small T-SQL query can help:

Simply open a new query windows, paste the query into the query windows and execute the query. The query outputs the progress in percent and the estimated completion time.

PowerShell ISE on steroids

I’m not a developer. I deal mainly with infrastructe, things like virtualization, storage & backup, networking etc. Sometimes I had to write scripts, primarily PowerShell, batch or Bash. Many years back, I also wrote Csh and Ksh scripts. In the past years, automation was one of the rising trends in the infrastructure segment. And with automation, new challenges came up. Today I have to work with Windows PowerShell, in case of VMware with PowerCLI (which bases on Windows PowerShell), and sometimes I have use with REST APIs. I’m still not a developer. Due to this fact, I need tools that help me getting my work done.

So I was searching for a tool, mainly for PowerShell development, and I’ve tried some tools. Microsoft Visual Studio was to complex. Microsoft Visual Studio Code was light, but offered not the features I needed. The Windows integrated PowerShell ISE was nice, but it also lacked some features. So I asked on Twitter:

The answer was simple: ISESteroids.

What is ISESteroids?

ISESteroids is not a standalone product. It’s a PowerShell module that extents the built-in PowerShell ISE. That’s nice, because you don’t have to install anything. Simply extract it. You don’t need any special privileges to install it. Load the PowerShell module, done.

ISESteroids offers a broad feature set and transforms the PowerShell ISE into a full-featured PowerShell IDE. Visit the ISESteroids homepage for a full feature list. Nothing I want to copy & paste here.

Why is ISESteroids helpful for me?

As already mentioned: I’m not a developer. Therefore, I’m thankful for all hints and tips to make my scripts better. One of the features that I noticed immediately was the light bulb on the left side of the scripting area. The icon indicates that there is an automatic fix. In my case, this is usually converting double into single quotes.

ISESteroids_01

Another often mentioned fix is the replacement of aliases with the full command names. Another feature I really like is the risk analyzer. Sometimes you use commands and functions, that might not work with future releases, or which involve other risks. The risk analyzer is an easy way to highlight these risky commands and functions.

ISESteroids_02

Green indicates: Everything’s fine. If something risky is found, you will get a explanation why this was marked as a risky element. If you still want to use it, you can add the marked element to a whitelist. Some risks, are not a risk at all. The risk analyzer will mark the usage of the cmdlet Move-VM as a risk. This is because cmdlets with the verb “Move” will move things. IN case of Move-VM, this is intended. That’s something you can certainly whitelists.

ISESteroids_03

One of my most used cmdlets is Get-Help. Intellisense is nice, but sometimes I have to look up the correct syntax or similar. ISESteroids offers a context sensitive help. Click on the icon with the question mark,

ISESteroids_04

and you will see a new add-on tab on the left. Very handy. Click on a command, and the help will appear help add-on tab.

ISESteroids_05

You might notice another add-on tab in the picture above: Variables. This tab belongs to the Variables monitor, which can be useful to watch the content of variables. I use it frequently in conjunction with the debugging function.

ISESteroids_07

You can set breakpoints, add variables to the monitor and then watch the content of the variable.

ISESteroids_08

But you can also take a look at the current content of variables, in this case $VMhostScsiLunPaths.

ISESteroids_09

The last feature I’d like to show, is the AutoRefactor. Usually, I tend to follow best practices (mostly my own…) to make my scripts more “readable”. The AutoRefactor feature of ISESteroids helps me to make my scripts cleaner and more readable. It’s customizable, so I can tweak it where necessary. You can enable the refactor add-on tab by clicking the small icon with the check mark.

ISESteroids_06

Write down the code and click “Fix Script Now”. Then watch the magic. ;)

Why didn’t I highlight the other cool features, like code signing, file version control, keyboard shortcuts or test arguments? Because I’m still not a developer. The features I mentioned in this blog post are worthy enough to buy a PowerShell ISE license. Check the full feature list, download and install the trial version. I really recommend to take a look at the trial version! I was sceptical until I worked with ISESteroids. It was a great recommendation!

Licensing

ISESteroids is available in two commercial licenses:

  • Professional
  • Enterprise

The Professional license is available for 99 €, the Enterprise license costs 249 €. Latter offers more features. For individuals (natural persons), a discounted Enterprise license (99 €) is available. Startups, MVPs, trainers etc. can request a discounted license. Check the order website for more details.

Reset the HP iLO Administrator password with hponcfg on ESXi

Sometimes you need to reset the ILO Administrator password. Sure, you can reboot the server, press F8 and then reset the Administrator password. If you have installed a HP customized ESXi image, then there is a much better way to reset the password: HPONCFG.

Check the /opt/hp/tools directory. You will find a binary called hponcfg.

All you need is a simple XML file. You can use the VI editor or you can copy the necessary file with WinSCP to the root home directory on your ESXi host. I prefer VI.

Press i to switch to the insert mode. Then paste this content into the file. You don’t have to know the current password!

Press ESC and then :wq<ENTER> to save the file and leave the VI. Now use HPONCFG together with the XML file to reset the password.

That’s it! You can now login with “Administrator” and “password”.

Using HP StoreOnce as target for Windows Server Backup (WSB)

Some days ago, I blogged about the new HP StoreOnce software release 3.13.0. This release included several fixes. One fix wasn’t mentioned by me, although it’s interesting.

  • Fixed issue where Windows 2012 R2 built-in native backup was not supported with 3.12.x software (BZ 61232)

Windows Server Backup (WSB) is part of Windows Server since Windows Server 2008. WSB can create bare metal backups and recover those backups. The same applies to system state backups, file level backups, Hyper-V VMs, Exchange etc. Very handy for small environmens. Backup can be stored on disk or on a file share. With Server 2012, the file share must be SMB3 capable. So if it’s not a Windows file server, the NAS that offers the file share has to be SMB3 capable. This doesn’t apply to Windows Server 2008 (R2).

With StoreOnce 3.13.0, HP has fixed this. Starting with 3.13.0, you can use a CIFS share on a StoreOnce appliance as a target for Windows Server Backup. This allows you to take advantage of the benefits of StoreOnce, like industry-leading deduplication and replication technology.

I was able to test this new feature with StoreOnce VSA appliances in my lab, as well as with a customers StoreOnce 4700 appliance.

Download you free copy of the HP StoreOnce Free 1 TB VSA today and give it a try!

PernixData Architect Software

With the general availability of PernixData FVP 3.1, PernixData released the first version of PernixData Architect.

One of the biggest problems today is, that management tools are often focused on deployment and monitoring of applications or infrastructure. This doesn’t lead to a holistic view over applications and related data center infrastructure. You have to monitor at several points within the application stack and even then, you won’t get a holistic view. Without proper information, you can’t make proper decisions. At this point, PernixData Architect comes into play.

PernixData Architect is a software platform and supports the complete IT life cycle from design and deployment over operation and optimization. It supports the decision making process with data gathering and big data analytics. PernixData Architect continuously generates information and recommendations based on gathered data from VMs, storage devices, vCenter, network etc. This information pool can analysed with big data techniques. Data are gathered, data is set into context (this is what information is) and information are linked and combined with recommendations. Here are some examples what PernixData Architect can do for you (Source)

  • Descriptive Analytics – Identify and profile the top 10 VMs on latency, throughput and IOPS.
  • Predictive Analytics – Calculate server-side resources needed to run a VM in Write Through versus Write Back mode, ensuring optimal hardware is allocated before a problem arises.
  • Prescriptive Analytics – Recommend ideal server-side resources based on application patterns.

PernixData Architect is a software-only solution and can deployed with our without PernixData FVP. Without FVP, Architect can be used as a monitoring tool and gives you visibility, management and recommendations. Architect works with any server and storage platform that is compatible with VMware vSphere!

I’ve installed the latest PernixData FVP 3.1 release in my lab and enabled the 30 days trial period for PernixData Architect. You can access Architect through the web UI.

prnx_architect_1

As you can see, I have two clusters in my lab and both are accelerated using PernixData FVP. One cluster uses Distributed Fault Tolerant Memory (DFTM), the other cluster uses SSDs as acceleration ressources. If Architect is enabled, FVP doesn’t display any stats and refers to the Architect UI. Below a screenshot of the summary screen which gives you a good overview at the first glance.

prnx_architect_2

Architect includes much more stats than FVP.

prnx_architect_3

On the “Intelligence” page, you get values for the working set for each ESXi host in the cluster. This is an important value for the right sizing of your acceleration ressources.

prnx_architect_4

As mentioned, PernixData Architect uses the gathered data to give you recommendations in realtime. Even in my lab cluster,  there are things to improve. ;)

prnx_architect_5

This is only a short overview about PernixData Architect. But you might see now what insight architect can give you. If you are curious to see what PernixData FVP and Architect can do for you, you can simply install both products as part of a proof-of-concept and test them for 30 days. Even if you don’t want to install FVP, Architect can used without FVP. And even FVP can used without acceleration ressources in a monitoring mode.

Outlook license requirements for Exchange features

Microsoft Exchange Server licensing is rather simple. You can choose between two Exchange licenses:

  • Standard (up to 5 mailbox databases)
  • Enterprise (up to 100 mailbox databases)

Standard and Enterprise only differ in the number of supported databases! Feedl free to use Exchange DAG with Exchange Standard and Windows Server Standard! To license your clients, you have to purchase a Client Access License (CAL) for each user or device that accesses your Exchange server environment. There are two types of CALs:

  • Standard
  • Enterprise (add-on for Standard CAL)

The Standard CAL is always necessary and enables most features of Exchange. The Enterprise CAL is an add-on license. If a user needs one of the Enterprise CAL features, you have to purchase a Standard AND an Enterprise CAL. The Enterprise CAL enables the following features:

  • In-Place Archive
  • Retention policies
  • Apply Information Rights Management (IRM)
  • Site mailboxes
  • DLP Policy Tips

Pretty simple, isn’t it? But have you thought about your Microsoft Outlook license? To use the Exchange Enterprise CAL features, you have to consider your Microsoft Outlook licensing! You have to use a Outlook version that is supported with your specific Exchange Server version, and you also have to consider if you have retail or volume license licenses. Microsoft Exchange Enterprise CAL features can be used with the following Microsoft Outlook licenses:

Outlook 2016

  • Outlook 2016 stand-alone (Retail or Volume License)
  • Outlook 2016 included with Microsoft Office Professional Plus 2016 (Volume License)

Outlook 2013

  • Outlook 2013 stand-alone (Retail or Volume License)
  • Outlook 2013 included with Microsoft Office Professional Plus 2013 (Volume License)

Outlook 2010

  • Outlook 2010 stand-alone (Retail or Volume License)
  • Outlook 2010 included with Microsoft Office Professional Plus Subscription (Retail)
  • Outlook 2010 included with Microsoft Office Professional Plus (Volume License)

Outlook 2007

  • Outlook 2007 stand-alone (Retail or Volume License)
  • Outlook 2007 included with Microsoft Office Ultimate 2007 (Retail)
  • Outlook 2007 included with Microsoft Office Professional Plus 2007 (Volume License)
  • Outlook 2007 included with Microsoft Office Enterprise 2007 (Volume License)

The correct Outlook client license is important! If you try to use Outlook 2013 included with Microsoft Office Professional (Retail) with In-Place Archive for example, the archive will not show up in Outlook. If everything is licensed correctly, your Outlook with enabled archiving should look like this:

in-place_archive_outlook2016

Please note, that Outlook 2007 is not supported with Exchange 2016. Please also note, that the Enterprise CAL features “Site mailboxes” and “DLP Policy Tips” can only be used with Outlook 2013 and later.

 

DataCore mirrored virtual disks full recovery fails repeatedly

Last sunday a customer suffered a power outage for a few hours. Unfortunately the DataCore Storage Server in the affected datacenter weren’t shutdown and therefore it crashed. After the power was back, the Storage Server was started and the recoveries for the mirrored virtual disks started. Hours later, three mirrored virtual disks were still running full recoveries and the recovery for each of them failed repeatedly.

virtual_disk_error_ds10_mirror

The recovery ran until a specific point, failed and started again. When the recovery failed, several events were logged on the Storage Server in the other datacenter (the Storage Server that wasn’t affected from the power outage):

Source: DcsPool, Event ID: 29

Source: disk, Event ID: 7

Source: Cissesrv, Event ID: 24606

The DataCore support quickly confirmed what we already knew: We had trouble with the backend storage on the DataCore Storage Server that was serving the full recovies for the recovering Storage Server. The full recoveries ran until the point at which a non-readable block was hit. Clearly a problem with the backend storage.

Summary

To summarize this very painful situation:

  • VMFS datastore with productive VMs on DataCore mirrored virtual disks with no redundancy
  • Trouble with the backend storage on the DataCore Storage Server, that was serving the mirrored virtual disks with no redundancy

Next steps

The customer and I decided to evacuate the VMs from the three affected datastores (each mirrored virtual disks represents a VMFS datastore). To avoid more trouble, we decided to split the unhealthy mirrors. So we had three single virtual disks. After the shutdown of the VMs on the affected datastores, we started a single storage vMotions at a time to move the VMs to other datastores. This worked until the storage vMotion hit the non-readable blocks. The storage vMotions failed and the single virtual disks went also into the status “Failed”. After that, we mounted the single virtual disks from the other DataCore Storage Server (that one, that was affected from the power outage and which was running the full recoveries). We expected that the VMFS on the single virtual disks was broken, but to our suprise we were able to mount the datastores. We moved the VMs from the datastores to other datastores. This process was flawless. Just to make this clear: We were able to mount the VMFS on virtual disks, that were in the status “Full Recovery pending”. I was quite sure that there was garbage on the disks, especially if you consider, that there was a full recovery running that never finished.

The only way to remove the logical block errors is to rebuild the logical drive on the RAID controller. This means:

  • Pray for good luck
  • Break all mirrored virtual disks
  • Remove the resulting single virtual disks
  • Remove the disks from the DataCore disk pool
  • Remove the DataCore disk pool
  • Remove the logical drives on the RAID controller
  • Remove the arrays on the RAID controller
  • Replace the faulty physical disks
  • Rebuild the arrays
  • Rebuild the logical drives
  • Create a new DataCore disk pool
  • Add disks to the DataCore disk pool
  • Add mirrors to the single virtual disks
  • Wait until the full recoveries have finished
  • Treat yourself to a beer

Final words

This was very, very painful and, unfortunately, not the first time I had to do this for this customer. The customer is in close contact to the vendor of the backend storage to identify the root cause.