This posting is ~4 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
One of my customers bought a very nice new backup solution, which consists of a
HPE StoreOnce 5100 with ~ 144 TB usable capacity,
and a new HPE ProLiant DL380 Gen10 with Windows Server 2016
as new backup server. StoreOnce and backup server will be connected with 8 Gb Fibre-Channel and 10 GbE to the existing network and SAN. Veeam Backup & Replication 9.5 U3a is already in use, as well as VMware vSphere 6.5 Enterprise Plus. The backend storage is a HPE 3PAR 8200.
This setup allows the usage of Catalyst over Fibre-Channel together with Veeam Storage Snapshots, and this was intended to use.
The OS on the StoreOnce was up-to-date (3.16.7), Windows Server 2016 was installed using HPE Intelligent Provisioning. Afterwards, a drivers and firmware were updated using the latest SPP 2018.11 was installed. So all drivers and firmware were also up-to-date.
After doing zoning and some other configuration tasks, I installed Veeam Backup and Replication 9.5 U3, configured my Catalyst over Fibre-Channel repository. I configured a test backup… and the server failed with a Blue Screen of Death… which is pretty rare since Server 2008 R2.
geralt / pixabay.com/ Creative Commons CC0
I did some tests:
backup from 3PAR Storage Snapshots to Catalyst over FC repository – BSoD
backup without 3PAR Storage Snapshots to Catalyst over FC repository – BSoD
backup from 3PAR Storage Snapshots to Catalyst over LAN repository – works fine
backup without 3PAR Storage Snapshots to Catalyst over LAN repository – works fine
backup from 3PAR Storage Snapshots to default repository – works fine
backup without 3PAR Storage Snapshots to default repository – works fine
So the error must be caused by the usage of Catalyst over Fibre-Channel. I filed a case at HPE, uploaded gigabytes of memory dumps and heard pretty less during the next week.
HPE StoreOnce Support Matrix FTW!
After a week, I got an email from the HPE support with a question about the installed HBA driver and firmware. I told them the version number and a day later I was requested to downgrade (!) drivers and firmware.
The customer has got a SN1100Q (P9D93A & P9D94A) HBA in his backup server, and I was requested to downgrade the firmware to version 8.05.61, as well as the driver to 9.2.5.20. And with this firmware and driver version, the backup was running fine (~ 750 MB/s hroughput).
I found the HPE StoreOnce Support Matrix on the SPOCK website from HPE. The matrix confirmed the firmware and driver version requirement (click to enlarge).
Fun fact: None of the listed HBAs (except the Synergy HBAs) is supported with the latest StoreOnce G2 products.
Lessons learned
You should take a look at those support matrices – always! HPE confirmed that the first level recommendation “Have you trieed to update to the latest firmware” can cause similar problems. The fact, that the factory ships the server with the latest firmware does not make this easier.
This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
When taking a backup with Veeam Backup & Replication, a VM snapshot is created to get a consistent state of the VM. The snapshot is taken prior the backup, and it is removed after the successful backup of the VM. The snapshot grows during its lifetime, and you should keep in mind, that you need some free space in the datastore for snapshots. This can be a problem, especially in case of multiple VM backups at a time, and if the VMs share the same datastore.
Benefit of storage snapshots
If your underlying storage supports the creation of storage snapshots, Veeam offers an additional way to create a consistent state of the VMs. In this case, a storage snapshot is taken, which is presented to the backup proxy, and is then used to backup the data. As you can see: No VM snapshot is taken.
Now one more thing: If you have a replication or synchronous mirror between two storage systems, Veeam can do this operation on the secondary array. This is pretty cool, because it takes load from you primary storage!
Backup from a secondary HPE 3PAR StoreServ array
Last week I was able to try something new: Backup from a secondary HPE 3PAR StoreServ array. A customer has two HPE 3PAR StoreServ 8200 in a Peer Persistence setup, a HPE StoreOnce, and a physical Veeam backup server, which also acts as Veeam proxy. Everything is attached to a pretty nice 16 Gb dual Fabric SAN. The customer uses Veeam Backup & Replication 9.5 U3a. The data was taken from the secondary 3PAR StoreServ and it was pushed via FC into a Catalyst Store on a StoreOnce. Using the Catalyst API allows my customer to use Synthetic Full backups, because the creation is offloaded to StoreOnce. This setup is dramatically faster and better than the prior solution based on MicroFocus Data Protector. Okay, this last backup solution was designed to another time with other priorities and requirements. it was a perfect fit at the time it was designed.
This blog post from Veeam pointed me to this new feature: Backup from a secondary HPE 3PAR StoreServ array. Until I found this post, it was planned to use “traditional” storage snapshots, taken from the primary 3PAR StoreServ.
With this feature enabled, Veeam takes the snapshot on the 3PAR StoreServ, that is hosting the synchronous mirrored virtual volume. This graphic was created by Veeam and shows the backup workflow.
Veeam/ Backup process/ Copyright by Veeam
My tests showed, that it’s blazing fast, pretty easy to setup, and it takes unnecessary load from the primary storage.
In essence, there are only three steps to do:
add both 3PARs to Veeam
add the registry value and restart the Veeam Backup Server Service
enable the usage of storage snapshots in the backup job
How to enable this feature?
To enable this feature, you have to add a single registry value on the Veeam backup server, and afterwards restart the Veeam Backup Server service.
Location: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\
This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
The Virtual Environment Integration (VE Integration) provides protection of VMs in virtual server environments. It is used o integrate HPE Data Protector with various virtualization environments, currently VMware vSphere and Microsoft Hyper-V. For Citrix XenServer is a script solution available. I will focus on VMware vSphere.
What is possible?
I took this table from the “HPE Data Protector 9.00 Integration Guide for Virtualization”.
Feature
VE Integration
Online backup
✓
Crash-consistent backup
✓
Application-consistent backup
✓
Granularity
vmdk, vmx
Full/ Incremental/ Differential
✓/ ✓/ ✓
Support for changed block tracking (CBT)
✓
Where does the Data Protector component need to be installed?
backup host
Extra licenses needed
1x On-Line Extension per ESXi host
As you can see, Data Protector offers all you need to create a crash-consistent backup of your VMs. HPE Data Protector relies on the VMware vSphere Storage APIs – Data Protection (formerly known as VMware vStorage APIs for Data Protection or VADP). Data Protector has to use the same API as Veeam, CommVault Simpana or any other product that can be used to backup VMs in a VMware vSphere environment. Therefore, most software products offer the same features.
How does it work?
HPE Data Protector uses the vStorage Image backup method to create a crash-consistent backup of your VMs. With this method, a backup host is used to create a backup of VMs hosted on a single or multiple ESXi hosts. The backup host can be a dedicated physical host, a virtual machine, or the Cell Manager (CM) itself (physical or virtual). All you need to make sure is, that the Data Protector Virtual Environment Integration component (VEAgent) is installed. During a vStorage Image backup, the VEAgent
establishes a connection between the backup host and the ESXi or vCenter server (depending if it’s a standalone host or a vCenter environment)
locks the VM, so that it can’t be migrated off the host by VMware vMotion
requests a snapshot of the VM
reads the VM data across LAN or SAN
initializes the Media Agent (MA) and controls the transfer of the data to to backup device
After finishing the backup of the VM, the snapshot is released and the VM is unlocked. I took this picture from the “HPE Data Protector 9.00 Integration Guide for Virtualization” to illustrate the data flow and what components interact with each other.
HPE/ hpe.com
If Data Protector requests the creation of a snapshot, the snapshot is always named “_DP_VEPA_SNAP_”. I often use this simple PowerCLI one-liner to search orphaned VEAgent snapshots:
To be honest: Orphaned snapshots only occur if a VEAgent backup failes before Data Protector can delete the snapshot. So an orphaned snapshot indicates some kind of failure during the backup. The number of snapshots that remain in the snapshot chain after a backup depends on three factors:
Wheather CBT is used or not
Selected snapshot handling mode
Backup type specified
The snapshot, that remain in a snapshot chain play a great role for incremental and differential VM backups. Data Protector can detect changes on
file level, or at
block level
Without CBT, Data Protector uses snapshots to identify changes on file level. With CBT, Data Protector identifies changes on block level. With CBT, the number of snapshots remaining after a backup is always 0. Without CBT, Data Protector keeps up to 2 snapshots (mixed snapshot handling). You must not delete these snapshots. Otherwise a full backup of a VM is necessary to create a new, valid backup chain.
Even if CBT is enabled, Data Protector requests the creation of a snapshot to get a consistent state of the VM. Because of this, a VM backup requires sufficient free disk space on the datastore where the VMDKs of the VM reside. The longer a backup takes, and the more changes are made, the bigger the snapshot gets. Here comes the free space required option into play. You can specify the amount of free disk space, that must be available at the start of the backup, e.g. 10% or 20%. The required free space is calculated based on the size of VMDKs of a VM just before the snapshot is created. Data Protector checks all datastores where the virtual machine disks reside. If a VM has a 100 GB VMDK and you set the free space required option to 10%, at least 10 GB free disk space is required in each datastore, where the VM has VMDKs located. The check is per VM!
By default, VMs are backed up in parallel. This greatly improves the overall backup performance. But in rare cases it can lead to problems. You can disable parallel backups by adding
OB2_VEAGENT_THREADED_BACKUP = 0
to the omnirc on the VEAgent backup host.
By default, a maximum of 10 concurrent threads are executed when backing up VMs using the VEAgent integration. This os good for the backup performance, but it also places load on the infrastructure. You can change this by adding the OB2_VEAGENT_VCENTER_CONNECTION_LIMIT variable to the omnirc on the VEAgent backup host.
I had several cases where VEAgent backups failed because the VEAgent (vepa_bar.exe) or the Backup Media Agent (bma.exe) failed with a memory dump during the backup, or during the initial environment discovery. In all cases, the VEAgent, the MA and the CM were located on a single physical host. This is highly not recommended according to the Data Protector Support. A possible solution is to deploy a Windows Server VM and push the VEAgent onto it. You can use this VM as VEAgent backup host, and the physical host acts only as MA and CM.
With the OB2_VEAGENT_BACKUP_DISK_BUFFER_SIZE option, you can modify the buffer size used during the backup. The SAN and the HotAdd transport mode support disk buffer sizes from 1 MB to 256 MB. By default, they use 8 MB disk buffers. The NBD and NBDSSL transport are always using 1 MB. Using bigger disk buffer sizes can improve the backup performance, but it also increases the memory consumption.
On Windows VMs it is possible to use Volume Shadow Copy Service (VSS) to quiesce the states of the applications running within a virtual machine before a snapshot is created. A ZIP archive is created that contains all the BCD and writer manifests. Please note that quiescence can slow down the performance of a backup sessions considerably.
TL;DR
During my last projects, I collected a number of common or best practices. I provide this “AS IS” with no warranties! Thanks to the HPE Data Protector support team for helping me during several support cases. Special thanks to Dimitar, Jose, Zhulien and Stephen!
Use multiple, smaller jobs instead of a few, bigger jobs
You should use jobs with a maximum of 30 VMs. Try to keep the size of a backup equal, but don’t add more than 30 VMs into a single job. If a job fails, you have to restart the job for 30 VMs, not for 200 or more VMs. With more jobs, you can execute jobs in parallel.
Use different hosts as Cell Manager, Media Agent and VEAgent
You shouldn’t combine CM, MA and VEAgent on a single physical or virtual server. Try to separate at least the VEAgent backup host. You can use a VM for this.
If you had to pack all services on a single server, reduce the load
Use OB2_VEAGENT_THREADED_BACKUP, or OB2_VEAGENT_VCENTER_CONNECTION_LIMIT, and/ or reduce the number of running MAs.
Always try to utilize CBT
Whenever possible, use CBT instead of single or mixed snapshot handling.
Use SAN Transport
Whenever possible, use SAN transport. If you can utilize SAN transport, try to use a virtual VEAgent backup host. In this case Data Protector will use HotAdd transport mode.
In case of StoreOnce: Single Object per Store Media
This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
HPE Data Protector stores multiple backup objects on a single Catalyst store item. A backup object can be a volume, a mount point, a database or a virtual machine. You can have multiple backup objects per backup client. If your filesystem backup job has four backup clients, and each client has two volumes, the backup job will contain 8 backup objects. Another example is a single database of a Microsoft SQL or Oracle database server (instance).
A Catalyst store item is an object of a StoreOnce Catalyst store and stores the data of a specific backup job. If you backup multiple VMs in a single VE Integration job, the Catalyst store item will include all VMs from that specific job. Or if you backup an Exchange server with three databases, the Catalyst store item is used to store these three databases. Due to this behavior, a single Catalyst store item can reach enormous sizes. Usually this is not a problem. But if you have to copy backup objects to other media (e.g. tape), Data Protector has to read the store medium for each backup object. As the name says: The copy operation in Data Protector is based on backup objects. If there are multiple backup objects on a Catalyst store item, a backup object copy can take some time.
Since HPE Data Protector 8.1, Data Protector offers an option to store a single backup object per Catalyst store item. You can enable this option in the properties of a StoreOnce D2D device (“Settings” tab).
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
With this option, Data Protector will create a single Catalyst store item for each backup object. This option can significantly speed up object copy operations. You should consider this option in the following cases:
You have to speed up object copy operations
You have multiple large backup objects per backup client (e.g. Exchange with multiple large databases, Microsoft SQL server with multiple large databases, VMware/ Hyper-V backups, file servers with large volumes etc.)
A possible disadvantage is the increasing number of Catalyst store items, especially if you have a large number of backup clients with many small backup objects. HPE Data Protector and StoreOnce have a limit with regard to the maximum number of Catalyst store items (which isn’t publicly documented …).
This posting is ~8 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
Some days ago, I blogged about the new HP StoreOnce software release 3.13.0. This release included several fixes. One fix wasn’t mentioned by me, although it’s interesting.
Fixed issue where Windows 2012 R2 built-in native backup was not supported with 3.12.x software (BZ 61232)
Windows Server Backup (WSB) is part of Windows Server since Windows Server 2008. WSB can create bare metal backups and recover those backups. The same applies to system state backups, file level backups, Hyper-V VMs, Exchange etc. Very handy for small environmens. Backup can be stored on disk or on a file share. With Server 2012, the file share must be SMB3 capable. So if it’s not a Windows file server, the NAS that offers the file share has to be SMB3 capable. This doesn’t apply to Windows Server 2008 (R2).
With StoreOnce 3.13.0, HP has fixed this. Starting with 3.13.0, you can use a CIFS share on a StoreOnce appliance as a target for Windows Server Backup. This allows you to take advantage of the benefits of StoreOnce, like industry-leading deduplication and replication technology.
I was able to test this new feature with StoreOnce VSA appliances in my lab, as well as with a customers StoreOnce 4700 appliance.
This posting is ~8 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
Since september 2015, the latest version of HP StoreOnce backup system software is available. The latest release 3.13.0 is available for HP StoreOnce VSA, 6500, B6200 multi-node and all single node systems running software version 3.x. This also applies to some D2D 2500, 4100 and 4300 single-node backup systems running software versions 2.x. Make sure that you take a look into customer notice c03729283 for details on performing the conversion.
This release comes with some nice enhancements, e.g.
support for jumbo frames
1 TB, 5 TB and 10 TB thin provisioned disks are now supported for the VSA
VSA Hyper-V PowerShell Installer script
HP StoreOnce VSA Ubuntu KVM bash Installer
Single Entitlement page on the HP StoreOnce GUI
Two fixes caught my attention, because I saw both of them in the wild:
Fixed issue where running NAS Replication data jobs are cancelled when files on target share are simultaneously accessed (BZ 63232)
I saw this error in environments, where customers used StoreOnce CIFS shares as backup target with Veeam Endpoint Backup together with StoreOnce Replication.
Fixed issue where NAS CIFS shares are inaccessible when special characters are in the share description (BZ 62263)
I strongly recommend to update to 3.13.0! You can download the software from the HP Software Depot (make sure that you download the right StoreOnce software for your HW appliance or VSA!). All you need is a HP Passport login. To update the software, upload the RPM into the repository folder on the appliance with SFTP. Login using SSH and execute three simple commands.
# system show repository
File
----
HP_StoreOnce_2700_2900_4500_4700_4900_System_Software_v3.13.0_EJ022-10542.rpm
# system load package HP_StoreOnce_2700_2900_4500_4700_4900_System_Software_v3.13.0_EJ022-10542.rpm
Command Successful
# system update packages
Initializing.
Verifying staged package repository integrity.
Package stage area verification successful.
Updating from version '3.12.1-1513.3' to '3.13.0-1529.2'.
Reboot will occur after update completes.
Installing update tool packages. Can take 5 minutes to complete.
Installing: D2D_UpdateTools
Install successful
Update tool package installation successful. 0 minutes elapsed.
Updating from version '3.12.1-1513.3' to '3.13.0-1529.2'.
Reboot will occur after update completes.
Verifying status of hardware and services.
Hardware and services status verification successful.
Stopping service sets. Can take 30 minutes to complete.
Stop service sets successful. 1 minute elapsed.
Stopping file services.
File service stop successful.
Starting management service maintenance. Can take 30 minutes to complete.
Management service maintenance started successful. 0 minutes elapsed.
Stopping management services.
Management services stop successful.
Cleaning up old packages
Removing test packages
Installing packages. Can take 60 minutes to complete.
Installing: D2D_Master-3.13.0-1529.2
Processing ... -
The update takes only a few minutes. Subsequently, the appliance reboots.
Install successful
Package installation successful. 12 minutes elapsed.
Removing update tools packages.
Start reboot of the system.
Processing ... /
Broadcast message from root (Thu Oct 29 11:10:13 2015):
System rebooting to complete the update process.
The system is going down for reboot NOW!
Update will complete after rebooting.
Ssh connection will be lost. Allow 15-20 minutes for reboot to complete before reconnecting.
This posting is ~8 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
While I was playing with my shiny, new HP StoreOnce VSA in my lab, I noticed a curious behavior. I created a NAS share for some tests with Veeam Backup & Replication. Creating a new share is nothing fancy. You can create a share in two ways:
using the GUI, or
using the CLI
So I created a new share:
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Nothing special, as you can see. I opened up a Explorer, typed in the IP address of my StoreOnce VSA and… saw no share.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
I repeated this process a couple of times, always with the same result. Then I went to the CLI and checked the newly created share:
# nas show share share0
ID 0
Protocol CIFS
Name veeam
Network Name \veeam
Description Share for Veeam B&R
Created On 11:35 2015/05/27
Modified On 11:35 2015/05/27
Status Online
Health OK
Replication Role Non Replicating
Replication Status Non Replicating
Replication Health Information
User Bytes 0
Disk Bytes 380
Dedupe Ratio 0.0
Files 0
Directories 1
Read-only No
Encryption Enabled No
Deduplication Enabled Yes
Share Version 2
Authentication None
Physical Data Size Quota (No Quota)
Logical Data Size Quota (No Quota)
Command Successful
So far, so good. I removed the share and tried to create the share using the CLI:
# nas create share set1 share0 proto cifs name "veeam" desc "B&R"
Command Failed -
The command failed, no share was created. I verified the syntax, but the syntax of the command was correct. I started to simplify the command and removed the description.
# nas create share set1 share0 proto cifs name "veeam"
Command Successful
# nas show share share0
ID 0
Protocol CIFS
Name veeam
Network Name \veeam
Description HP StoreOnce Share 1
Created On 11:43 2015/05/27
Modified On 11:43 2015/05/27
Status Starting
Health Information
Replication Role Non Replicating
Replication Status Non Replicating
Replication Health Information
User Bytes 0
Disk Bytes 380
Dedupe Ratio 0.0
Files 0
Directories 1
Read-only No
Encryption Enabled No
Deduplication Enabled Yes
Share Version 2
Authentication None
Physical Data Size Quota (No Quota)
Logical Data Size Quota (No Quota)
Command Successful
The share was added with the default description. I removed the share and tried it again with my description. The command failed again. After removing the ampersand (&) from the description, the share could be added. I tried the same from the GUI. Using the GUI, a share with a ampersand (&) in the description field could be added, but it wasn’t accessible. Even if I removed the ampersand (&) from the share description. I had to remove and re-create the share with a valid description. Unfortunately the GUI allows you to create the share, even if the CLI command fails with the same settings. The GUI also doesn’t allow you to create the share with an empty description.
At this point, I can’t say if this is a bug or a known behaviour. I’m in contact with HP to clarify this. But you should avoid the usage of special characters in the NAS share description.
EDIT
Today, I got an e-mail from the HP StoreOnce Engineering. They informed me, that it’s not only the ampersand (&) you should avoid. You should avoid a set of special characters
`
*
&
%
+
multiple space in a row
These characters can cause minor issues with Windows tools, like the Explorer. As a result, these special characters were banned in the latest 3.12.x CIFS server code. However this ban was not messaged in the GUI. As a fix, this ban will be lifted from 3.12.2 software to allow the use of the above mentioned special characters.
This posting is ~8 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
When talking to SMB customers, most of them don’t want to talk about their backup strategy. It’s paradox: They know that data loss can ruin their business, but they don’t want to invest money into a fully tested recovery concept (I try to avoid the word “backup concept” – Recovery is the key). Because of tight budgets and lacking knowledge, many customers use traditional concepts in a virtualized world. This often ends in traditional backup applications with agents deployed into guest OS, and backups that are written to tape (or worse: On USB disks). If you ask a customer “Why do you store your data on tape?”, only a few argue with costs per GB or performance. Most the customer argue with something like
“We’re doing this for years, so why we should change it?”
“We have to store our tapes offsite”
“There is a corporate policy that forces us to store our backups on tape”
In most cases, the attempt to sell a backup-to-disk appliance (like HP StoreOnce backup system) dies with the last arguments. Customers tend to doesn’t trust designs in which they don’t have a backup on tape. Some customers have a strong desire to have a tape which is labled with “MONDAY” or “FRIDAY FULL”. To be honest: Usually I see this behaviour only at SMB customers. Backup-to-disk appliances are often described as
expensive,
complex, and
vulnerable
None of them applies to a HP StoreOnce backup system. Not even expensive, if you not only focus on CAPEX.
HP StoreOnce
Please allow me to write some sentences about HP StoreOnce.
A HP StoreOnce backup system is available as physical or virtual appliance. HP offers a broad range of physical appliances that can store between 5,5 TB and 1.728 TB BEFORE deduplication. The virtual StoreOnce VSA is available with a capacity of 4 TB, 10 TB and 50 TB before deduplication. And don’t forget the free 1 TB StoreOnce VSA! All HP StoreOnce backup systems, regardless if physical appliance or VSA, share the same StoreOnce deduplication technology, as well as the same replication and security features. In fact, the StoreOnce VSA runs the same (linux based) software as the physical applanices and vice versa. You can add features by adding software options:
HP StoreOnce Catalyst
HP StoreOnce Replication
HP StoreOnce Security Pack
HP StoreOnce Enterprise Manager
HP StoreOnce Catalyst allow the seamless movement of deduplicated data across StoreOnce capable devices. This means, that a HP Data Protector media agent can deduplicate data during a backup, write the data to a HP StoreOnce backup system, and then the data can replicated to another HP StoreOnce backup system. All without the need to rehydrate on the source, and deduplicate it on the destionation again. The StoreOnce VSA includes a HP StoreOnce Catalyst license!
HP StoreOnce Replication enables an appliance or a VSA to act as a target in a replication relationship. Only the target needs to be licensed. Fan-in describes the number of possible source appliances.
Model
Fan-in
StoreOnce VSA
8
StoreOnce 2700
8
StoreOnce 2900
24
StoreOnce 4500
24
StoreOnce 4700
50
StoreOnce 4900
50
StoreOnce 6200
384
As you can see, even the StoreOnce VSA can used as a target for up to 8 source appliances. Replication is a licensable feature, except for the StoreOnce VSA. The StoreOnce VSA includes the replication license!
HP StoreOnce Enterprise Manager can be obtained for free and allows you to monitor up to 400 physical appliances or StoreOnce VSAs. It provides monitoring, reporting, trend analysis and forcasting. It integrates with the StoreOnce GUI for single pane-of-glass management for physical appliances and VSA.
HP StoreOnce Security Pack enables data-at-rest and data-in-flight encryption (using IPsec and only for StoreOnce Catalyst), as well as secure data deletion. Here applies the same as for the HP StoreOnce Catalyst and Replication license: The StoreOnce VSA includes this license already.
HP StoreOnce Deduplication
Deduplication is nothing really new. In simple terms it’s a technique to reduce the amount of stored data by removing redundancies. Data that is being detected as redundant, isn’t stored again on the disks. Only a pointer to the stored data is set. This runs the risk of potential data loss. What if the original block gets corrupted? Grist to the mill of the tape lovers (Tapes never fail… for sure…).
Integrity Plus
Don’t worry. I won’t bore you with stuff about a dead (or nearly dead) CPU architecture. Integrity Plus is HPs approach for an end-to-end verification process. Let’s take a look on how data comes into a StoreOnce backup system. From a client perspective, you can choose between Virtual Tape Library (VTL), NAS emulation (CIFS or NFS) and StoreOnce Catalyst.
When data is written to a VTL, a CRC is computed for each block and it’s stored together with the data block on disk. During a restore, a CRC is computed for every block that is read from disk and it’s compared to the initial stored CRC. If it differs, a SCSI check condition is reported. Because NAS emulation and StoreOnce Catalyst doesn’t use SCSI protocol, no CRC is computed and stored to disk. The integrity of the written data is guaranteed in other ways.
At the beginning of the deduplication process, the incoming data is divided into chunks. HP uses a variable length for each data chunk, but in average a data chunk is 4 KB. A smaller chunk size leads to better deduplication results. A SHA-1 (AFAIK 160 bit) hash is computed for each data chunk. This chunk hash is used to identify duplicate data by comparing it to other chunk hashes. At this point, a sparse index is used to find possible candidates of redundant data chunks. Instead of holding all chunk hashes in the memory, only a few hashes are stored in the RAM. The remaining chunk hashes are stored as metadata on disk. The container index contains a list of chunk hashes and a pointer to the data container where the data chunk is stored. Before data chunks are stored on disk, multiple chunks are compressed (using LZO) and a SHA-1 checksum is computed for the compressed chunks. This checksum is stored on disk. When the compressed data is decompressed, a new checksum is computed and it’s compared to the stored SHA-1 checksum. Metadata and container index files are protected with MD5 checksums. In addition, a transaction log file is maintained for the whole process and the sparse index is frequently flushed to disk.
When data is coming into the StoreOnce backup system, a match with a chunk hash in the memory can lead the system (using the sparse index, metadata and container index files) to containers with associated data chunk (e.g. data chunks that represent a backup VM). And if a data chunk of the incoming data is a duplicate, it is very likely that many of the following data chunks are also duplicates.
All physical appliances use RAID 6 to protect data in case of disk failures. Only the HP StoreOnce 2700 uses a RAID 5, because the appliance can only hold 4 SAS-NL disks. When using StoreOnce VSA, you can use any RAID level for the underlying storage. But you should use something above RAID 0…
Conclusion
Let’s summarize:
RAID
Supercapacitors on RAID controllers to protect write cache in case of power loss
ECC memory
Integrity Plus to protect the data within the StoreOnce backup system
StoreOnce Replication to replicate data to another HP StoreOnce backup systems
data-at-rest, data-in-flight encryption and secure deletion with StoreOnce Security Pack
Sounds very safe to me. Tape isn’t dead. Tape has its right to exist. But backup to tape isn’t safer than a backup to a StoreOnce backup system. Latter can offer you faster backups AND restores, new backup and recovery options (e.g. backups in RoBo offices that are replicated to the central datacenter). Think about the requirements for storing tapes (temperature, humidity, physical access), regular recovery tests, copy tapes to newer tapes etc. Consider not only CAPEX. Also remember OPEX.
A HP StoreOnce backup system is perfect for SMBs. It simplifies backup and recovery and it can offer new opportunities. Testdrive it using the free 1 TB StoreOnce VSA! Remember: The StoreOnce VSA includes StoreOnce Replication, Catalyst and the Security Pack! Even the free 1 TB StoreOnce VSA.
This posting is ~8 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
A free StoreOnce VSA, like the well known 1 TB StoreVirtual VSA? That would be too cool to be real. But it is real! Since February, HP offers a free 1 TB version of their StoreOnce VSA. I totally missed this announcement, but thanks to Calvin Zito I noticed it today:
Can you protect your data for free? Introducing the new free 1TB StoreOnce VSA http://t.co/71464n0iZp
— Storage Godfather (HPEStorageGuy) (@CalvinZito) April 19, 2015
HP StoreOnce VSA runs with the same software as the hardware-based StoreOnce appliances, but it’s delivered as a VM. You can run the VM on top of VMware ESXi, Microsoft Hyper-V or KVM. Beside the free 1 TB license, the StoreOnce VSA can purchased with 4 TB, 10 TB or 50 TB capacity (usable, non-deduplicated). In contrast to the hardware-based appliances, the StoreOnce VSA comes with licenses for replication and StoreOnce Catalyst. This makes the StoreOnce VSA a perfect fit for remote and branch offices. You can quickly deploy the StoreOnce VSA and replicate the backuped data to the central datacenter. But you can also deploy the VSA with the 4 TB, 10 TB or 50 TB license in your central datacenter and use it as a replication target for StoreOnce VSAs in the remote and branch offices (the replication target needs the replication license). A single VSA can act as replication target for up to 8 StoreOnce VSA and/ or StoreOnce appliances. You can scale the free 1 TB license with license upgrades to 4 TB, 10 TB and 50 TB. The StoreOnce VSA supports Catalyst, VTL (iSCSI) and as NAS (CIFS or NFS) backup targets. Take a look into the QuickSpecs for more information. I also recommend to read the two blog posts from Ashwin Shetty on Around the Storage Block:
Last year I’ve published several posts about the StoreOnce VSA. I recommend to download the free 1 TB StoreOnce VSA and to play with it. Some of my blog posts should help you get started.
This posting is ~9 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.
HP StoreOnce Catalyst is an enterprise-wide deduplication algorithm which is used in HP StoreOnce appliances, HP StoreOnce VSA and HP Data Protector. With StoreOnce Catalyst, deduplicated data can be moved between Catalyst capable devices without the need to rehydrate the data. Think about the backup in your remote location. You can move the deduplicated data to your central StoreOnce appliance in your HQ without rehydrating the data. This saves bandwidth and time. Or think about source-side deduplication, which means that a server deduplicates the data during the backup and sends the deduplicated data to the backup device. This saves bandwidth and increases the performance (with the downside of more cpu load on the server…).
I would like to show you how you can configure StoreOnce Catalyst with StoreOnce VSA and HP Data Protector. Catalyst is a licensable feature for the StoreOnce appliances, but it’s included in the StoreOnce VSA.
Configuration of a Catalyst store
The creation of a Catalyst store is quite easy. After you have logged in into the StoreOnce VSA, choose “StoreOnce Catalyst” > “Stores” from the left menu. Then click the “Create” button on the upper right.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Select a descriptive name and click “Create”
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
That’s it. Your first StoreOnce Catalyst store.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Create a StoreOnce device in HP Data Protector
Now you have to create a backup-2-disk device in HP Data Protector. I used a Data Protector 8.10 on a Windows 2008 R2 server. Open the Data Protector GUI and add a new device.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Enter a descriptive name and select “Backup To Disk” as device type. Interface type has to be set to “StoreOnce Backup system”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Enter the ip address or the FQDN in the field “Deduplication System”. Select the store you’ve created in the StoreOnce VSA GUI.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
When you enable the checkbox “Source-side deduplication”, a windows opens.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
This checkbox lets you create a implicit gateway. An implicit gateway is used for source-side deduplication. Using source-side deduplication allows you to deduplicate data BEFORE it is sent to the media agent or backup device. To use source-side deduplication you have to install a Data Protector media agent on each server, that should be able to do source-side deduplication.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
An implicit gateway is optional. An explicit gateway is a must. An explicit gateway can to deduplication (server-side deduplication) or can only transfer the data to the backup device (target-site deduplication). In order to create a gateway for server-side deduplication, select a server with a media agent from the drop-down menu and click “add”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Again a windows pops up.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Switch to the “Settings” tab and click “Advanced”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Enable the checkbox “Server-side deduplication” and click “OK”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click “Check”. “Server-side deduplication” should say “Yes” and “Status” OK. Click “Next”
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click “Next”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
You can confirm the message about backup size quote. Simply click “OK”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click “Finish”. Switch into the properties of your newly created StoreOnce VSA Backup-2-Disk device.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
We will now add a second explicit gateway which is used for target side deduplication. This means, that this media agent does nothing with the data. It simply forwards it to the StoreOnce VSA, which does the deduplication. Select a server from the drop-down menu and click “Add”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Switch to the “Settings” tab.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click “Advanced”
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
The checkbox for server-side deduplication must not be enabled. This is the default setting. An explicit gateway does usually target-side deduplication. Server-side deduplication has to be enabled. Click “OK”.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Click “Check”. You can see, that one media agent has server-side deduplication enabled, and the other media agent not.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
That’s it. Now you can use source-side deduplication, server-side deduplication and target-side deduplication. This is possible, because StoreOnce Catalyst is included in the Data Protector disk and media agent.
Creation of a backup job
If you create a new backup job, you can enable the “Source-side deduplication” checkbox.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
If you do so, you only can select the source-side gateway! A source-side gateway has two downsides: No object copies are possible and you have to install a media agent on each server, that should use source-side deduplication.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
If you leave this checkbox unchecked, you can select one of the explicit gateways (I highlighted the last added explicit gateway, which is configured for target-side deduplication. Server-side deduplication is done by the server, that was selected during the creation of the explicit gateway. Target-side deduplication is done by the backup device itself.
Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0
Summary
A StoreOnce Catalyst store is quickly created, no big deal. The device creation in Data Protector is a bit tricky, but as soon as you have understood the difference between source-, server- and target-side deduplication, it’s easy to implement and to use. And the results are awesome! Grab a evaluation copy of HP Data Protector and HP StoreOnce VSA from HP and try it in your lab.
To change your privacy setting, e.g. granting or withdrawing consent, click here:
Settings
