Backing up Hyper-V guest cluster based on VHD set

Challenge

VHD Set is a new shared Virtual Disk model for guest clusters in Windows Server 2016. VHD Set files can be included in application-consistent checkpoints and backed up, but there are some limitations.

Cause

Please make sure the following requirements are met:

• Each guest VM should have the cluster feature installed.
• All members of guest cluster should be on-line.
• VHD Set shouldn't be used as Cluster Shared Volume (CSV)
• Folder on CSV containing VHD Set files should have sufficient permissions.
  Note: See how to configure missing permission in Event 19100 section below.
• Guest operating system should be Windows 2016 or later.
• All VHD Set files must reside on CSV or SMB shares.
• Microsoft supports legacy shared VHDX on Windows 2016, but this feature cannot be used like a workaround, because it cannot be included in application-consistent checkpoints, therefore Veeam doesn't support backup of such disks on Windows Server 2016.
• If your VMs reside on a CSV, ensure ConfigStoreRootPath cluster parameter is defined.

Otherwise backup job might fail on checkpoint creation with one of the errors listed below.
 

Solution

Error code: '32768'. Failed to create checkpoint on collection 'Hyper-V Collection'
This issue occurs because Windows can't query the cluster service inside the guest VM. To fix this issue, make sure that the cluster feature is installed on all guest VMs and cluster service is running.

Error code: '32770'. Active-active access is not supported for the shared VHDX in VM group
This issue occurs because the VHDS disk is used as a Cluster Shared Volume (CSV), which cannot be used for creating checkpoints. To fix this issue, you need to use each disk as a shared disk instead of a Cluster Shared Volume. This can be done by using the "Remove from Cluster Shared Volume" option in cluster manager GUI.

Error code: '32775'. More than one VM claimed to be the owner of shared VHDX in VM group 'Hyper-V Collection'
This issue occurs because the shared drive was offline in the guest cluster. To fix this issue, make sure that all shared drives in the cluster that are part of the backup are online.

Error Event 19100 Hyper-V-VMMS 19100 'BackupVM' background disk merge failed to complete: General access denied error (0x80070005) This issue occurs because of a permission issue. To fix this issue, the folder that holds the VHDS files and their snapshot files must be modified to give the VMMS process additional permissions. To do this, follow these steps:

1. Determine the GUIDS of all VMs that use the folder. To do this, start PowerShell as administrator, and then run the following command: 

   get-vm | fl name, id

   sample output: 

   Name : BackupVM
   Id : d3599536-222a-4d6e-bb10-a6019c3f2b9b
   Name : BackupVM2
   Id : a0af7903-94b4-4a2c-b3b3-16050d5f80f

2. For each VM GUID, assign the VMMS process full control by running the following command:

   icacls <Folder with VHDS> /grant "NT VIRTUAL MACHINE\<VM GUID>:(OI)F"

   Example:

   icacls "c:\ClusterStorage\Volume1\SharedClusterDisk" /grant "NT VIRTUAL MACHINE\a0af7903-94b4-4a2c-b3b3-16050d5f80f2:(OI)F"
   icacls "c:\ClusterStorage\Volume1\SharedClusterDisk" /grant "NT VIRTUAL MACHINE\d3599536-222a-4d6e-bb10-a6019c3f2b9b:(OI)F"

       3. If after the icals permissions were corrected issue persists, place cluster nodes in different backup jobs.