Veeam Agent for Linux 6 blksnap module - Xen PV/PVHVM filesystem corruption
This article is relevant only if the answer to all three of these questions is "Yes."
- Is Veeam Agent for Linux 6 in use?
- Is Veeam Agent for Linux installed within a Xen PV/PVHVM virtual machine, specifically Citrix XenServer or XCP-ng?
- Is that virtual machine running one of these Linux distros?
- RHEL/Oracle Linux 9
- SLES 15 SP3+
- openSUSE Tumbleweed/Leap 15.3+
- Ubuntu 22.04+
- Debian 11
Challenge
After Veeam Agent for Linux completes a volume-level backup or file-level backup using a snapshot, filesystem corruption is detected.
/var/log/syslog example:
EXT4-fs error (device dm-0): ext4_lookup:1752: inode #129793: comm ls: iget: checksum invalid EXT4-fs error (device dm-0): ext4_lookup:1752: inode #129793: comm ls: iget: checksum invalid EXT4-fs warning (device dm-0): ext4_dirblock_csum_verify:400: inode #12: comm ls: No space for directory leaf checksum. Please run e2fsck -D. EXT4-fs error (device dm-0): htree_dirblock_to_tree:1042: inode #12: comm ls: Directory block failed checksum
Affected Configuration
Veeam Agent for Linux 6 is installed on a Xen PV/PVHVM virtual machine running a Linux distribution that uses blksnap kernel module by default (RHEL/Oracle Linux 9, SLES 15 SP3+,openSUSE Tumbleweed/Leap 15.3+,Ubuntu 22.04+, or Debian 11). For more information about which distributions use blksnap, refer to KB2804.
Cause
The issue is currently under investigation.
Initial results indicate that the problem is related to how ftrace, which is used by the blksnap module, is implemented on Xen PV/PVHVM machines.
Solution
This article will be updated as more information becomes available.
Workarounds
- Use Snapshot-Less File-Level Backup
- If the Linux kernel is 5.18 or lowerThe maximum Linux kernel version supported by the veeamsnap module is 5.18., contact Veeam technical support for instructions on forcing Veeam Agent for Linux to use the veeamsnap module.
More Information
Veeam Agent for Linux User Guide — System Requirements
For virtual machines, only full virtualization type is supported. Oracle VM virtual machines are supported with limitations. Virtual I/O (VirtIO) devices have experimental support status. Other containers and paravirtualized instances are not supported and may result in corruption of the source file system — for more information, see this Veeam KB article.
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.