Challenge
Veeam Agent for Microsoft Windows provides advanced capabilities of file filters. As a result, some files or folders will be
excluded to the backup or, vice versa, only those objects which are specified in the filter will be
included into the final backup point.
Cause
Underlying exclusion techniques have different specific factors which must be taken into account:
- Veeam Agent uses various algorithms, some of them are based on VSS techniques we rely on (the job triggers communication with the FilesNotToSnapshot registry key which helps to exclude specific files from a shadow copy triggered by our backup followed by exclusions those files from our backup - we read data from VSS shadow copies, so as longs as files are excluded from the shadow copy, we won't process them too).
- Those VSS exclusions are not implemented in specific job modes and we read all source data but copy only those data we specified in the backup set considering all file filters - this algorithm is implemented in the file-level job mode when individual folders are selected for the backup.
Unfortunately, techniques above have some limitations which could case some challenges with the file filters implementation.
Solution
This article describes basic file filter scenarios you may want to implement in your infrastructure.
All backup scenarios are implemented by using the file-level backup mode (it is the only backup mode that allows implementing file filters).
However, the file-level backup job may contain whole volumes and individual folders in it (it's called hybrid backup) - depending on a backup source, eventually, file filters may work differently. Scenarios below help to go through all possible points you may want to achieve.
-
Backup one folder without one subfolder
-
Exclude subfolder using environment variables
-
Exclude files from backup using mask
-
Exclude files and folders using wildcard mask in path
-
Exclude files and folders using wildcard mask in file names
-
Include files using wildcard mask in file names
-
Include only selected files in the backup
-
Include files using wildcard mask and exclude some of them using wildcard mask
-
Include files to backup which apply some convention
-
Include some folder and exclude one or several subfolders
-
Include folder and exclude subfolders using wildcard mask
-
Include folder and exclude subfolders using environment variables
-
Processing of exclusions when folder and volume are added
Backup one folder without one subfolder:
![User-added image]()
Result: The resulting backup will contain all folders and files that reside on the volume C:\ except files that reside in the C:\excltest folder.
Exclude subfoler using environment variables:
![User-added image]()
Exclusions are specified as the custom system environment variable ex1.
Result: The resulting backup will contain all folders and files that reside on the volume C:\ except files that reside in the C:\excltest folder which is defined the custom system variable ex1.
Veeam Agent for Microsoft Windows helps to implement all system environment variables as include and exclude masks, however, mind the following:
- To specify include and exclude masks, you can use only system environment variables — variables defined for the Local System account on computers added to the backup job, and cannot use user environment variables (Veeam Agent works under the NT AUTHORITY\SYSTEM account, so all exclusions are treated accordingly).
- You cannot use environment variables that contain multiple values or other environment variables to specify include and exclude masks, such setup is not supported.
Exclude files from backup using mask:
![User-added image]()
Result: The resulting backup will contain all folders and files that reside on the volume C:\ except files that reside in folders whose names start with "1".
Exclude files and folders using wildcard mask in path:
![User-added image]()
Result: Exclusions won't be applied because exclusions of folders with wildcards in the middle of the path are not supported (e.g., C:\excltest\*\2)
![User-added image]()
However, there is the following workaround for it:
To exclude recursively the files from specific subfolders of the folder the OS mechanism can be utilized, so no need to apply Veeam job exclude filters:
![User-added image]()
The registry key
HKLM:\SYSTEM\ControlSet001\Control\BackupRestore\FilesNotToSnapshot contains the list of files which are excluded from the processing during the shadow copy creation, so we may apply the OS mechanism to exclude data from a VSS shadow copy followed by exclusions those files from our backup (we read data from VSS shadow copies, so as longs as files are excluded from the shadow copy, we won't process them too).
![User-added image]()
In the example above, to exclude all files in subfolders named as “2” from C:\excltest\* the corresponding registry entries should be created. In some cases, manual adding all subfolders is not that straightforward because the number of folders is really huge.
As a workaround here is an example of the PowerShell script that upon the execution will add all such folders to the single registry value
$fold =(ls C:\excltest -Recurse -Directory).FullName | ? {$_ -like 'C:\excltest\*\2'} | % {"$_\* /s"}
New-ItemProperty -Path "HKLM:\SYSTEM\ControlSet001\Control\BackupRestore\FilesNotToSnapshot" -Name "Excl" -PropertyType MultiString -Value $fold
![User-added image]()
The script adds all folders named as "2" recursively to FilesNotToSnapshot.
As a result, the backup will contain all folders and files that reside on the volume C:\ except files that reside in folders which are defined by the mask C:\excltest\*\2.
The script above can be changed depending on your needs. Keep in mind that in case of a large number of excluded folders, MS Copy Optimization Writer, depending on available resources, may fail to exclude all data. Consider switching over to selecting only a specific folder as the backup source or thing through reducing the number of excluded folders. More details are in the article - https://www.veeam.com/kb2073
Please note that Veeam Technical Support does not provide the scripts on demand and does not troubleshoot custom scripts.
Include files using wildcard mask in file names:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all files residing in the specified folder except all files with the *.bak extension.
! C:\excltest here and below is a folder name, not a volume mount point mounted to a folder.
In case when the job is configured with the following parameters:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all data residing on volume C:\, no exclusions will be applied.
Exclude files and folders using wildcard mask in file names:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all files residing in the specified folder except all files containing the *veea* in names (*veea* folders are not affected).
Include only selected files in the backup:
Result: The resulting Veeam Agent backup will contain all data residing on volume C:\, no include masks will be applied.
You cannot specify include masks if you add an entire volume in the backup.
To specify file names and/or masks for file types that you do not want to back up proceed with the following scenario:
![User-added image]()
Result: Veeam Agent for Microsoft Windows will create a backup only for selected files. Other files will not be backed up.
Include files using wildcard mask and exclude some of them using wildcard mask:
![User-added image]()
Result: Veeam Agent for Microsoft Windows will include in the backup all files of the DOCX format that do not contain Test in their names.
Include files to backup which apply some convention:
![User-added image]()
Result: Veeam Agent for Microsoft Windows will include in the backup all files in C:\New folder except those which fit this convention -Test_?_C.txt.
![User-added image]()
Include some folder and exclude one or several subfolders:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all data residing in the specified folder C:\excltest except the folder “C:\excltest\1”.
Include folder and exclude subfolders using wildcard mask:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all data residing in the specified folder except all folders with the name starting with symbol “1”.
Include folder and exclude subfolders using environment variables:
![User-added image]()
Result: The resulting Veeam Agent backup will contain all data residing in the specified folder except all folders defined by the custom system variable ex1.
As it states above, to specify include and exclude masks, you can use only system environment variables — variables defined for the Local System account on computers added to the backup job, and cannot use user environment variables (Veeam Agent works under the NT AUTHORITY\SYSTEM account, so all exclusions are treated accordingly).
You cannot use environment variables that contain multiple values or other environment variables to specify include and exclude masks, such setup is not supported.
Processing of exclusions when folder and volume are added:
![User-added image]()
