Challenge

Cause

Solution

Default security settings differ for each Linux OS distribution and may require additional configuration to allow connections to the required ports.

Firewall rules

Debian
In most Debian distributions, ports from 2500 to 5000 are opened by default. If connections to the Linux server fail, make sure that Linux OS does not have restrictions for this range of ports.

SLES
Use the iptables utility to allow connections to the port range 2500–5000:i

iptables -I INPUT -p tcp —dport 2500:5000 -m state —state NEW -j ACCEPT

RHEL/CentOS
Use the iptables utility to allow connections to the port range 2500–5000:

iptables -I INPUT -p tcp —dport 2500:5000 -m state —state NEW -j ACCEPT

SELinux policy

SELinux policy must be configured to allow connections to the required range of ports. By default, SELinux policy restricts such connections.

To disable SELinux policy, open the /etc/selinux/config file and disable the SELinux mode. Save the file and reboot your system.

More Information

Challenge

Solution

1. Create a new HPE StoreOnce Catalyst Store.
2. Make sure the Veeam PowerShell Module is loaded by running  the following command:

   Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue
3. Use the following cmdlet to create repositories. Run the script multiple times to create multiple repositories.

   Add-VBRBackupRepository -Name HPE-StoreOnce-01 -Folder storeonce://storeonce_fqdn:objectstore_name@/subfolder_name -Type HPStoreOnceIntegration -StoreOnceServerName storeonce_fqdn -UserName user -Password password
   
   Notes:
   
   The format above is for stores over IP (Catalyst over Ethernet). For FC connected stores (Catalyst over Fiber Channel), replace the StoreOnce name with the “COFC” identifier for FC.
   Replace HPE-StoreOnce-01 with the selected name of the Veeam backup repository
   Replace storeonce://storeonce_fqdn with the short name and FQDN of the StoreOnce appliance
   Replace objectstore_name with StoreOnce appliance name
   Replace subfolder_name with the subfolder name
   Replace storeonce_fqdn with the FQDN of the StoreOnce aplpiance
   Replace user with Admin
   Replace password with <password for Admin user>

4. Once the repositories are created, create Veeam backup jobs to use them normally.

More Information

Challenge

Cause

Solution

To improve Veeam ONE SQL database performance, partitioning can be used. Note, that SQL Server Enterprise Edition is required to use partitioning. Partitioned tables and indexes are created by two scripts. Download them here.

NOTE: Do not deploy database partitioning if you use Always On Availability Groups on the SQL server.

Before running the scripts, it is recommended to create a backup of Veeam ONE database. The scripts can be executed in any order. Before applying them, you will need to specify the following values:

1. Locate %UserName% parameter in the script and specify DB admin (sa or any domain account that has full control permissions to Veeam ONE DB).
2. Locate %Path% parameter and specify directory where database partitions will be created/stored.
3. If Veeam ONE database name is different from default, then change the name of the database (VeeamOne) in both scripts manually
4. Make sure SQL Agents are running before executing the scripts

After successful script execution you should see new functions and partition schemes: 



SQL Server agent will have two maintenance jobs created:



The directory specified for database partitions will have files like this:

VeeamOneLow_FG1.ndf
...
VeeamOneLow_FG13.ndf

VeeamOneMed_FG1.ndf
...
VeeamOneMed_FG15.ndf

More Information

Challenge

"HotAdd is not supported for this disk, failing over to network mode"

Solution

Requirements:

1. If the Veeam Proxy being used to process the source VM is on a VMFS 3 datastore it must be formatted with proper block size to be able to mount the largest virtual disk of the hot added VMs:
   • 1MB block size – 256GB maximum file size
   • 2MB block size – 512GB maximum file size
   • 4MB block size – 1024GB maximum file size
   • 8MB block size – 2048GB maximum file size
2. The Proxy being used to back up the VM must be a Virtual Machine on a host that is able to access the datastore where the source VM’s disks are stored
3. For vSphere 5.1 the maximum supported VMDK size is 1.98 TB
4. For vSphere 5.5 and older the maximum supported VMDK size is 62 TB
5. HotAdd is a SCSI feature and does not work for IDE disks
6. VMware Tools must be installed and up-to-date on the Veeam server and any Veeam proxies
7. Datastore needs sufficient space for a VM snapshot
8. Backup and restore of SATA disks in the Virtual appliance mode is supported if you use VMware vSphere 6.0 and later

Known Issues:

• If the VM disk resides on a Vvol the Hot Add Proxy VM's configuration files must also be on the same Vvol.
• HotAdd may fail if any disk was created with a newer hardware version than the VM being backed up. For example, if a disk was moved from a hardware version 8 VM to a hardware version 7 VM. To resolve, upgrade the hardware version of the VM.
• A single SCSI controller can have a maximum of 15 disks attached. To run multiple concurrent jobs with more than 15 disks, you need to add more SCSI controllers to your Veeam Proxy server that is responsible for hot adding the disks. (kb.vmware.com/kb/1037094)
• In case of standalone host connection (no vCenter added to the console), you can only hot add disks of VMs which are located on the same host as the proxy server.  This may require additional proxy servers to be configured on each host in the environment.
• HotAdd may fail if you are trying to back up the VM through the host added as a standalone server into Veeam Backup Infrastructure but actually being managed by vCenter. The following error message can be found in the source agent log:

  "Reconfigure attempt failed for VM "" with Vmomi::MethodFault::Exception ("vim.fault.HostAccessRestrictedToManagementServer")"

• If you plan to process VMs that store disks on the NFS datastore, you must deploy the backup proxy on the same ESX(i) host where these VMs reside.
• Virtual Appliance backup job fails after you’ve cloned or restored your VBR server. The reason for that is incorrect UUID reference for Veeam Backup VM which can be found in the following registry key of the Veeam server. Please see http://www.veeam.com/kb1332
• HotAdd fails if VM disks being backed up don't have unique ddb UUIDs across the vCenter, for example if such VMs were deployed from a single template to different hosts.  Please refer to the VMware KB article for details: http://kb.vmware.com/kb/2006865
• Backup in HotAdd mode may fail if there is a SATA controller configured on proxy virtual machine. Please refer to the VMware KB article for details: https://kb.vmware.com/s/article/2151091 

Troubleshooting:

• Test to see if the disk of the VM to be backed up can be manually hotadded to the proxy. See: http://www.veeam.com/kb1184
• Check all Veeam Proxies for unmounted HotAdded disks and release them manually. See: http://www.veeam.com/kb1775
• Disable Automount option on the backup proxy, as documented in http://www.veeam.com/kb1882.

More Information

Please contact Veeam Technical Support for further assistance.

Challenge

Cause

Solution

Challenge

• Your PC needs to be restarted to finish setting up this device: LSI Adapter
• Your PC needs to be restarted to finish setting up this device: VMware Paravirtual Adapter

Cause

Solution

• Add a second adapter of the same type.
• Power off the VM where the backup proxy is installed
• Relocate the boot drive to the new adapter (1:0)
• Power on the backup proxy and resume operations

• Add an LSI Logic SAS adapter
• Resume operations

Challenge

Solution

Challenge

Cause

Solution

More Information

Challenge

Cause

Last-minute kernel patches for Red Hat Enterprise Linux and its derivatives may cause Veeam Agent for Linux to go unstable (veeam.com/kb2786).
 

VAL build 3.0.1.1046

VAL build 3.0.2.1185

RHEL 7.7 and derivatives (kernel 3.10.0-1062 or higher) are ONLY supported by VAL build 3.0.2.1190.

For standalone installations you can obtain build 3.0.2.1190 from repository.veeam.com.

For instructions on how to deploy build 3.0.2.1190 on installations managed by Backup & Replication please open a support ticket.

Challenge

Cause

Solution

More Information

Challenge

[18.07.2019 15:49:24] <01> Warning TapeDrive alert: The tape drive has a problem with the application client interface
...
[18.07.2019 15:49:58] <01> Error Tape error: '1117' (The request could not be performed because of an I/O device error) (Veeam.Backup.Common.CCppComponentException)
[18.07.2019 15:49:58] <01> Error in c++: TapeHandle failed to get tape drive parameters.
[18.07.2019 15:49:58] <01> Error in c++: Tape initialization FAILED.

Cause

Solution

1. Download the hotfix.
2. Check that you have version 9.5.4.2866 of Veeam Backup & Replication installed.
3. Make sure that no jobs are running, close the Veeam console and stop all Veeam services on the backup server.
4. Stop all Veeam services on the tape server.
5. On the tape server, rename original file VeeamAgent.exe into VeeamAgent.exe_original in folders: C:\Program Files (x86)\Veeam\Backup Transport\x64 and C:\Program Files (x86)\Veeam\Backup Transport\x86.
6. Copy VeeamAgent.exe from the hotfix into folders C:\Program Files (x86)\Veeam\Backup Transport\x64 and C:\Program Files (x86)\Veeam\Backup Transport\x86 on the tape server.
7. Start Veeam services on the backup server and on the tape server.
8. Disconnect the tape drive from the power for a couple of minutes.

More Information

Challenge

Cause

Solution

More Information

Challenge

Cause

Solution

5. Repeat steps 3-4 for all the Veeam Proxies and Repositories.

6. (Optional) If you would like to disable Veeam Database, please do the following:

          6.1  Open regedit, go to HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\
and find values of the following keys:

SqlDatabaseName (default VeeamBackup)

SqlInstanceName

SqlServerName


You can use Microsoft SQL Server Management Studio to connect to the instance on the found SQL server where VeeamBackup database is located.

In this example, the instance name is VEEAM and the database name is default.

7. After performing the steps above you can be sure that all Veeam activities are stopped. Now you can perform hotfix or update installation or any other kind of measures required by a Support Engineer.

Bring Veeam Services back online

1. Start the services in the following order: Repository servers -> Proxy servers -> Veeam Server.
2. Re-enable the jobs you disabled previously:


3. Once the jobs are re-enabled, your Veeam Server will be ready to resume operations.

More Information

You can automate the stopping/starting of the Veeam Services with the following PowerShell commands:

Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Stop-Service
Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Start-Service

Once the shell prompt returns control, all Veeam Services should be started. 

Challenge

Cause

Solution

1. Depends on you version please download private fix and extract on the Veeam Backup Server. 
2. Right-click on the installation file and click "Run as administrator"
3. Go through the wizard to install private fix.
4. Relaunch Veeam Backup Console and let Veeam automatically update hosts added to the Veeam server.

More Information

Challenge

Cause

Solution

5. Repeat steps 3-4 for all the Veeam Proxies and Repositories.

6. (Optional) If you would like to disable Veeam Database, please do the following:

          6.1  Open regedit, go to HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\
and find values of the following keys:

SqlDatabaseName (default VeeamBackup)

SqlInstanceName

SqlServerName


You can use Microsoft SQL Server Management Studio to connect to the instance on the found SQL server where VeeamBackup database is located.

In this example, the instance name is VEEAM and the database name is default.

7. After performing the steps above you can be sure that all Veeam activities are stopped. Now you can perform hotfix or update installation or any other kind of measures required by a Support Engineer.

Bring Veeam Services back online

1. Start the services in the following order: Repository servers -> Proxy servers -> Veeam Server.
2. Re-enable the jobs you disabled previously:


3. Once the jobs are re-enabled, your Veeam Server will be ready to resume operations.

More Information

You can automate the stopping/starting of the Veeam Services with the following PowerShell commands:

Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Stop-Service
Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Start-Service

Once the shell prompt returns control, all Veeam Services should be started. 

Challenge

Cause

Solution

1. Depends on you version please download private fix and extract on the Veeam Backup Server. 
2. Right-click on the installation file and click "Run as administrator"
3. Go through the wizard to install private fix.
4. Relaunch Veeam Backup Console and let Veeam automatically update hosts added to the Veeam server.

More Information

Challenge

Cause

Solution

5. Repeat steps 3-4 for all the Veeam Proxies and Repositories.

6. (Optional) If you would like to disable Veeam Database, please do the following:

          6.1  Open regedit, go to HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\
and find values of the following keys:

SqlDatabaseName (default VeeamBackup)

SqlInstanceName

SqlServerName


You can use Microsoft SQL Server Management Studio to connect to the instance on the found SQL server where VeeamBackup database is located.

In this example, the instance name is VEEAM and the database name is default.

7. After performing the steps above you can be sure that all Veeam activities are stopped. Now you can perform hotfix or update installation or any other kind of measures required by a Support Engineer.

Bring Veeam Services back online

1. Start the services in the following order: Repository servers -> Proxy servers -> Veeam Server.
2. Re-enable the jobs you disabled previously:


3. Once the jobs are re-enabled, your Veeam Server will be ready to resume operations.

More Information

You can automate the stopping/starting of the Veeam Services with the following PowerShell commands:

Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Stop-Service
Get-Service | Where-Object {$_.DisplayName.StartsWith("Veeam")} | Start-Service

Once the shell prompt returns control, all Veeam Services should be started. 

Challenge

Cause

Solution

1. Depends on you version please download private fix and extract on the Veeam Backup Server. 
2. Right-click on the installation file and click "Run as administrator"
3. Go through the wizard to install private fix.
4. Relaunch Veeam Backup Console and let Veeam automatically update hosts added to the Veeam server.

More Information

Challenge

When backing up to a deduplication storage appliance from Veeam Backup & Replication, performance or deduplication ratio is low without using best practice settings. 

Solution

The default options of Veeam Backup & Replication are intended for non-deduplication storage.

If you use deduplication storage systems, you need to additionally configure the backup architecture and Veeam settings to achieve best performance. Below are some hardware and software configuration options that can be used with deduplication storages. In some cases, you may need to combine more than one option, but in all cases, it is strongly encouraged that you consult your deduplication appliance vendor and/or a value added reseller to discuss individual needs.

For the general architecture planning, Veeam’s best practices is to follow the industry 3-2-1 backup rule:

3: Maintain at least three copies of your data and applications. That’s the one copy you’re using and two backups. This way, if one of your backups is unavailable for any reason (for example, if you store the backup in the same locations with the copy you use and both go down), you can still recover what you need.

2: Store your backups on at least two different types of media or storage controller logics. One reason for this is that each type of media has its own vulnerabilities, and you don’t want both of your backups susceptible to the same problem. 

For example: run your business on a general purpose fast storage system and store your backups on a storage system that is specialized for this purpose, or a separate server with local disks.

1: Keep one of the backups in a different location. Consider a crisis in your primary data center, such as a fire or power failure. If all your copies are collocated, they can all be affected, taking down your organization.

To achieve this, use the Veeam best practices. However, each deduplication storage vendor has specific unique feature sets that allow the usage of their storages in different ways. Please check the following links for vendor specific guidelines:

• Best Practices for HPE
• Best Practices for Exagrid
• Best Practices for Dell EMC DataDomain
• Best Practices for Quantum

General Deduplication Storage Best Practices

Step 1:  Creating backups on the first site with long term retention.

Step 2: Add offsite target.

In addition to the above scenarios, you can use the following options to create an offsite copy of your data.

3. Place a deduplication storage on primary side and use object storage or tape on second site. Use Veeam backup to tape jobs or Veeam scale-out backup repository cloud tier (connection to object storages) to store data offsite.

The scenario above reflects general best practices. Please contact your deduplication storage vendor for further guidance and check the vendor links provided above for additional usable scenarios with the specific storage.
 

Best Practices For Deduplication Storage Systems

Backup Chains

• General Deduplication Storages / Exagrid / Quantum DXI + Backup Copy Job That Reads From Source Backup Chain:

 

•  HPE StoreOnce Catalyst / Dell EMC DataDomain DDboost  + Copy Over WAN + Backup Copy Job:

The previous described option provides fast processing in situations that do not involve transport over WAN links. Also, optimized integration with Catalyst and Dell EMC DDBoost allows using synthetic processing. For these environments uncheck the “Read the entire restore point from source backup instead of synthesizing it from increments” but leave the weekly GFS backup enabled.

• Special Scenarios Based on Vendor Own Guidelines + Backup Job With Active Full or Synthetic Full Backup:

Inline Deduplication

Compression

Block Size

• Storage optimization: Local target (large blocks)

• Storage optimization: Local target 

Use this option for Exagrid.

Block Alignment

Encryption

Health Check

  
Veeam Defragmentation And Compact Settings

More Information

Challenge

Product Information:

Company name: Quest Software, Inc.
Product Family: QoreStor
Status: Veeam Ready – Archive
Classification Description: Verified disk archive storage that can be used as a Backup Copy target. Synthetic full backups, granular restores, and vPower features may not provide sufficient performance or be supported.

Solution

Product Details:

Model number: ADISTEC-M
Storage Category: Software Defined Secondary Storage
Drive quantity, size, type: 6 – 3.49TB SAS SSD
Storage configuration: RAID5
Firmware version: V5.1.0 Build 377
Connection protocol and speed: CIFS, 10 GbE x 2 Bonded ALB
Additional support: All models and configurations of QoreStor ADISTEC-M with specifications equivalent or greater than the above

Veeam testing configuration:

Veeam Build Number: 9.5.4.2753

Job Settings:

Repository Settings:

Vendor recommended configuration:

Hardware Settings:

• Jumbo frames used on network configuration for testing (MTU=9000)
• Array deduplication used during testing – yes
• Array compression used during testing – yes