Veeam Support Knowledge Base

Challenge

This KB article provides a list of all versions and respective build numbers for the following products:

Veeam Agent for Windows
Veeam Agent for Linux
Veeam Endpoint Backup

Solution

Veeam Agent for Windows

Version	Build Number	Release Date	Release Notes
Veeam Agent for Windows GA 3.0.1	3.0.1.1039	2019-03-26	https://www.veeam.com/veeam_agent_windows_3_0_release_notes_rn.pdf
Veeam Agent for Windows GA 3.0	3.0.0.748	2019-01-22	--
Veeam Agent for Windows GA 2.2	2.2.0.589	2018-07-03	https://www.veeam.com/veeam_agent_windows_2_2_release_notes_rn.pdf
Veeam Agent for Windows GA 2.1	2.1.0.423	2017-12-19	https://www.veeam.com/veeam_agent_windows_2_1_release_notes_rn.pdf
Veeam Agent for Windows GA 2.0	2.0.0.700	2017-05-11	https://www.veeam.com/veeam_agent_windows_2_0_release_notes_rn.pdf

Veeam Agent for Linux

Version	Build Number	Release Date	Release Notes
Veeam Agent for Linux 3.0.1 GA	3.0.1.1046	2019-03-26	https://www.veeam.com/veeam_agent_linux_3_0_release_notes_rn.pdf
Veeam Agent for Linux 3.0 GA	3.0.0.865	2019-03-26	--
Veeam Agent for Linux 2.0.1 GA	2.0.1.665	2018-07-03	https://www.veeam.com/veeam_agent_linux_2_0_release_notes_rn.pdf
Veeam Agent for Linux 2.0 GA	2.0.0.400	2017-12-21	--
Veeam Agent for Linux 1.0 U1	1.0.1.364	2017-05-22	https://www.veeam.com/veeam_agent_linux_1_0_release_notes_rn.pdf
Veeam Agent for Linux 1.0 GA	1.0.0.944	2016-12-13	--

Veeam Endpoint Backup

Version	Build Number	Release Date	Release Notes
Veeam Endpoint Backup GA 1.0	1.0.0.1954	2015-04-14	https://www.veeam.com/veeam_endpoint_free_1_0_release_notes_rn.pdf
Veeam Endpoint Backup GA 1.1	1.1.1.212	2015-10-05	https://www.veeam.com/veeam_endpoint_free_1_1_release_notes_rn.pdf
Veeam Endpoint Backup GA 1.5	1.5.0.306	2016-03-17	https://www.veeam.com/veeam_endpoint_free_1_5_release_notes_rn.pdf

Challenge

This KB article provides a list of all versions of Veeam Backup & Replication and the respective build numbers.

Solution

The following table lists Veeam Backup & Replication versions and their build numbers. Also provided is a link to the specific version release notes.

Version	Build Number	Release Date	Release Notes\Download
Veeam Backup & Replication 9.5 U4a GA	9.5.4.2753	2019-03-26	https://www.veeam.com/kb2926
Veeam Backup & Replication 9.5 U4 GA	9.5.4.2615	2019-01-22	https://www.veeam.com/kb2878
Veeam Backup & Replication 9.5 U4 RTM	9.5.4.2399	2018-12-28	--
Veeam Backup & Replication 9.5 U3a	9.5.0.1922	2018-07-02	https://www.veeam.com/kb2646
Veeam Backup & Replication 9.5 U3	9.5.0.1536	2017-12-18	https://www.veeam.com/kb2353
Veeam Backup & Replication 9.5 U2	9.5.0.1038	2017-05-12	https://www.veeam.com/kb2283
Veeam Backup & Replication 9.5 U1	9.5.0.823	2017-01-20	https://www.veeam.com/kb2222
Veeam Backup & Replication 9.5 U1 RC	9.5.0.802	--	--
Veeam Backup & Replication 9.5 GA	9.5.0.711	2016-11-16	https://www.veeam.com/veeam_backup_9_5_release_notes_rn.pdf
Veeam Backup & Replication 9.5 RTM	9.5.0.580	--	--
Veeam Backup & Replication 9.0 U2	9.0.0.1715	2016-08-05	https://www.veeam.com/kb2147
Veeam Backup & Replication 9.0 U1	9.0.0.1491	2016-03-24	https://www.veeam.com/kb2114
Veeam Backup & Replication 9.0 GA	9.0.0.902	2016-01-12	https://www.veeam.com/older-versions-download.html
Veeam Backup & Replication 9.0 RTM	9.0.0.773	--	--
Veeam Backup & Replication 8.0 U3	8.0.0.2084	2015-10-08	https://www.veeam.com/kb2068
Veeam Backup & Replication 8.0 U2b	8.0.0.2030	2015-04-28	https://www.veeam.com/kb2024
Veeam Backup & Replication 8.0 U2a	8.0.0.2029	--	--
Veeam Backup & Replication 8.0 U2 GA	8.0.0.2021	--	--
Veeam Backup & Replication 8.0 U2 RTM	8.0.0.2018	--	--
Veeam Backup & Replication 8.0 P1	8.0.0.917	2014-12-25	https://www.veeam.com/kb1982
Veeam Backup & Replication 8.0 GA	8.0.0.817	2014-11-06	https://www.veeam.com/older-versions-download.html
Veeam Backup & Replication 8.0 RTM	8.0.0.807	--	--
Veeam Backup & Replication 7.0 P4a	7.0.0.871	2014-06-05	http://www.veeam.com/kb1891
Veeam Backup & Replication 7.0 P4	7.0.0.870	--	--
Veeam Backup & Replication 7.0 P3a	7.0.0.839	2014-02-13	http://www.veeam.com/kb1854
Veeam Backup & Replication 7.0 P3	7.0.0.833	--	--
Veeam Backup & Replication 7.0 R2a	7.0.0.771	2013-11-13	http://www.veeam.com/kb1831
Veeam Backup & Replication 7.0 R2	7.0.0.764	--	--
Veeam Backup & Replication 7.0 P1	7.0.0.715	2013-09-30	http://www.veeam.com/kb1823
Veeam Backup & Replication 7.0 GA	7.0.0.690	2013-08-20	--
Veeam Backup & Replication 6.5 P3	6.5.0.144	2013-04-29	https://www.veeam.com/kb1751
Veeam Backup & Replication 6.5 P1	6.5.0.128	2012-12-24	https://www.veeam.com/kb1714
Veeam Backup & Replication 6.5 GA	6.5.0.109	2012-10-09	--
Veeam Backup & Replication 6.5 RTM	6.5.0.106	--	--
Veeam Backup & Replication 6.1 P1a	6.1.0.205	2012-08-15	https://www.veeam.com/kb1671
Veeam Backup & Replication 6.1 P1	6.1.0.204	--	--
Veeam Backup & Replication 6.1 GA	6.1.0.181	2012-06-04	--
Veeam Backup & Replication 6.0 P3	6.0.0.181	2011-12-14	http://www.veeam.com/kb1442
Veeam Backup & Replication 6.0 P2	6.0.0.164	--	--
Veeam Backup & Replication 6.0 P1	6.0.0.158	--	--
Veeam Backup & Replication 6.0 GA	6.0.0.153	2011-08-22	--
Veeam Backup & Replication 5.0 R3 Final	5.0.2.230	--	--
Veeam Backup & Replication 5.0 R3	5.0.2.224	--	--
Veeam Backup & Replication 5.0 R2	5.0.1.198	--	--
Veeam Backup & Replication 5.0 RTM	5.0.0.179	2010-08-30	--
Veeam Backup & Replication 4.1 R3	4.1.2.125	--	--
Veeam Backup & Replication 4.1 R2 P1	4.1.1.105	--	--
Veeam Backup & Replication 4.1 R2	4.1.0.96	--	--
Veeam Backup & Replication 4.0 GA	4.0.0.80	--	--

Challenge

Release information for Veeam Backup & Replication 9.5 Update 4

Note: A cumulative hotfix rollup that addresses issues reported by customers on the original build of Update 4 is now available, click here for more information https://www.veeam.com/kb2926.

Cause

Please confirm that you are running version 9.0 Update 2 (build 9.0.0.1715) or later prior to installing this update. You can check this under Help | About in Veeam Backup & Replication console. After upgrading, your build number will be 9.5.4.2615.

Solution

Please refer to What’s New and Release Notes to get detailed information on the new version.

More Information

This update has been superseded by Update 4a for Veeam Backup & Replication 9.5.

Challenge

Backup job retention processing on ReFS-based backup repositories leads to following issues:

Repository server becomes unresponsive or crashes.
I/O performance on the server is reduced significantly.
The server experiences high memory usage.

These issues can in turn lead to unexpected jobs termination.

Cause

As described in Microsoft article KB4016173 the root cause resides in the ReFS metadata management logic of the Windows OS system memory manager.

Solution

Operating System
Use the following OS versions, which include fixes to the ReFS metadata management logic:

Windows Server 2019 (requires Veeam Backup and Replication 9.5 Update 4 or later)
Windows Server 2016 patched to at least September 2018 updates (KB4343884 or later)
Windows 10 Pro for Workstations

Note: refs.sys driver version should be at least 10.0.14393.2457 or later. However, for best performance and security, we always recommend keeping all systems fully patched with the most recent Windows Updates .

Hardware
Backup repository must meet minimum system requirements. In particular, pay attention to the following in case of all-in-one backup appliances:

CPU: ensure at least 1 core per each concurrent backup proxy task AND at least 1 core for each two concurrent repository tasks are available.
RAM: ensure the repository server has enough RAM (at least the sum of RAM requirements for each individual Veeam component installed on the server).
Storage: use 64KB cluster size when formatting the ReFS volume.

3rd Party Software
Uninstall the following 3rd party software that have been reported to cause ReFS stability and/or performance issues:

Antivirus software (except Windows Defender)
Microsoft Configuration Manager Client

Note: To ensure no conflicting software is present, we recommend deploying Windows OS using the original ISO file from Microsoft (as opposed to using an image provided by the server vendor, which can potentially include bloatware).

More Information

We continue to collaborate with Microsoft ReFS development team closely on ReFS improvements. To monitor the current status, please refer to this Veeam forums discussion.

Challenge

A job fails with an error message reporting an error “The RPC server is unavailable.” The following error can occur when using Application-Aware Image Processing:

[timestamp] <01> Error    VSSControl: IsSnapshotInProgress failed. Transaction logs will not be truncated.<…>
[timestamp] <01> Error    RPC function call failed. Function name: [IsSnapshotInProgress]. Target machine: [xxx.xx.x.xx].
[timestamp] <01> Error    RPC error:The RPC server is unavailable.
[timestamp] <01> Error     Code: 1722

Cause

1. The ports that Veeam Backup & Replication is attempting to use are blocked by a firewall.
2. The VeeamVSSSupport service is stuck on the Guest OS
3. Antivirus or Antimalware software may attempt to interact with the VeeamVSSSupport service executable.

In rarer cases dynamic port allocation needs to be configured.
Note: If the VM that is failing to be connected to is Windows Server 2003, you need to use the RPC Configuration Tool (RPCCfg.exe) from the Windows Server 2003 Resource Kit to complete the process that is described in this article.

Additional troubleshooting steps can be found in the following Microsoft KB article:
http://social.technet.microsoft.com/wiki/contents/articles/4494.windows-server-troubleshooting-the-rpc-server-is-unavailable.aspx

Solution

──────────────────────────────────────────────────────────
Firewall / Ports
──────────────────────────────────────────────────────────
A full list of used ports can be found here:  https://helpcenter.veeam.com/docs/backup/vsphere/used_ports.html?ver=95u4

When Veeam Backup & Replication connects to a VM it creates firewall exclusions for the ports and processes it uses. Veeam Backup & Replication however is not able to add firewall exclusions to hardware or third-party software firewalls.

The most common ports that cause this issue when using Application-Aware Image Processing are the Dynamic RPC ports that the temporary guest agents are assigned. These ports are:
•1025 to 5000 (for Microsoft Windows 2003)
•49152 to 65535 (for Microsoft Windows 2008 and newer)

Note: The NIC’s Network Location being set to Public mode will also cause this, as Public mode sets the firewall it’s strictest settings.

──────────────────────────────────────────────────────────
VeeamVSSSupport service cleanup
──────────────────────────────────────────────────────────
When no job is processing the VM connect to it and check the following things:
1. Make sure the VeeamVSSSupport service is no longer present.
      •If it is, use the command ‘sc delete VeeamVSSSupport’ to remove the service.
2. Make sure the c:\Windows\VeeamVSSSupport folder is not present.
      •If it is, delete the folder. If it fails to delete reboot or resolve the file lock.

──────────────────────────────────────────────────────────
Antivirus/Antimalware
──────────────────────────────────────────────────────────
Please review the full list of Antivirus/Antimalware exclusions listed here: http://www.veeam.com/kb1999

The most common solution is to exclude the C:\Windows\VeeamVSSSupport\ folder from being scanned.

──────────────────────────────────────────────────────────
Dynamic Port Configuration
──────────────────────────────────────────────────────────
To configure dynamic port allocation follow the steps outlined in the following Microsoft KB article.
These changes will need to be made on the server that Veeam Backup & Replication is having trouble connecting to.

http://support.microsoft.com/kb/154596

Challenge

The upgrade to Veeam Backup & Replication U4 fails with "Database version downgrade detected ... Reboot and restart the setup". You may find following error in the C:\ProgramData\Veeam\Setup\Temp\BackupSrvLog.log:

***Veeam*** SRV: [31.01.2019 11:39:43]:<00000001> [VEEAM] Database version has been incremented to <5186>
***Veeam*** SRV: [31.01.2019 11:39:43]:<00000001> [VEEAM] Executing version command (ContentType=[Schema], Version=<5186>)
***Veeam*** SRV: [31.01.2019 11:39:43]:<00000001> [VEEAM] Adding trusted hosts (InitialCatalog=[VeeamBackup], DatabaseVersion=<5186>)
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] declare @id uniqueidentifier;
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] set @id = '52107791-e0ff-4538-a26e-4717962aef77';
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] declare @usn bigint;
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] exec IncrementUsn @usn output;
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] BEGIN
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] INSERT INTO [dbo].[Backup.Model.HostSshFingerprint]
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] (
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] [id], [network_name], [accepted_fingerprint], [current_fingerprint], [public_key], [agent_id], [network_name_hash], [last_update_time], [usn]
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] )
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] VALUES
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] (
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] @id, '<IP Address>', '', '', '', '00000000-0000-0000-0000-000000000000', '', '<Day>/<Month>/Year <time>', @usn
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] )
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] END;
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM] [SQL|ERROR] GO
***Veeam*** SRV: [31.01.2019 11:39:44]:<00000001> [VEEAM|ERROR] Conversion failed when converting date and/or time from character string. (System.Data.SqlClient.SqlException)

Cause

This is a known issue in Veeam Backup & Replication 9.5 U4. There are 2 mandatory conditions for this issue to raise:

System time format uses a.m. and p.m. (in small symbols with dots)
Application-aware processing, or guest OS file indexing is enabled for a Linux machine.

Affected regional formats and countries are:

English (New Zealand)
Greek (Greece)
Maori (New Zealand)
Spanish (Argentina)
Spanish (Bolivia)
Spanish (Costa Rica)
Spanish (Honduras)
Spanish (Nicaragua)
Spanish (Peru)
Spanish (Puerto Rico)
Spanish (Paraguay)
Spanish (El Salvador)
Spanish (Bolivarian Republic of Venezuela)
Galician (Galician)
Guarani (Paraguay)
K'iche (Guatemala)
Quechua (Bolivia)
Quechua (Peru)
Romanian (Romania)

Solution

Note: This issue is now resolved in Update 4a. Please download the new Update ISO and install Update 4a instead of Update 4.

Challenge

This article documents the path to be taken to get to the most recent version.

Solution

If you are using Cloud Connect, please consult with your provider to ensure you are on the same, or earlier, version than the provider. A tenant cannot be on a newer version than the provider.

Please see the supported versions and system requirements page in the User Guide for the most up-to-date information prior to upgrading. Links included under More Information at the bottom of this article.

Please refer to the following links for steps and more information on upgrading to 9.0 or 9.5:

Upgrade to 9.0: https://helpcenter.veeam.com/backup/vsphere/upgrade_vbr.html
Upgrade to 9.5: https://helpcenter.veeam.com/docs/backup/vsphere/upgrade_vbr.html?ver=95

The latest Veeam updates:
https://www.veeam.com/updates.html

Old versions of Veeam Backup & Replication can be found here:
https://www.veeam.com/older-versions-download.html

7.x
7.x -> 7.0.0.871 -> 9.0.0.x -> 9.5.4.2753

Install Veeam Backup & Replication 7.0 Patch 4 (7.0.0.871)
Install Veeam Backup & Replication 9.0.0.902
Install Veeam Backup & Replication 9.0 Update 2 (9.0.0.1715)
Install Veeam Backup & Replication 9.5 Update 4a 9.5.4.2753

Refer to above links for steps to upgrade.

8.x
8.x -> 8.0.0.2084 -> 9.0.0.902 -> 9.0.0.1715 -> 9.5.4.2753

Veeam Backup & Replication 8.0 Update 3: (8.0.0.2084)
Install Veeam Backup & Replication 9.0.0.902
Install Veeam Backup & Replication 9.0 Update 2 (9.0.0.1715)
Install Veeam Backup & Replication 9.5 Update 4a 9.5.4.2753

Refer to above links for steps to upgrade.

9.0.x
9.0.x > 9.0.0.1715 -> 9.5.4.2753

Install Veeam Backup & Replication 9.0 Update 2 (9.0.0.1715)
Install Veeam Backup & Replication 9.5 Update 4a9.5.4.2753

Refer to the above links for steps to upgrade.

More Information

Challenge

A job fails with an error related to NFC (Network File Copy) connectivity. For example:

Error: NFC storage connection is unavailable.

Failed to create NFC download stream.

Failed to create NFC upload stream.

User-added image

Cause

The cause of the majority of NFC errors fall in to 3 primary categories:

Port (902)
Permissions*
DNS

*If the account that Veeam Backup & Replication is using to communicate with the VMware Environment has granular permissions set please confirm all permissions are set according to the Granular Permissions Guide.

Due to the natue of NFC as a client server type connection, there is also potential that all things may be correct on the Veeam client side and there exists an issue within the VMware Infrastructure itself.

Solution

Note: When investigating this issue it is advisable to reconfigure the job to use a specific proxy. This is done to isolate an issue specific to a single proxy, as well as make it easier to identify which server the logs will found on.

Port, Permission, or DNS: Investigation

To investigate the potential of a Ports, Permissions, or DNS issue you will need to review the Agent logs.

Navigate to the following location, on the Veeam Proxy:
```
%programdata%\Veeam\Backup
```
Open the folder that matches the name of the job that is having an issue.
Within this folder you will need to find the agent log for the specific VM.
- For a Backup Job:
```
Agent.<JobName>.Source.<VMName>.log
```
- For a Replication Job:
```
Agent.<JobName>.Source.<VMName>.log
```
```
Agent.<JobName>.Target.<VMName>.log
```
Search for the following:
```
Creating NFC download stream
```
Here you will want to look for the following items that could point to the cause of the error.
1. Port
  If there is an issue with ports, there will be an error after the following:
```
nfc| Establishing connection with the host [esx1]. Port: [902]. Failed.
```
2. Permissions
  If there is an issue with permissions, there will be an error after the following:
```
nfc| Sending authd message: [SESSION ID NUMBER].
nfc| Sending authd message: [Name of Host].
nfc| Waiting for the authd reply message...
```
3. DNS
  If there is an issue with DNS, the following error will be found.
```
nfc| Resolving host name (esx1) to IP address...
nfc| Resolving host name (esx1) to IP address... Failed.
```

Port Issues

Further testing can be done using the following PowerShell command, the destination should be the hostname/FQDN/IP shown in the logs for the host.

Test-NetConnection -ComputerName "<destination_host>" -Port 902

If the Proxy is unable to reach port 902, check all applicable Firewalls between the server and the ESXi host.

Permissions Issues

When a vCenter server or standalone ESXi host is added to the [Backup Infrastructure] section of Veeam Backup & Replication, the user assigns an account to be used. That account should have sufficient privaleges to all objects related to the VM being backed up.

DNS Issues (Most Common)

The proxy must be able to resolve the hostname or FQDN of the hosts in the Virtual Infrastructure.

In some rare cases the Management Agents for the ESXi host may need to be restarted. https://kb.vmware.com/kb/1003490

More Information

The proxy being used to process a VM can be identified by opening the “Task” log specific to the VM from inside the folder named after the job in %programdata%\Veeam\Backup. Searching the Task log for the words “starting agent” will allow for the identification of which server performed the task.

Further information can be found in this forum post: http://forums.veeam.com/veeam-backup-replication-f2/failed-to-create-nfc-download-stream-t2991-15.html#p69806

The NFC connection requires the following:

DNS Resolution of target host
Port 902 is open to/from Backup Server/Proxy to ESX(i) host
Permissions to download files via ESX(i) host and/or vCenter

An issue with Port 902 may represent an issue with a firewall on the ESXi host, Veeam Proxy, or the connection between the two.

Challenge

Veeam incorrectly displays portgroups in Virtual Lab -> Settings -> Isolated Networks and when trying to add such a port group the following message box is displayed:
User-added image

Cause

It is a known VMware SDK limitation where special characters are not escaped when used in names of vSphere entities such as port groups. It might still be possible to add that port group as your production network in the Virtual Lab Properties -> isolated network settings, but the Surebackup job will not be able to match a port group with a special char to the production one resulting a Sure Backup job failure.

It's also true for Hyper-V environments, characters to avoid are:

the < is assumed to be the beginning of a tag;
the & is assumed to be the beginning a entity reference.

Solution

To resolve this issue, avoid using special characters in the vSphere and Hyper-V entities names.

Challenge

After upgrading to Veeam Backup and Replication 9.5 Update 4, you may start receiving failed protection group rescan reports saying:

No new hosts found.
[Error] Failed to process 1 host processed

Cause

This behavior is related to the introduction of an option to automatically assign licenses to all incoming standalone agents.

Solution

Currently this behavior is by design and may be changed in future releases.
As a workaround, you may go to Inventory node, right-click the Manually Added group and select Disable.

Challenge

The following error message appears when starting Veeam Backup and Replication console:
User-added image

Cause

Veeam Backup and Replication self-signed certificate expires 11 months after installation. The certificate is renewed, but Veeam Backup Service still has information about the old one in the cache, and shows the error message.

Solution

Important! Please note that this process will automatically restart the Veeam Backup Service.

Ensure no Backup/Replication Jobs or Restores are running before applying these steps.

The current workaround is manual generation of a new certificate (if a self-signed certificate is used) as described here.

If you use your own certificates, you will need to import renewed certificates:

Please note that if you do not apply the above fix, after one month you will begin to see errors with Veeam's Agent Management functionality, as well as with granular restores of any type, so it is recommended that you schedule a maintenance window at your earliest convenience to apply the fix. If you were not able to apply the fix and issues appeared, schedule a Backup & Replication server reboot to clean old certificate cache, it will also help to sort these issues out.

The issue is fixed in Veeam Backup and Replication 9.5 Update 4.

Challenge

Cause

This behavior is related to the introduction of an option to automatically assign licenses to all incoming standalone agents.

Solution

Currently this behavior is by design and may be changed in future releases.
As a workaround, you may go to Inventory node, right-click the Manually Added group and select Disable.

Challenge

Encountering issue "Error: Username or password is incorrect" when adding vCloud Director.

Cause

When adding the vCloud Director there are a few possible reasons as to why it fails.

Solution

Within the VeeamShell.log you will see the following:

VeeamShell.log:
Error Failed to login to vCloud server: 'https://IP Address:443'.
Error One or more errors occurred.
Error An error occurred while sending the request.
Error Unable to connect to the remote server
Error No connection could be made because the target machine actively refused it <Public Address>:443
Error Failed to connect to vCloud Director server https://IP Address/.
Error Username or password is incorrect.

When a job fails with the above error you will need to perform the following steps to resolve this issue:

Verify that the correct password is being utilized.
Ports, Permissions, or DNS can cause this failure:

a.)   Ensure connectivity between Veeam server and vCloud Director address.
b.) Confirm DNS is correctly configured - FQDN in Public URL must be resolvable from Veeam server
c.)   Verify ports between Veeam server and vCloud Director are open
d.)   Validate that the credentials being leveraged have the correct permissions

Additional Information:
The VeeamShell.log may be as follows:

Error Failed to login to vCloud server: 'https://IP Address:443'.
Error The given header was not found.
Error Failed to connect to vCloud Director server https://IP Address/.
Error Username or password is incorrect.

It is possible that your public address for the vCloud Directory is incorrectly configured, you can verify these settings by following the steps within the following KB article provided by VMware.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2053817

View ‘Resolution’ for steps to verify Public Addresses.

More Information

Additional Information: If your error message is not represented in the same format as above, but still shows ‘Username or password is incorrect’, it is recommend to start with a reboot of the Veeam server and vCloud Director cells.

Challenge

Product Information:

Company name: Western Digital
Product Family: Ultrastar Data
Status: Veeam Ready - Repository
Classification Description: Verified backup storage that supports all Veeam backup and restore features.

Solution

Product Details:

Model number: Data60
Storage Category: Hybrid JBOD
Drive quantity, size, type: 60 x 6TB SAS drives
Storage configuration: 24-disk striped volume (vSphere PCI passthrough to Windows VM)
Firmware version: 2020-023
Connection protocol and speed: 12Gb/s
Additional support: All models and configurations of Ultrastar Data with specifications equivalent or greater than the above

General product family overview:

The Ultrastar Data60 and Data102 are fully redundant hybrid 4U JBOD disk enclosures. The enclose have dual IO modules with 6 Mini-SAS HD ports per module. The Data60 and Data102 can be populated with up to 60 or 102 SAS/SATA drives respectively. Each enclosure will support up to 24 SAS or SATA SSDs making them a high-performance hybrid storage platform.

Veeam testing configuration:

Note: The following settings were used by the vendor to meet Veeam Ready testing requirements and should not be considered best practices. Additional changes or settings may be needed to meet the storage efficiency or performance needs for each environment. For each setting, reference links are provided for further clarification.

Veeam Build Number: 9.5.4.2615

Job Settings:

Deduplication:	Enabled (Default)
Compression:	Optimal (Default)
Storage Optimization:	Local Target (Default)

Repository Settings:

Repository Type:	Windows
Align backup file blocks:	Disabled (Default)
Decompress before storing:	Disabled (Default)
Per-VM Backup Files:	Enabled

Challenge

Product Information:

Company name: ExaGrid
Product Family: ExaGrid with Veeam Accelerated Data Mover
Status: Veeam Ready - Repository
Classification Description: Integrated storage where joint development activities between the manufacturer and Veeam have occurred to create advanced backup or restore functionalities.

Solution

Product Details:

Firmware version: 4.7, 4.8
Additional support: All EX Series Models

General product family overview:

ExaGrid understands that deduplication is required, but how you implement it changes everything in backup. ExaGrid has a unique landing zone where backups can land straight to disk without any inline processing. Backups are fast and the backup window is short. Deduplication and offsite replication occur in parallel with the backups. Deduplication and replication never impede the backup process as they always are second order priority. ExaGrid calls this “adaptive deduplication.”

Veeam testing configuration:

Veeam Build Number: 9.0.0.902

Job Settings:

Deduplication:	Enabled (Default)
Compression:	Dedupe Friendly
Storage Optimization:	Local Target (Default)

Repository Settings:

Repository Type:	Deduplicating Storage Appliance
Align backup file blocks:	Disabled (Default)
Decompress before storing:	Disabled (Default)
Per-VM Backup Files:	Enabled

Special Considerations:

Scale-out Backup Repositories supported on ExaGrid EX series appliances starting from Exagrid Software version 5.0

Challenge

Product Information:

Solution

Product Details:

General product family overview:

Veeam testing configuration:

Veeam Build Number: 9.5.4.2615

Job Settings:

Deduplication:	Enabled (Default)
Compression:	Optimal (Default)
Storage Optimization:	Local Target (Default)

Repository Settings:

Repository Type:	Windows
Align backup file blocks:	Disabled (Default)
Decompress before storing:	Disabled (Default)
Per-VM Backup Files:	Enabled

Challenge

Veeam Recovery Media Creation process is failing. The error will typically be:

Windows recovery image file not found:
User-added image

Cause

Veeam Recovery Environment is built off the Windows Recovery Environment, and the Recovery Environment is made from the Windows Preinstallation Environment.
If a system is missing the Windows PE/RE components, Veeam Recovery Media will fail to create.

Solution

To resolve this issue, we will need to Install the Windows Assessment and Deployment Kit (ADK) (Windows 8 and later), also known as Windows Automated Installation Kit (AIK) for Windows 7. The Server OS versions will use the same Kits as their matching Desktop OS. In this article, we will cover the installation for Windows 10 and Windows 7, as AIK and ADK have different installation steps.

For the first step, you will want to download the appropriate Kit from this link.

For Windows 10 1809 and later: Download the WinPE add-on for the ADK which is available here - https://docs.microsoft.com/en-us/windows-hardware/get-started/adk-install

Windows 10 - ADK

0. Install deployment tools from the ADK package
User-added image

1. Start the installation from the WinPE addon

User-added image

2. Specify Location. You do not need to change the default path. Click on Next:
User-added image

3. Select if you want to join the Microsoft CEIP and Click Next:
User-added image

4. Accept the License Agreement
User-added image

5. Proceed the installation process
User-added image

6. Once the install is completed, click Close
User-added image

7. Proceed to “Setting Registry Values”

Windows 7 - AIK

For Windows 7 (and Server 2008 R2), Microsoft had the Windows AIK, instead of the ADK.
The installation for the AIK is different, and we will be walking through these differences in this section.

1. AIK is provided as an ISO. To begin the install you will need to first mount the ISO file.
2. Once the ISO is mounted, you will be able to start the installer
User-added image

3. You will need to select the “Windows AIK Setup” option to start the install
User-added image

4. Click Next on the Welcome screen for the install
User-added image

5. Accept the License Agreement and click Next
User-added image

6. Specify Location. You do not need to change the default path. Click on Next:
User-added image

7. Click Next to confirm the Install.
User-added image

8. Click "Close" to finish the install
User-added image

9. Proceed to “Setting Registry Values”

Setting Registry Values

ForceUseAdkForRecoveryMedia

This key Forces Veeam to only use the ADK(or AIK) component.
It is the same key name for both kits.
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

Name: ForceUseAdkForRecoveryMedia
Type: REG_DWORD
Value: 1

4. Restart the “Veeam Agent for Microsoft Windows” service

DismPath

This key is used to specify the path toward the DISM tool, which is used to mount the Windows image for setup. This key will only need to be used if the default location for the dism.exe is changed.
The default location is typically: C:\Windows\System32

The dism.exe tool is included within the installation folder path of ADK(AIK), and there will be separate dism.exe(s) for various CPU Architectures. You will want to use Windows Explorer to locate the appropriate dism.exe for your CPU Architecture.

For example, on a 64-bit Windows 10 Install, the dism.exe will be located under:
C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\amd64\DISM\Dism.exe

To add this key, you will need to perform these steps:
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

Name: DismPath
Type: String
Value: Full path to your EXE, including the dism.exe

EXAMPLE: C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\amd64\DISM\Dism.exe

4. Restart the “Veeam Agent for Microsoft Windows” service

More Information

Starting with Windows 10, version 1809, WinPE is an add-on to the Windows Assessment and Deployment Kit (ADK).

Challenge

To configure Linux guests and repositories to work with Veeam Backup and Replication and utilize our software’s features.

Cause

There are several requirements that must be met, by a Linux OS, in order for us to effectively utilize all of our features and make sure they run consistently. On top of that we require root permissions to launch several modules. If these settings are not in place, then we won’t be able to properly write to and recover the system.

Solution

Authentication Methods

Linux servers registered with Veeam Backup & Replication infrastructure must use Password or Certificate-based authentication methods. At this time we do not support any other forms of authentication like biometrics, smart cards, etc.

Bash

We require Bash to be installed as shell on the Linux environment. At the time we do not support any other command shell. This is usually the default shell set in place by most Linux distributions.

"sudo"

Important Note: accounts using "sudo" must be in the "/etc/sudoers" file and have the ‘nopasswd’ setting put into the "/etc/sudoers" file.

In order to launch our script we require the "sudo" command to be able to launch from the credentials we are given at the time of setup. "sudo" is a command that allows us to make administrator level (root) changes and executions in the Linux environment.

One of the primary reasons that we use "sudo" is to make sure that we can access the /tmp folder and launch a script from it. There are many other ways you can launch scripts from the "/tmp" folder but "sudo" is the most consistent way across multiple distribution’s to do it. This also allows us to complete the operation without having change user permissions so the user doesn’t have to modify them afterword’s to get them back to their original configuration, as well as we will be able to relinquish root privileges on the fly.

Perl

Many Linux distributions come with Perl pre-installed but this does not mean that it has all the Perl modules that are required for us to utilize the server in a backup and restore function and once all of the correct modules are install we must be able to launch these modules as needed. Some systems are setup with software called SELinux. This software will make it so that only certain modules can be launched from SSH. The Linux distribution must allow us to launch all of these modules from "SSH" at any amount at a time.

Here is a list of the modules that we require for Veeam to work properly with a Linux install:

Constant
Carp
Cwd
Data::Dumper
Encode
Encode::Alias
Encode::Config
Encode::Encoding
Encode::MIME::Name
Exporter
Exporter::Heavy
File::Path
File::Spec
File::Spec::Unix
File::Temp
List::Util
Scalar::Util
SOAP::Lite
Socket
Storable
Threads

You can find out which modules are installed by running:

perldoc -q installed

One of the most widely used Linux distribution’s currently is CentOS 6/7 and in this distribution the "Data::Dumper Perl" module is not installed by default.

SSH

SSH is a program for logging into a remote machine and for executing commands on a remote machine. This is software that we use to log in to a Linux environment. Most systems have this software installed by default but if the Veeam server is having issues logging into a Linux environment you should check if SSH is installed.

The configuration of your SSH install must allow using tunneled clear text passwords. Info on how to do this can be found on this KB: http://www.veeam.com/kb1512

SCP

SCP copies files between hosts on a network. It uses SSH for data transfer, and uses the same authentication and provides the same security as SSH. We use this software as our file mover on Linux. It is common for Centos core to not include this SCP module in the default install.

"mlocate", "gzip" and "tar"

"mlocate" is a new implementation of locate, a tool to find files anywhere in the file system based on their name, using a fixed pattern or a regular expression. We utilize "mlocate" to locate files on a Linux system.

"gzip" and "tar" are archive tools very much like WinZip. We use these to compress and decompress files going to and from the server.

All of these files and their required configurations must be installed on the Linux environment prior to its usage with Veeam Backup and Replication to fully use all of our features.

More Information

http://www.veeam.com/kb2007 merged

Challenge

Consider the following scenario:

You're running Windows Server 2008 R2 SP1 with the latest Windows updates installed. During the installation of a Veeam Agent CBT driver you get an error:

Microsoft Windows Security Update KB3033929 is missing

On a standalone installation the error looks like this:

User-added image
On a managed installation the error is the same:

User-added image

Solution

KB3033929 is most likely not installed. Install this KB update from the Microsoft Update Catalog.
If KB3033929 is already installed, you may get the following message:

User-added image

KB3033929 is a part of Monthly Quality Rollup updates from Microsoft Corporation. If you have at least October, 2016 Security Monthly Quality Rollup (KB3185330) or later installed on your system, you can apply the following registry value on the Veeam Agent machine and continue installation:

Name:DisableCBTKBCheck
Type:REG_DWORD
Path:HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup
Value:1

NOTE! Make sure that you have at least KB3033929 installed in your system. Installing a CBT driver on a Windows Server without any updates will cause boot issues.

More Information

Please refer to 'Package Details' tab (Microsoft Update Catalog) for details on updates replacements.

Challenge

When creating a Reseller or a Company you cannot assign vCloud Director resources to it even though vCD has been added to the Cloud Connect server

Cause

Due to the discovered issue in Veeam Backup & Replication 9.5 Update 4 GA, vCloud objects are not properly collected by Availability Console.

Solution

The issue is addressed in Veeam Backup & Replication Update 4a. Once Update is installed, vCloud data will be gathered within the standard automated collection.

More Information

If you require a hotfix for Update 4 GA or RTM, please contact Veeam Support.