Build numbers and version of Veeam Agent for Windows | Veeam Agent for Linux

July 10, 2018, 7:52 am

≫ Next: Failed to mount backup

≪ Previous: Build numbers and version of Veeam Backup & Replication

$

0

0

Challenge

This KB article provides a list of all versions and respective build numbers for the following products:

• Veeam Agent for Windows
• Veeam Agent for Linux
• Veeam Endpoint Backup

Solution

Veeam Agent for Windows

Version	Build Number	Release Date	Release Notes
Veeam Agent for Windows GA 2.2	2.2.0.589	2018-07-03	https://www.veeam.com/veeam_agent_windows_2_2_release_notes_rn.pdf
Veeam Agent for Windows GA 2.1	2.1.0.423	2017-12-19	https://www.veeam.com/veeam_agent_windows_2_1_release_notes_rn.pdf
Veeam Agent for Windows GA 2.0	2.0.0.700	2017-05-11	https://www.veeam.com/veeam_agent_windows_2_0_release_notes_rn.pdf

Veeam Agent for Linux

Version	Build Number	Release Date	Release Notes
Veeam Agent for Linux 2.0.1 GA	2.0.1.665	2018-07-03	https://www.veeam.com/veeam_agent_linux_2_0_release_notes_rn.pdf
Veeam Agent for Linux 2.0 GA	2.0.0.400	2017-12-21	--
Veeam Agent for Linux 1.0 U1	1.0.1.364	2017-05-22	https://www.veeam.com/veeam_agent_linux_1_0_release_notes_rn.pdf
Veeam Agent for Linux 1.0 GA	1.0.0.944	2016-12-13	--

Veeam Endpoint Backup

Version	Build Number	Release Date	Release Notes
Veeam Endpoint Backup GA 1.0	1.0.0.1954	2015-04-14	https://www.veeam.com/veeam_endpoint_free_1_0_release_notes_rn.pdf
Veeam Endpoint Backup GA 1.1	1.1.1.212	2015-10-05	https://www.veeam.com/veeam_endpoint_free_1_1_release_notes_rn.pdf
Veeam Endpoint Backup GA 1.5	1.5.0.306	2016-03-17	https://www.veeam.com/veeam_endpoint_free_1_5_release_notes_rn.pdf

---

Failed to mount backup

July 11, 2018, 3:18 am

≫ Next: Update Veeam MP Core Services to Support TLS 1.2 Protocol

≪ Previous: Build numbers and version of Veeam Agent for Windows | Veeam Agent for Linux

$

0

0

Challenge

Recovery from backup on a machine running Linux kernel of one of versions 4.0.0-4.1.33 typically fails with ‘FUSE mount is not supported on kernel versions 4.0.0-4.1.33. Upgrade the kernel and try again.’

Cause

FUSE mounts on mentioned kernels fail due to a known kernel bug which was fixed in later versions. General recommendation is to upgrade kernel to a more recent version. However some in distributions fix was backported to older kernels, so you might want to try the following solution first (applicable to Agent for Linux version 2.0 and later only).

Solution

1. In global configuration file of Agent for Linux – /etc/veeam/veeam.ini – change value of ‘ignoreFuseBug’ parameter from ‘false’ to ‘true’ and uncomment respective line
2. Restart ‘veeamservice’ service to apply changes
3. Retry recovery

Update Veeam MP Core Services to Support TLS 1.2 Protocol

July 12, 2018, 3:04 am

≫ Next: Veeam Agent – Setup ADK/AIK for Recovery Media

≪ Previous: Failed to mount backup

$

0

0

Challenge

Starting with vSphere 6.5, the TLS protocol version 1.2 is enabled by default.
 

Cause

To allow Veeam Virtualization Extensions Service and Veeam VMware Collectors to connect to vCenter Servers that use the TLS 1.2 protocol, you must update Veeam MP core services.
 

Solution

NOTE:
The described procedure applies only to Veeam Virtualization Extensions Service and Veeam VMware Collectors version 8.0.

Update Veeam MP core services by completing the steps below on the machine that runs the Veeam Virtualization Extensions Service and on all machines running Veeam VMware Collector services.

1.    Insert the installation disc into the CD/DVD drive or mount the installation image.
2.    Navigate to the <installation media>\Update folder.
3.    Launch VeeamMP80_Update5_Patch1.exe.
4.    Click Next to start the installation.



NOTE:
You will not be able to proceed to the Veeam MP patch installation process if Microsoft .NET Framework 4.5 is not installed. Download Microsoft .NET Framework 4.5 from the Microsoft website, and install it following the instructions provided by Microsoft.

5.    Click Install and wait for the update installation process to complete.

More Information

You may install this update in silent mode by using this cmdlet: VeeamMP80_Update5_Patch1.exe /silent

Veeam Agent – Setup ADK/AIK for Recovery Media

July 12, 2018, 4:09 am

≫ Next: How to move the Veeam Backup & Replication software to another server

≪ Previous: Update Veeam MP Core Services to Support TLS 1.2 Protocol

$

0

0

Challenge

Veeam Recovery Media Creation process is failing. The error will typically be:

Windows recovery image file not found:

 

Cause

Veeam Recovery Environment is built off the Windows Recovery Environment, and the Recovery Environment is made from the Windows Preinstallation Environment.
If a system is missing the Windows PE/RE components, Veeam Recovery Media will fail to create.

 

Solution

To resolve this issue, we will need to Install the Windows Assessment and Deployment Kit *ADK* (Windows 8 and later), also known as Windows Automated Installation Kit *AIK* for Windows 7. The Server OS versions will use the same Kits as their matching Desktop OS. In this article, we will cover the installation for Windows 10 and Windows 7, as AIK and ADK have different installation steps.

For the first step, you will want to download the appropriate Kit from this link.

Windows 10 - ADK

1. Start the installation from the downloaded file


2. Specify Location. You do not need to change the default path. Click on Next:

3. Select if you want to join the Microsoft CEIP and Click Next:

4. Accept the License Agreement

5. You will only need the following two items checked on the install

6. Once the install is completed, click Close

7. Proceed to “Setting Registry Values”
 

Windows 7 - AIK

For Windows 7 (and Server 2008 R2), Microsoft had the Windows AIK, instead of the ADK.
The installation for the AIK is different, and we will be walking through these differences in this section.

1. AIK is provided as an ISO. To begin the install you will need to first mount the ISO file.
2. Once the ISO is mounted, you will be able to start the installer

3. You will need to select the “Windows AIK Setup” option to start the install

4. Click Next on the Welcome screen for the install

5. Accept the License Agreement and click Next

6. Specify Location. You do not need to change the default path. Click on Next:

7. Click Next to confirm the Install.

8. Click "Close" to finish the install

9. Proceed to “Setting Registry Values”
 

Setting Registry Values

ForceUseAdkForRecoveryMedia

This key Forces Veeam to only use the ADK(or AIK) component.
It is the same key name for both kits.
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

• Name: ForceUseAdkForRecoveryMedia
• Type: REG_DWORD
• Value: 1

4. Restart the “Veeam Agent for Microsoft Windows” service
 

DismPath

This key is used to specify the path toward the DISM tool, which is used to mount the Windows image for setup. This key will only need to be used if the default location for the dism.exe is changed.
The default location is typically: C:\Windows\System32

The dism.exe tool is included within the installation folder path of ADK(AIK), and there will be separate dism.exe(s) for various CPU Architectures. You will want to use Windows Explorer to located the appropriate dism.exe for your CPU Architecture.

For example, on a 64-bit Windows 10 Install, the dism.exe will be located under:
C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\x86\DISM\Dism.exe

To add this key, you will need to perform these steps:
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

• Name: DismPath
• Type: String
• Value: Full path to your EXE, including the dism.exe

EXAMPLE:  C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\x86\DISM\Dism.exe

4. Restart the “Veeam Agent for Microsoft Windows” service

How to move the Veeam Backup & Replication software to another server

June 25, 2018, 7:04 am

≫ Next: Veeam Agent – Setup ADK/AIK for Recovery Media

≪ Previous: Veeam Agent – Setup ADK/AIK for Recovery Media

$

0

0

Challenge

You need to move Veeam Backup & Replication software and database to a new server.

Solution

Note: This section will refer to the server where Veeam Backup & Replication is installed prior to the move as the “old server”, and the location after the move as the “new server”.

---

There are two methods to perform this task:

• Migrate using Configuration Backup (recomended)
• Migrate using SQL Backup

 

---

Migrate using Configuration Backup

Perform the following steps on the old server:

1. Stop and disable all jobs.
2. Manually perform a configuration backup.
3. Close Veeam GUI and stop all Veeam Services.

Perform the following steps on the new server:

1. Install Veeam Backup & Replication with a brand new DB.
2. Apply the latest patch to Veeam Backup & Replication: http://www.veeam.com/patches.html
3. Make sure all local drives that were being used as repositories on the old server are now attached with the same drive letters on the new server.
4. Perform configuration backup restore:
   Note: It will be necessary to re-enter the password for every account during restore of non-encrypted configuration backups.
5. Run a test job to make sure everything moved correctly.

 

---

Migrate using SQL Backup

IMPORTANT: Use this option only if Configuration Backup is not available. If encryption was used on the previous installation, a newly installed backup server will not be able to decrypt such information if attached to the existing database, so any encrypted information will have to be supplied manually – existing encrypted backups have to be re-imported to provide the password.

Perform the following steps on the old server:

1. Stop and disable all jobs.
2. Close Veeam GUI and stop all Veeam Services.
3. Create backup of the SQL database. (http://vee.am/kb1471)
   
   To find out where the VeeamBackup database is located, open regedit on the old server and navigate to: [HKLM\SOFTWARE\Veeam\Veeam Backup and Replication]
    
   There you will find 3 dword values:
   • SqlServerName
   • SqlInstanceName
   • SqlDatabaseName

 

Perform the following steps on the new server:

1. Install Veeam Backup & Replication with a brand new DB.
2. Apply the latest patch to Veeam. http://www.veeam.com/patches.html
3. Make sure all local drives that were being used as repositories on the old server are now attached with the same drive letters on the new server.
4. Restore the Veeam database to the instance on the new server or on a SQL server which will host Veeam DB.
   Click (here) for information regarding restoring databases.
5. To associate Veeam with the restored database, please use the DBConfig Utility to change the Database that Veeam is connecting to.
6. Restart Veeam Services to apply the changes.
7. Open the Veeam console and using the “Manage Credentials” button from the main menu and check the passwords for every account to confirm they are up to date.
8. Run a test job to make sure everything moved correctly.

More Information

In situations where physical local storage is used please see the following KB that details how to relocate Veeam Backup and Replication backup file locations: https://www.veeam.com/kb1729

For steps on migrating the Veeam database to another SQL Instance or Server, please refer to KB1250.

Veeam Agent – Setup ADK/AIK for Recovery Media

July 12, 2018, 4:09 am

≫ Next: Oracle database backup fails with event ID 44

≪ Previous: How to move the Veeam Backup & Replication software to another server

$

0

0

Challenge

Veeam Recovery Media Creation process is failing. The error will typically be:

Windows recovery image file not found:

 

Cause

Veeam Recovery Environment is built off the Windows Recovery Environment, and the Recovery Environment is made from the Windows Preinstallation Environment.
If a system is missing the Windows PE/RE components, Veeam Recovery Media will fail to create.

 

Solution

To resolve this issue, we will need to Install the Windows Assessment and Deployment Kit *ADK* (Windows 8 and later), also known as Windows Automated Installation Kit *AIK* for Windows 7. The Server OS versions will use the same Kits as their matching Desktop OS. In this article, we will cover the installation for Windows 10 and Windows 7, as AIK and ADK have different installation steps.

For the first step, you will want to download the appropriate Kit from this link.

Windows 10 - ADK

1. Start the installation from the downloaded file


2. Specify Location. You do not need to change the default path. Click on Next:

3. Select if you want to join the Microsoft CEIP and Click Next:

4. Accept the License Agreement

5. You will only need the following two items checked on the install

6. Once the install is completed, click Close

7. Proceed to “Setting Registry Values”
 

Windows 7 - AIK

For Windows 7 (and Server 2008 R2), Microsoft had the Windows AIK, instead of the ADK.
The installation for the AIK is different, and we will be walking through these differences in this section.

1. AIK is provided as an ISO. To begin the install you will need to first mount the ISO file.
2. Once the ISO is mounted, you will be able to start the installer

3. You will need to select the “Windows AIK Setup” option to start the install

4. Click Next on the Welcome screen for the install

5. Accept the License Agreement and click Next

6. Specify Location. You do not need to change the default path. Click on Next:

7. Click Next to confirm the Install.

8. Click "Close" to finish the install

9. Proceed to “Setting Registry Values”
 

Setting Registry Values

ForceUseAdkForRecoveryMedia

This key Forces Veeam to only use the ADK(or AIK) component.
It is the same key name for both kits.
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

• Name: ForceUseAdkForRecoveryMedia
• Type: REG_DWORD
• Value: 1

4. Restart the “Veeam Agent for Microsoft Windows” service
 

DismPath

This key is used to specify the path toward the DISM tool, which is used to mount the Windows image for setup. This key will only need to be used if the default location for the dism.exe is changed.
The default location is typically: C:\Windows\System32

The dism.exe tool is included within the installation folder path of ADK(AIK), and there will be separate dism.exe(s) for various CPU Architectures. You will want to use Windows Explorer to located the appropriate dism.exe for your CPU Architecture.

For example, on a 64-bit Windows 10 Install, the dism.exe will be located under:
C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\x86\DISM\Dism.exe

To add this key, you will need to perform these steps:
1. Open Regedit
2. Browse to “HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup”
3. Create a new Registry key with these details:

• Name: DismPath
• Type: String
• Value: Full path to your EXE, including the dism.exe

EXAMPLE:  C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\x86\DISM\Dism.exe

4. Restart the “Veeam Agent for Microsoft Windows” service

Oracle database backup fails with event ID 44

July 12, 2018, 4:47 am

≫ Next: Troubleshooting Error Code ‘32768’, Failed to create VM recovery snapshot

≪ Previous: Veeam Agent – Setup ADK/AIK for Recovery Media

$

0

0

Challenge

Veeam Backup & Replication or Veeam Agent for Microsoft Windows jobs with Application-Aware Processing fail to process a Windows guest with Oracle database installed.
The following sympthoms are observed.

The job fails with this or a similar error:

Creating VSS snapshot Error: Failed to create snapshot: Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
A VSS critical writer has failed. Writer name: [Oracle VSS Writer - ORCL]. Class ID: [{26d02976-b909-43ad-af7e-62a4f625e372}].
Instance ID: [{f58dfa7d-06fa-485c-87fb-29649223a590}]. Writer's state: [VSS_WS_FAILED_AT_PREPARE_BACKUP]. Error code: [0x800423f4].

 

At the same time, you observe a critical event with EventID 44 in the "Application" windows event log on the machine

Log Name: Application
Source: Oracle.VSSWriter.ORCL
Date:  5/30/2018 5:04:03 AM
Event ID: 44
Task Category: (255)
Level: Error
Keywords: Classic
User: N/A
Computer: DB.VEEAM.LOCAL

Description:
VSS-00044: Database in NOARCHIVELOG mode must be in a consistent state for backup.  
 Cause : The database was open read/write or mounted after a SHUTDOWN IMMEDIATE or SHUTDOWN ABORT.  
 Action : Either mount the database in a consistent state or open it read-only to back up the database files.

 

Cause

This error appears only for databases operating in NOARCHIVELOG mode.
A database operating in NOARCHIVELOG mode cannot be backed up when it’s in an inconsistent state.

The Oracle VSS Writer detects the database state before creating a shadow copy, and won’t allow to do this if the database is in inconsistent state.
 

Solution

This VSS failure is triggered by Oracle VSS Writer component.
If you want to keep the database in NOARCHIVELOG mode. there is no proper fix from the Veeam configuration side, as this component is provided by Oracle and does its own checks before database processing.
If you want to fully resolve the issue, you would want to returned it to a consistent state. This is a task suitable for a database administrator.

Another solution would be to switch the database to ARCHIVELOG mode.
 

More Information

However, there is a workaround to skip this Oracle VSS Writer failure.
Please note that you’re doing all the next steps at your own risk and confirm that you’re using a different way to backup such databases or understand possible aftermath.

The following workaround can be applied:

1. Ask your DBA to remove Oracle VSS Writer which is responsible for this Oracle database
2. Instruct Veeam to stop processing Oracle information in this guest machine. On the guest machine, add a registry key of REG_DWORD type with a name DisableOracleProcessing and a value 1.

1. If it is backed up by Veeam Backup & Replication, add the key to ‘HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication’.
2. If it is backed up by Veeam Agent for Windows, add the key to ‘HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Endpoint Backup’ and restart Veeam Agent for Microsoft Windows service.

Using this workaround, you can skip this Oracle VSS Writer failure and make image-level backup of your system. However, please note the following restrictions:

• Your database files for this particular Oracle SID will be still in inconsistent state and may be unrecoverable.
• Database properties will not be collected for any database on this server, as a result no Oracle archived redo log backup or database restore using Veeam Explorer for Oracle is possible.
• If any other database doesn’t have VSS writer installed, database files for such databases may be in a crash-consistent state too.

Doing this you confirm that such databases are being backed up by a different or native RMAN solution. It’s important to keep databases in a consistent state in order to be able to restore them. You can choose to refer this task to your DBA or to Oracle support if necessary.
 

Troubleshooting Error Code ‘32768’, Failed to create VM recovery snapshot

July 13, 2018, 9:25 am

≫ Next: Restoring domain controller from an application-aware backup

≪ Previous: Oracle database backup fails with event ID 44

$

0

0

Challenge

Backups fail with the error Warning 'VM' could not initiate a checkpoint operation: %%2147754996 (0x800423F4). (Virtual machine ID 869a2794-6a7e-4104-8395-8f6a2313d7fa)'). Error code: '32768'.

Cause

If your version of Veeam Backup & Replication is 9.5 update 2 (9.5.0.1038) or prior, there was a known issue that could be addressed by a fix applicable only to that version. However, that known issue has been addressed in update 3 and newer (9.5.0.1536) so if your installation of Veeam Backup & Replication is not fully updated, it is strongly recommended that you consider doing so, or contact Support if you are unable to update and need the fix.

There is a distinct known cause for this error message when there are underlying issues with the SQL VSS writer on the guest VM. You should be able to see error messages in the guest VM’s Event Logs stating, “A critical component required by the SQL writer is not registered” or the VSS writer rejecting the event.

Running the “vssadmin list writers” command might also show the SqlServerWriter in a “Non-retryable error” state.

 

Solution

If you see these errors in the guest, this issue can generally be corrected by repairing the SQL instance on the guest VM as detailed below:

---

Restoring domain controller from an application-aware backup

July 13, 2018, 9:36 am

≫ Next: Backup Copy or Replication job using WAN accelerators fails with “Source WAN accelerator error: Failed to decompress LZ4 block”

≪ Previous: Troubleshooting Error Code ‘32768’, Failed to create VM recovery snapshot

$

0

0

Challenge

Application Aware Image Processing is required to be enabled and working as per Microsoft in order to functionally restore from a DC.

Please refer to this Microsoft Page for more information:
https://technet.microsoft.com/en-us/library/d2cae85b-41ac-497f-8cd1-5fbaa6740ffe(v=ws.10)#backup_and_restore_considerations_for_virtualized_domain_controllers


Since Active Directory implements multi-master replication, where multiple domain controllers sync changes with each other, one of the key challenges is the DC recovery process. This article outlines different DC restore scenarios and goes into some specifics of when and why this or that type of restore is required as well as gives instructions on the manual steps to perform proper DC recovery from backup created with Veeam B&R.

Before going into details, it is worth stressing that by default Veeam B&R performs automated non-authoritative restore of domain controller and in most cases when you need to recover failed DC, authoritative restore is not required.

The following situations are possible:

• Restoring single lost DC in a multi-DC environment
• Restoring entire AD infrastructure (AKA “all DC’s are lost”)
• Restoring from Active Directory corruption

Depending on the scenario, different steps (or no steps at all) are required to perform DC restore. All of the scenarios assume application-aware image processing was enabled in the backup job that backed up the DC being restored.

Solution

Restoring entire AD infrastructure (AKA “all DC’s are lost”)

As mentioned above, the automatic recovery process performs a non-authoritative restore, where the DC reboots and starts looking for other DC’s to sync up. However, in a scenario where all DC’s are gone, there are no other partners available and replication may take quite long (15-30 minutes) to start. To avoid wasting the time attempting to contact replication partners, it is recommended to the primary domain controller, power it on, wait for it to reboot, and force it to become authoritative for SYSVOL, so that it can start replicating. Then restoring other DC’s will be similar to the previous scenario, i.e. will be 100% automatic.

Note: During the restore procedure, make sure the restored DC’s DNS records point to available DNS servers (i.e. to itself).

The procedure for designating DC as authoritative for SYSVOL varies based on whether FRS or DFS-R is used for SYSVOL replication. To determine if you are using FRS or DFSR for SYSVOL in the production environment check the value of the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\DFSR\Parameters\SysVols\Migrating Sysvols\LocalState registry subkey. If this registry subkey exists and its value is set to 3 (ELIMINATED), DFS-R is being used. If the subkey does not exist, or if it has a different value, FRS is being used. 
 
Once you have determined which method is being used for SYSVOL replication, perform the following to designate the restored DC as authoritative:
 
For DFS-R:
1. When booted the second time, navigate to HKLM\System\CurrentControlSet\Services\DFSR registry hive, create a key Restore and create SYSVOL string with the value authoritative.
This value is read by the DFSR service. If this value is not set, the SYSVOL restore is performed non-authoritatively by default.
2. Navigate to HKLM\System\CurrentControlSet\Control\BackupRestore, create a key SystemStateRestore and create a LastRestoreId string with any GUID value. (Example: 10000000-0000-0000-0000-000000000000).
3. Restart DFSR service.
 
For FRS:
1. When booted the second time, navigate to HKLM\System\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup registry hive and change the value of the key Burflag to 000000D4 (hex) or 212 (dec).
This effectively forces the Domain Controllers still using the old FRS technology to start the replication in an authoritative mode. More details about FRS recovery.
2. Restart the NTFRS service.


Notes:
• If you’re restoring a DC without certain FSMO roles, you might want to transfer them to it manually after the restore, using the ntdsutil seize command.
• This type of restore is similar to what Veeam B&R performs automatically when restoring DC within SureBackup isolated virtual lab.
 
 

 

 
The Following should be removed from KB 2119:
 
For VM’s that use DFRS (Server 2008 Domain functionality level and higher) you'll want to follow the following MS KB:
http://technet.microsoft.com/en-us/library/cc816897(WS.10).aspx
 
 
For VM's that use File Replication service (Server 2003 Domain functionality level), this is done by setting the burflags through the registry:
http://support.microsoft.com/kb/290762
 

More Information

How to recover a Domain Controller: Best practices for AD protection (Part 2)
Veeam recovery of a domain controller
Active Directory backup and recovery with Veeam
Recovering Your Active Directory Forest
Windows Server - How to Perform an Authoritative Restore of Active Directory Object
Restoring The SYSVOL (Non-)Authoritatively When Either Using NTFRS Or DFS-R (Part 1)
Restoring The SYSVOL (Non-)Authoritatively When Either Using NTFRS Or DFS-R (Part 2)
Restoring The SYSVOL (Non-)Authoritatively When Either Using NTFRS Or DFS-R (Part 3)

Backup Copy or Replication job using WAN accelerators fails with “Source WAN accelerator error: Failed to decompress LZ4 block”

July 13, 2018, 9:49 am

≫ Next: SQL processing fails with "Logon failure: the user has not been granted the requested logon type at this computer" error after upgrade to Update 3a

≪ Previous: Restoring domain controller from an application-aware backup

$

0

0

Challenge

Backup Copy or Replication job is failing with errors such as, or similar to, the following:
 
Error: Source WAN accelerator error: Failed to decompress LZ4 block: Bad crc
 
Error: Source WAN accelerator error: Failed to decompress LZ4 block: Incorrect decompression result or length
 
Error: Source WAN accelerator error: Client error: Zlib decompression error: [-3].
 

Cause

Cause:    Cyclic Redundancy Check (CRC) failed while extracting data from a backup file indicates that one or more blocks in that file are corrupted.

Typically, this error indicates hardware malfunction during storage or transmission of the backup file. Most types of storage are subject to a small degree of unrecoverable failure – this is commonly called Bit Rot.

In a small sample of cases this has been seen to be caused by corruption in the WAN accelerator Cache itself rather than the backup file.

 

Solution

To isolate, start by configuring the backup copy/replication job to use the direct transport method temporarily disabling WAN acceleration.

If the job works after disabling WAN acceleration, the problem is with the WAN accelerator cache and creating a new volume/rebuilding the WAN accelerator cache may be the next step.

If the error persists (typically seen only in a backup copy, not replication) it means the problem most likely exists with backup files not being able to be recompressed. In this case:

2. Run an active full backup on the source job.

The backup file will be completely recreated from source data. Once done, re-run the backup copy job and it will grab the data from this backup file and should bypass the error. If not:

3. Trigger an active full backup on the backup copy job.

Veeam will read the data entirely from the new backup file, bypassing all previous backup files on both the source and the target repository. This will isolate the problem from any previous backup file corruption. Should the error persist at that point:

4. Run a chkdsk or equivalent function on both the source storage, the VM datastore storage, and the target storage for this backup copy job. We’ve recreated, at this point, source and target backup files and ruled out the WAN accelerator components, so as uncommon as it is, it’s likely corruption on one of the storage volumes causing consistency errors.

This is the same as the errors mentioned in the below KB:
https://www.veeam.com/kb1795

 

More Information

The Windows event log on the repository server may contain Disk Event 7: “The device… has a bad block.” If so, run CHKDSK /F or /R. A small number of bad blocks are normal on most storage devices. If additional bad block events occur after running CHKDSK, check the SMART status and consider replacing the disk. If using a Network Attached Storage device, contact your storage vendor to determine what diagnostics are available.

It is strongly recommended that you test your backup files onsite before sending them to an offsite location for storage.  You can do this through test restores or by using the SureBackup feature included in Enterprise or Enterprise Plus licensing. For more information on SureBackup see the below links:

Practical Guide to SureBackup
Veeam Backup & Replication - SureBackup Step-by-Step Configuration Guide


 

SQL processing fails with "Logon failure: the user has not been granted the requested logon type at this computer" error after upgrade to Update 3a

July 13, 2018, 12:25 pm

≫ Next: Veeam Agent for Microsoft Windows deployment fails with “Failed to call RPC function 'PckgCheckSignature'

≪ Previous: Backup Copy or Replication job using WAN accelerators fails with “Source WAN accelerator error: Failed to decompress LZ4 block”

$

0

0

Challenge

Just after upgrade Veeam B&R to Update 3a you may notice failures on following operations:

• truncate SQL transaction logs;
• collect SQL instances for Veeam Explorer for SQL (VESQL);
• back up database transaction logs;

While the error message varies depending on actual operation there is a common error "Logon failure: the user has not been granted the requested logon type at this computer. Code: 1385" thrown in the job sessions.

Cause

The issue is caused due to additional permission and roles required for Update 3a:

1. Account used for application-aware processing should have the following security policies applied:
   • Logon as a batch job granted.
   • Deny logon as a batch job not set.
2. In case you need to provide minimum permissions for the account to perform backup operations additionally db_datawriter role should be applied on system database msdb

 

Solution

Please adjust account's security rights and roles on msdb according to the updated system requirements.
Alternatively you may install private fix with following workflow:

1. After failing to logon as a batch job, the job will immediately retry using interactive logon (pre-U3a way) before failing completely.

2. Warning about backupset update failure was changed to info event which additionally explains the issue in better details:
Transaction logs backup will not be possible due to insufficient permissions to update backupset for SQL instance: %SQLInstance%


Here is instruction how to install private fix:

1. Stop Veeam jobs and Veeam services;

2. Take a backup of the following backup files on Veeam B&R server and VSS proxies:

• C:\Program Files\Veeam\Backup and Replication\Backup\Veeam.Backup.Core.dll
• C:\Program Files\Veeam\Backup and Replication\Backup\Veeam.Backup.VssProvider.dll
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\sql\VeeamLogShipper_X64.exe
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\sql\VeeamLogShipper_X86.exe
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\VeeamGuestHelpers\VeeamVssSupport2008R2_X64.dll
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\VeeamGuestHelpers\VeeamVssSupport2003_X64.dll
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\VeeamGuestHelpers\VeeamVssSupport2003_X86.dll
• C:\Program Files\Veeam\Backup and Replication\Backup\VSS\VeeamGuestHelpers\VeeamVssSupportXP_X86.dll

3. Replace the file with one from the fix;

4. Start Veeam services.

Veeam Agent for Microsoft Windows deployment fails with “Failed to call RPC function 'PckgCheckSignature'

July 16, 2018, 2:05 am

≫ Next: Red Hat Enterprise Linux 7.5 is not supported by Veeam Agent for Linux 1.x and 2.0

≪ Previous: SQL processing fails with "Logon failure: the user has not been granted the requested logon type at this computer" error after upgrade to Update 3a

$

0

0

Challenge

When a computer is being added to protection group, Veeam Agent for Microsoft Windows deployment starts and  fails with the following:
 

Info [UploadManager] Checking windows package 'C:\ProgramData\Veeam\Agents\VAW\Veeam_B&R_Endpoint_x64.msi' signature on host 'HOSTNAME'
Error Failed to call RPC function 'PckgCheckSignature': Signature of module 'C:\ProgramData\Veeam\Agents\VAW\Veeam_B&R_Endpoint_x64.msi' is invalid..
Error Signature of module 'C:\ProgramData\Veeam\Agents\VAW\Veeam_B&R_Endpoint_x64.msi' is invalid Error --tr:Failed to call DoRpc. CmdName: [PckgCheckSignature].

Cause

The certificate which is used to sign the Veeam Agent installation package is not installed in the Trusted Root Certification Authority store on the client computer.

Solution

1. Download DigiCert Assured ID Root CA here (right click on Download button - save as)
2. Double click the downloaded .srt file
3. Click Install Certificate
4. Choose Local Machine and click on Next
5. Place the certificate into Trusted Root Certification Authorities by clicking the Browse button.

 

Red Hat Enterprise Linux 7.5 is not supported by Veeam Agent for Linux 1.x and 2.0

July 16, 2018, 2:07 am

≫ Next: Recovery Media based on Windows 1709 fails to connect to SMB1 shares

≪ Previous: Veeam Agent for Microsoft Windows deployment fails with “Failed to call RPC function 'PckgCheckSignature'

$

0

0

Challenge

Using Veeam Agent for Linux on Red Hat Enterprise Linux (RHEL) 7.5 and its derivatives (CentOS 7.5, Oracle RHCK 7.5) causes the operating system to crash during backup snapshot creation.

Cause

Veeamsnap module used in Veeam Agent for Linux 2.0 and earlier is not compatible with RHEL 7.5 kernel 3.10.0-862

Solution

Please submit a support case in order to obtain the hotfix

Recovery Media based on Windows 1709 fails to connect to SMB1 shares

July 16, 2018, 2:08 am

≫ Next: Item is not suitable for restore

≪ Previous: Red Hat Enterprise Linux 7.5 is not supported by Veeam Agent for Linux 1.x and 2.0

$

0

0

Challenge

When booting with Recovery Media which is based on Microsoft Windows 10 Fall Creators Update or Microsoft Windows Server 1709, a connection attempt to CIFS shares fails with:

You can't connect to the file share because it's not secure. This share requires the obsolete SMB1 protocol, which is unsafe and could expose your system to attack.
Your system requires SMB2 or higher. For more info on resolving this issue, see: https://go.microsoft.com/fwlink/?linkid=852747

Cause

In Windows 10 Fall Creators Update and Windows Server, version 1709 (RS3), the Server Message Block version 1 (SMBv1) network protocol is no longer installed by default. It was superseded by SMBv2 and later protocols starting in 2007. Microsoft publicly deprecated the SMBv1 protocol in 2014.

Solution

Enable SMBv2 or SMBv3 feature on your NAS device.

More Information

https://support.microsoft.com/en-us/help/4034314/smbv1-is-not-installed-windows-10-and-windows-server-version-1709

Item is not suitable for restore

July 16, 2018, 2:10 am

≫ Next: System logs rapidly grow in size filled by veeamsnap errors

≪ Previous: Recovery Media based on Windows 1709 fails to connect to SMB1 shares

$

0

0

Challenge

Upon attempt to recover LVM PV or LV from backup using Agent for Linux recovery media ‘Item is not suitable for restore’ error is thrown if ‘Restore whole disk from…’ option is selected for target disk:





 

Solution

Navigate to desired PV or LV in backup, select it and use ‘Restore volume to…’ option instead:









 

---

System logs rapidly grow in size filled by veeamsnap errors

July 16, 2018, 2:12 am

≫ Next: Veeam repository update fails, GPG key error

≪ Previous: Item is not suitable for restore

$

0

0

Challenge

Agent for Linux kernel driver (veeamsnap) fills system logs with memory allocation error messages rapidly, system might get low on disk space.
 

Cause

It's a known issue, which will be fixed in upcoming releases.
Example:

Apr 19 11:26:45 hostname kernel: ERR veeamsnap:page_array_alloc Cannot allocate page. Schedule.

Affected OS: Red Hat Enterprise Linux version 6 (or derivatives)
Affected kernel version: kernel-2.6.32-279.el6.i686
Affected veeamsnap version: kmod-veeamsnap-2.0.0.400-1.el6.i386
 

Solution

Use dkms version of veeamsnap (http://repository.veeam.com/backup/linux/agent/rpm/el/6/i386/veeamsnap-2.0.0.400-1.noarch.rpm).
 

More Information

Useful articles:

• User Guide: Installing Veeam Agent for Linux
• User Guide: Installing Veeam Agent for Linux in Offline Mode
• KB 2260: Failed to load module [veeamsnap] (express version)

Veeam repository update fails, GPG key error

July 16, 2018, 2:14 am

≫ Next: Veeam Agent Management Linux user account requirements

≪ Previous: System logs rapidly grow in size filled by veeamsnap errors

$

0

0

Challenge

When attempting to update the Veeam Agent for Linux or veeam-release repository package, an error is encountered:
 

GPG key signature on key http://repository.veeam.com/keys/RPM-GPG-KEY-VeeamSoftwareRepo.asc does not match CA Key for repo: veeam

or
 

The following signatures were invalid: EXPKEYSIG 54706D7C5A382FCD Veeam Software Repository key support@veeam.com

Cause

Veeam repository GPG key has changed, import of a new key is required.

Solution

To resolve this issue please install new GPG keys manually.

For .deb based distributions:

curl http://repository.veeam.com/keys/veeam.gpg | apt-key add

For .rpm based distributions:

rpm --import http://repository.veeam.com/keys/RPM-EFDCEA77

 

More Information

Alternatively you can download and install the updated veeam -release package from the website or from the repository.

Veeam Agent Management Linux user account requirements

July 16, 2018, 2:15 am

≫ Next: How to change SQL log backup location for Veeam Agent for Microsoft Windows backup

≪ Previous: Veeam repository update fails, GPG key error

$

0

0

Challenge

Granular ‘sudo’ configuration for a Linux user account to be used for Agent Management.

Cause

As per Agent Management User Guide, a user account to be used for Agent Management should have administrative permissions on the computer that you want to add to a protection group or a job. If you would like to avoid using ‘root’ account in favor of a sudoer account, you might also want to set permissions granularly for one. This document helps you to achieve this goal.

Solution

• Linux user account used by Veeam Backup and Replication server for any Veeam Agent for Linux deployment and management operations must have /bin/bash shell set by default.
• /etc/sudoers file should contain a line with ‘requiretty’ parameter negated explicitly for desired Linux user account
• The account must have either ‘root’ permissions or permissions to execute a specific list of commands as a sudoer on the target Linux machine.

Here is the example of /etc/sudoers file containing proper settings (for SLES 11):

Defaults:username !requiretty


# Protection Group Rescan
username ALL=(root) NOPASSWD: /bin/hostname
username ALL=(root) NOPASSWD: /bin/uname
username ALL=(root) NOPASSWD: /bin/arch
username ALL=(root) NOPASSWD: /bin/cat
username ALL=(root) NOPASSWD: /usr/bin/test
username ALL=(root) NOPASSWD: /usr/mkdir
username ALL=(root) NOPASSWD: /usr/rmdir
username ALL=(root) NOPASSWD: /usr/chown
username ALL=(root) NOPASSWD: /tmp/*/usernameid
username ALL=(root) NOPASSWD: /usr/bin/scp
 
# Agent config/backup
username ALL=(root) NOPASSWD: /usr/bin/veeamconfig
 
# Agent deployment
username ALL=(root) NOPASSWD: /usr/bin/zypper
username ALL=(root) NOPASSWD: /bin/rm
 
# FLR Restore via VBR console
username ALL=(root) NOPASSWD: /bin/sh
username ALL=(root) NOPASSWD: /bin/touch
username ALL=(root) NOPASSWD: /bin/chmod
username ALL=(root) NOPASSWD: /tmp/Username*-*-*-*
username ALL=(root) NOPASSWD: /usr/rm
username ALL=(root) NOPASSWD: /usr/ps
username ALL=(root) NOPASSWD: /usr/mv

For distributions other than SLES replace /usr/bin/zypper with a path to corresponding package manager binary.

How to change SQL log backup location for Veeam Agent for Microsoft Windows backup

July 16, 2018, 2:18 am

≫ Next: Using a CA-signed server certificate in the Veeam Agent management infrastructure

≪ Previous: Veeam Agent Management Linux user account requirements

$

0

0

Challenge

Veeam Agent for Microsoft Windows copies transaction log files from the log archive destination (set by the Microsoft SQL Server administrator) to a temporary folder on the Veeam Agent computer file system. By default the path is: C:\ProgramData\Veeam\Endpoint\SqlLogBackup. In case if the default destanation should be changed, you can specify it in SqlTempLogPath(STRING) registry key.

Solution

Important: the key should be created on the Veeam Agent for Windows machine!

SqlTempLogPath
KEY: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication
TYPE: REG_SZ
VALUE: path to desired location (should be without \ in the end)

Using a CA-signed server certificate in the Veeam Agent management infrastructure

July 16, 2018, 7:18 am

≫ Next: How to test hotadd manually

≪ Previous: How to change SQL log backup location for Veeam Agent for Microsoft Windows backup

$

0

0

Challenge

To allow communications between Veeam Agents and VBR, TLS certificates are used. By default, Veeam Backup & Replication uses a self-signed certificate.

Solution

In order to use a certificate signed by Certification Authority (CA), the following requirements should be met:

• Veeam Agents must trust the Certification Authority and the VBR signed certificate (they must be added to the Trusted Root Certification Authority store on the clients)
• Certificate revocation List (CRL) should be accessible from Veeam Agents and VBR server

A certificate signed by Certification Authority should have the following key usage to sign and deploy child certificates on Veeam Agents:

• Digital Signature
• Certificate Signing
• Off-line CRL Signing
• CRL Signing (86)

For example a subordinate CA Certificate template in Windows has the required key usages:

After applying the signed certificate on the VBR server according to the User Guide, on the next job run Veeam Agents will receive child certificates. The resulting certification path will look like this:

More Information

Note: Veeam Agent for Microsoft Windows version 2.1 has a known issue with CRL check if a signed certificate is installed on the VBR server. Please contact technical support in order to obtain a fix.