Quantcast
Channel: Veeam Support Knowledge Base
Viewing all 4476 articles
Browse latest View live

Web plugin troubleshooting guide

$
0
0

Challenge

Web plugin is a new feature, which has little in common with other features and it may seem to be difficult to troubleshoot. It is suggested to follow this guide rather than trying to analyze error message, as it is not very informative and almost always misleading.

 

Solution

Known issues and solutions

Unable to connect Enterprise Manager
Make sure that Enterprise Manager and RestAPI services are running. Please disable proxy in Internet Explorer on vCenter and EM servers, disable firewalls.
If it doesn't help, then review plugin log on vCenter, if you see something like "Certificate error: Remote servers certificate thumbprint does not match the thumbprint stored in the application settings" then
1. Perform plugin uninstall with cleanup(see section below)
2. Reinstall Enterprise Manager and select required certificate during installation

Invalid SSO Token
Steps to resolve:
1. Make sure, that EM and RestAPI services are running under domain account with local administrator priveleges.
2. Add user(that is used to log on to vSphere web client) in EM Roles explicitly.
If it doesn't help, then analyze SSO token.

Server error: null
Plugin files are not completely installed, corrupted or 2 versions of plugin are installed. If you have vShpere 5.5 then plugin might have old version.
Steps to resolve:
1. Verify plugin version installed in EM(Configuration - vCenter Servers) and EM version(EM - configuration - about - Veeam Backup Enterprise Manager Service). For vSphere 5.5 it should be not lower than 7.0.0.764. If needed, apply the latest patch once again. If Enterprise Manager and Veeam B&R are installed on different servers, make sure that the latest patch is installed on both servers.

2. Perform Plugin uninstall with cleanup
3. Install plugin

Installation fails with message "Check if there is an existing one already installed" but the plugin is not already installed
If Svc.VeeamBES.log contains “NoPermissionFault”: Make sure the user account that is accessing the vCenter is listed in the vCenter permissions at the root level (or is part of a group at that level), and is assigned to a role with Extension privileges.
If Svc.VeeamBES.log contains “null reference exception”: In IIS Manager on the Enterprise Manager server, make sure that an SSL certificate is assigned to the HTTPS binding for the VeeamBackup site.




Generic troubleshooting guide

If the issue is not listed in known issues section or it didn't help, then follow this guide before escalating a ticket.
1. Disable proxy in IE and firewall both on EM and VC
2. Verify that EM version is current (EM - configuration - about - Veeam Backup Enterprise Manager Service) and install latest patch if needed. In case of upgrade, uninstall plugin from EM, restart vSphere web client service and install it again.
3. Add user(used to log on to VC) to EM roles explicitly
4. Make sure that RestAPI and EM services are running under domain account with local admin priveleges

Check if RestAPI service is available and responding:
1. Login to EM server and open in browser https://<em-server>:9398/web. You should receive an XML. Username and password may need to be entered. You can safely ignore untrusted certificate warning.
2. If this page doesn't open, check if RestApi service is started
3. If service is started, run netstat -abn from command line to check if port 9398 is listened by Veeam.Backup.EnterpriseService.exe
4. Try to open "http://<em-server>:9399/web", if this page opens while "https://<em-server>:9398/web" doesn't - you need to re-install SSL certificate for RestAPI (see corresponding chapter below - RestAPI certificate change)

If nothing, listed above helps:
1. Perform plugin uninstall with cleanup(see section below)
2. Reinstall EM
3. Install plugin
If the above steps do not resolve the issue, please contact Veeam Support.


Thoubleshooting steps

Plugin uninstall with cleanup
1. Uninstall plugin from EM
2. Restart vCenter Web Client Service. On windows server from services.msc console, on VC appliance from Vcenter Management web page

3. Remove all pieces of web plugin, that were not removed:
3a. Open SSH session to VC appliance and remove all enties that are returned by find / -name '*veeam*'.
3b. On windows server remove all files and folders containing veeam in %programdata%\vmware and %programfiles%\Vmware



SSO token analyze
1. Open Svc.VeeamRestAPI.log
2. SSO token is Base64 encoded. Copy encoded text between "Failed to authorize user. SSO token: [" and closing "]" into notepad and remove all occurrences of "\n" (replace to empty string)
3. Decode resulting Base64 string. You can use http://www.base64decode.org/ for that.
4. In the resulting xml look for tag "saml2:Issuer". There is a URL to the VC SSO service. Verify if the URL points to the VC. Try opening it from Veeam server. Response file in XML format should be opened or downloaded, depending on browser settings. Certificate not trusted error can be safely ignored. If the URL is wrong, then customer should reconfigure SSO.

RestAPI certificate change
To view all SSL certificates assigned on different ports, run command prompt on EM server "netsh http show sslcert".
Locate 2 records:
0.0.0.0:9443 (stands for Enterprise Manager)
0.0.0.0:9398 (RestAPI service)
1. Copy "Application ID" value from the 0.0.0.0:9398 section
2. Copy "Certificate Hash" value from 0.0.0.0:9443 section
3. Execute "netsh http delete sslcert ipport=0.0.0.0:9398" to delete existing cert
4. Execute "netsh http add sslcert ipport=0.0.0.0:9398 certhash=<hash> appid=<id>"(where <id> and <hash> are values, gathered before) to set new certificate

More Information

Required logs
1. (From EM) Svc.VeeamBES.log
2. (From EM) Svc.VeeamRestAPI.log
3. (From VC) VMWare web client log(logs location on VC - http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2004090
4. (From VC) For vSphere 5.1 or older, web plugin logs from VC. Should be located in the same folder, where VMWare Web Client logs are. For vSphere 5.5 there is no dedicated Veeam Plugin log, info is written in Web Client log instead

5. (From VC) For vSphere 6 please see - https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2110014


 

How to Map Replicas in Cloud Connect

$
0
0

Challenge

When creating a replication job, seeding or mapping can be used to minimize the amount of traffic sent to a Cloud Service Provider. In most cases, seeding is the preferred method because it is not possible to map a Cloud Connect replication job to a replica that was not created with Cloud Connect. However, it is possible for a Cloud Service Provider to work around this limitation by replicating the existing replica.

Note: it is not possible to use a backup of a replica as a seed, because Veeam searches the backup file for the ID of the VM to be replicated, which is different from the ID of the replica.
 

Solution

These steps must be performed by a Cloud Connect Service Provider. 
Tenants who are unable to map replicas should ask their service provider whether this workaround is available.
  1. Connect a console to a Veeam backup server with access to the infrastructure containing the original replica. Typically this will be a backup server with a per-socket license, and not the SP Veeam backup server used to provide cloud resources to tenants.
  2. Use the tenant’s credentials to connect to a cloud gateway.
  3. Create and run a job to replicate the original replica into the cloud infrastructure. When configuring the job, consider that adding the default suffix will result in VMs named *_replica_replca.
  4. Delete the replication job created in step 3; do not delete the replica.
  5. Remove the service provider that was configured in step 2. Advise the tenant to proceed with steps 6-9, below.
Note: The console used for steps 1-5 may still display the replica after the service provider is removed in step 5; the replica should not appear the next time the console is opened.

These steps must be performed by the tenant:
  1. In the Backup Infrastructure view, add or rescan the cloud service provider.
  2. Verify that the replica is visible in the Replicas node of the Backup & Replication view. Press F5 to refresh the view if needed.
  3. Map the replica to a replication job. This must be done manually; the Detect option is not available.
  4. Run the mapped job and test the replica. Let the service provider know that they may proceed with the final steps.
These steps must be performed by a Cloud Connect Service Provider:
  1. Both SP and tenant will see duplicate replicas (with different source VMs) in the console. To resolve this, the SP must remove the older replica from configuration.
  2. (Optional) When the new replica has been tested successfully, the Service Provider may delete the original replica that is no longer needed.

Changing SQL Server or database connection settings for Veeam ONE

$
0
0

Challenge

You need to change SQL Server or database connection settings for Veeam ONE.

Solution

To change SQL Server or database connection settings for Veeam ONE Server, you can use the Veeam ONE Settings utility.

To modify settings, follow the instructions below.

Provision a domain account for Veeam ONE Monitor and Veeam ONE Reporter on the new SQL Server. This can be one account or two different accounts. Move the database to the new SQL Server. Grant to the account(s) db_owner permissions on the VeeamOne database.  

Next, modify the accounts used by monitoring and reporting components of Veeam ONE.

First, modify connection settings for Veeam ONE Monitor. To accomplish this, follow the next steps:
  1. Launch the Veeam ONE Settings utility available under the following path:

    C:\Program Files\Common Files\Veeam\Veeam ONE Settings\VeeamOneSettings.exe

  2. In the menu on the left, select General, open the Database tab and make the necessary changes in the SQL Server section.

    User-added image

  3. Make sure this account has sufficient privileges on the Veeam ONE database on the SQL Server side, as it will be used for performance data collection.

  4. Click the Test Connection button to check whether new configuration settings are correct.

  5. Save changes.

  6. Restart Veeam ONE Monitor Server service.

Then modify connection settings for Veeam ONE Reporter and Business View. To accomplish this, follow the next steps:

  1. In the list of services, locate Veeam ONE Reporter Server service and choose to modify its properties. Open the Log On tab and specify the necessary account for the service in the Log on as section.

  2. Make sure this account has sufficient privileges on the Veeam ONE database on the SQL Server side, as it will be used for data collection.

  3. Launch the Veeam ONE Settings utility available under the following path:

    C:\Program Files\Common Files\Veeam\Veeam ONE Settings\VeeamOneSettings.exe

  4. In the menu on the left, select General, open the Web Identity tab and make the necessary changes in the Web identity section.

    User-added image

  5. Make sure this account has sufficient privileges for Veeam ONE's database on a SQL Server side, as it will be used for overall Web UI operation and reports creation.

  6. Save changes.

  7. Restart Veeam ONE Reporter Server service.

More Information

If you have any questions or problems, please contact Veeam Support at http://www.veeam.com/support.html

Topology construction failing due to a NullReferenceException

$
0
0

Challenge

Management Pack fails to build the environmental topology. The vmspi.log (default location: C:\Program Files\Veeam\Veeam Virtualization Extensions for System Center\Collector\Log) contains the following entries:
 
+** System.NullReferenceException: Object reference not set to an instance of an object. at nworksCore.TopologyNode.Clone()

Cause

Some host hardware objects may fail to return mandatory properties, such as NIC MAC address or NIC name. In this case Veeam Colelctor would fail to process such objects.
 

Solution

The hotfix prevents Veeam Collector from crashing and provides additional output about the object with missing properties. Download the hotfix here.

Follow these steps to apply the fix.

  1. Stop Veeam VMware Collector service.
  2. Make a backup of VeeamCollector.Core.dll (default path: C:\Program Files\Veeam\Veeam Virtualization Extensions for System Center\Collector).
  3. Unpack the downloaded .zip and replace VeeamCollector.Core.dll with the file from the archive.
  4. Start Veeam VMware Collector service.

More Information

If you have any questions, contact Veeam Support.

Hardware provider is missing in Veeam Backup and Replication user interface

$
0
0

Challenge

Backups using a specific hardware provider do not work, provider is missing from the user interface:

User-added image

Cause

Veeam fails over to a software provider which does not have the functionality of a hardware provider (e.g. off-host proxy, fast snapshot creation, etc.)

Solution

On the Veeam Backup & Replication server, please create the following registry key:

HKLM\SOFTWARE\VeeaM\Veeam Backup and Replication UIShowAllVssProvidersForVolume
REG_DWORD
1


After creating the registry key, please restart the Veeam services and make sure that the required provider is visible and can be selected.

More Information

This value disables the advanced compatibility step which automatically chooses the best compatible providers.

Windows File Level Restore hangs on "Starting restore session"

$
0
0

Challenge

Windows File Level Restore hangs on "Starting restore session" for abnormally long period of time while free disk space on drive C:\ keeps decreasing dramatically.
You may find following path with cache file constantly growing:
C:\Windows\system32\config\systemprofile\AppData\Local\mount_cache{GUID} or C:\Windows\SysWOW64\config\systemprofile\AppData\Local\mount_cache{GUID}.
 

Cause

The issue occurs when guest Windows has Transactional NTFS (TxF) turned on. When mounting to Veeam server it can cause NTFS driver to enter infinite loop trying to re-apply transaction log.

Solution

To resolve:

1. Login to guest OS as administartor. Run cmd with elevated privileges. Execute 2 following commands:
fsutil resource setautoreset true
fsutil resource setavailable
2. Make a new backup point of that VM. verify that File Level Restore starts fine from new backup point

More Information

For more information about Transactional NTFS (TxF)
https://msdn.microsoft.com/ru-ru/library/windows/desktop/bb968806(v=vs.85).aspx

Validating Veeam Endpoint Backup service properties

$
0
0

Challenge

While running Veeam Endpoint Backup Free you receive a “Services suddenly stopped working” error in the Veeam Endpoint Backup Control Panel.
 

Cause

Veeam Endpoint Backup service is either not started or is missing from the Windows Services console. This could happen for various reasons.
 

Solution

1. Reboot your computer and check if the error appears again.
2. If the error persists, go to Control panel > Administrative Tools and run Services. Check if the Veeam Endpoint Backup service is set to Automatic start. To change the startup type, please right click Veeam Endpoint Backup and select Properties. In the Startup Type drop down menu choose Automatic and hit Apply.
3. On the Log On tab, make sure that Log on as is set to Local System Account.
4. To restart the service manually, right click Veeam Endpoint Backup and choose Restart.
5. Set up antivirus exclusions as described in this article.

More Information

If the issue persists, check this Veeam KB article.

If the described methods do not resolve the issue, please open a support case by going to Veeam Endpoint Backup Control Panel > Support > Technical Support.

Failed job is marked as Successful on the Cloud Provider side

$
0
0

Challenge

In some cases a SP backup server receives "Success" status of a tenant job even when the job has failed on the Tenant's server.

Cause

Generally, cloud job sessions are marked failed on the SP backup server in the following cases:
  • All cloud gateways are unavailable
  • There is not enough space on the disk
  • An existing connection was forcibly closed by the remote host
  • Repository quota exceeded

All issues above should be corrected by a SP, and therefore Veeam reports such jobs as failed. For other types of issues, a failed job may be marked successful because it cannot be fixed on the SP side and requires involving a tenant.

Veeam Backup & Replication - SureBackup Configuration Guide v8.x

Application-Aware Image Processing failure with SQL 2012 SP2 case-sensitive collation database

$
0
0

Challenge

Job using Application-Aware Image Processing fails during VSS processing with:
 
Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. A VSS critical writer has failed. Writer name: [SqlServerWriter]. Class ID: [{}]. Instance ID: [{}]. Writer's state: [VSS_WS_FAILED_AT_PREPARE_SNAPSHOT]. Error code: [0x800423f4].] 

Cause

This is a known issue documented in this Microsoft KB: http://support2.microsoft.com/kb/2987610

Solution

Run this query to see if the issue is present:

SELECT 1 FROM SYS.AVAILABILITY_REPLICAS AS AR INNER JOIN SYS.DATABASES AS D ON DB_ID(N'xxxx') = D.DATABASE_ID AND AR.REPLICA_ID = D.REPLICA_ID INNER JOIN SYS.DM_HADR_AVAILABILITY_REPLICA_STATES ST ON AR.REPLICA_ID = ST.REPLICA_ID AND ST.ROLE = 2
 
If you receive the following, this confirms the issue.
Invalid object name 'SYS.AVAILABILITY_REPLICAS'.
 
This issue is resolved by the following Microsoft KB:  http://support2.microsoft.com/kb/2983175

 

Windows File Level Restore running from Remote Shell on Windows 10 failing unexpectedly

$
0
0

Challenge

You are trying to kick of Windows File Level Restore running from Remote Shell on Windows 10 but it fails unexpectedly with a general error:

User-added image

In the log files (C:\ProgramData\Veeam\Backup\FLRSessions\Windows\FLR__<vm_name>_\VeeamAgent.MountVm.Client.log) you may observe following error code:

[05.09.2016 18:30:14] <  4728> mnt| Virtual space {b7ac05db-bfce-41fb-8d5c-780993e049de} was destroyed
[05.09.2016 18:30:14] <  4728> mnt| VeeamFCT driver communication library was deinitialized
[05.09.2016 18:30:14] <  4728> drv| ERR |CVeeamFCTDriver::FileMount: Failed to mount file 'C:\WINDOWS\system32\config\systemprofile\aoivmtwx.mot\veeamflr-0000417000007499-0000.flat'.
[05.09.2016 18:30:14] <  4728> drv| >>  |Details: FctVsAddFile() : HRESULT = 0x80070003
 

Cause

Windows environment variables for local system (SYSTEM) are missing or not set.

Solution

You need to recreate missing values in the following registry key on Windows with Remote Shell running: 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment 

Name: TMP 
Type: Expandable String Value 
Value: %SystemRoot%\TEMP 

Name: TEMP 
Type: Expandable String Value 
Value: %SystemRoot%\TEMP

Afterwards please restart Windows.

Appliance Mode (Hotadd) isn't avaiable if proxy VM and its replica are running under the same vCenter

$
0
0

Challenge

Proxy VM is no longer processing VMs over Virtual Appliance mode (Hotadd) or does it randomly  after upgrade to v9 despite all requirements are met

Cause

Starting with v9 Veeam B&R is using VDDK 6.0 in order to communicate with VMware infrastructure. This version of VDDK libraries introduced additional logic to validate possible transport modes on a proxy. This validation procedure for Virtual appliance (Hotadd) fails on this proxy because BIOS UUID isn't unique. Most likely there is a replica of a proxy VM registered within same vCenter Server.
 

Solution

Due to this additional check implemented in the VMware VDDK 6.0 it is advisable  to either stop replicating Veeam B&R or proxies within same vCenter or replicate them to standalone ESXi hosts added explicitly to the Veeam server. In order to protect you Veeam B&R server please use Configuration Backup job (see more details: https://helpcenter.veeam.com/backup/hyperv/vbr_config.html).

More Information

Per Virtual Disk Development Kit 6.0 Release Notes https://www.vmware.com/support/developer/vddk/vddk-600-releasenotes.html#knownissues:
HotAdd skipped if proxy has same BIOS UUID as the target VM.
The VixDiskLib_ConnectEx function contains a pre-check for HotAdd to validate that the proxy is virtual machine. (HotAdd is not allowed on physical machines.) If the proxy VM and the target VM have the same BIOS UUID, validation fails because multiple VMs are returned from pre-check. This results in skipping HotAdd mode and reverting to another transport. The workaround is to make the BIOS UUID keys unique.

VM Uptime in Hyper-V Manager Might Reset During Backup

$
0
0

Challenge

System uptime reported by Hyper-V manager appears inaccurate after Application-Aware enabled backup or replication. System uptime inside the guest OS is accurate.

Cause

The Application-Aware backup process in Hyper-V 2012 R2 will use Saved State backup if Online Backup is not available. The Saved State backup process causes the system uptime counter in Hyper-V manager to reset.

Solution

Find the issue preventing online backup, causing Veeam to use Saved State backup.
 
Ignore the time reported by Hyper-V Manager and instead reference the uptime reported by the guest OS.
 
Enable Application-Aware only for VMs which need it to minimize the issue.

More Information

This issue is present with any software using Saved State backups, including Windows Backup.

Before Backup:

User-added image


After AAIP enabled backup:

User-added image

 

EMC Data Domain Storage with Veeam Backup & Replication: Configuration Best Practices and Performance Expectations

$
0
0

Challenge

This article documents general performance expectations, best practices, and configuration advice, when using an EMC Data Domain appliance with deduplication as a repository for Veeam Backup & Replication.

Solution

For further information regarding how Veeam Backup & Replication works with EMC Data Domain DDBoost please review:
http://helpcenter.veeam.com/backup/80/vsphere/deduplicating_appliance_emc.html
 
Performance Expectations
 
EMC Data Domain Deduplication Storage Systems provide both high compression and deduplication ratios so that data can be kept for extended periods.  When a Data Domain is configured as a repository for Veeam Backup & Replication, write performance may vary depending upon the particular EMC Data Domain Deduplication System model, protocol, and backup infrastructure architecture.
 
When attempting to read data from a Data Domain it must rehydrate and decompress each block, for this reason operations which read from the Data Domain will perform slower than non-dedupilcated storage, this is more noticeable with operations which use random I/O. All restores will occur as fast as the environment can accept new information, and as fast as the Data Domain can decompress and rehydrate the blocks.
 
For quick recovery you may consider using fast primary storage and keeping a several restore points (3-7) for quick restore operations such as Instant Recovery, SureBackup, Windows or Other-OS File restores since they generate the highest amount of random reads. Then use the DataDomain as a secondary storage to store files for long term retention. If an EMC Data Domain Deduplication System will be used as primary storage, it is strongly suggested to leverage alternative restore capabilities within Veeam Backup & Replication such as Entire VM restore and VM files restore. This may result in faster recovery capabilities when used with EMC Data Domain Deduplication Systems than Instant Recovery and File Level Restore operations.
 
Instant Recovery
  • This type of restore can be effected adversely by the aforementioned limitations of a Data Domain appliance, and also the type of VM being restored. Highly transactional VMs will require more IOPS from the Data Domain during the Instant Recovery than others. With this in mind you can expect to only be capable of running only a few Instant Recoveries simultaneously. Instant Recovered VMs that are started from a backup file stored on a Data Domain may react or start slowly as the majority of their read operations will be hindered by the Data Domain.
  • For VMware users it is highly advised when performing an IR that the user select to have virtual disk updates redirected to a high performance Datastore. This will improve performance by caching written blocks to low latency storage.
  • It is advised that if the VM is intended to be made permanent that the VM be migrated to production storage as soon as possible after the Instant Recovery has begun.

File Level Restores
When performing Veeam Backup & Replication File Level Restore (FLR) capabilities slow recovery times may be experienced.  During Veeam FLR recovery capabilities, a significant amount of read activity occurs when accessing the Veeam “service data” metadata for each individual file as the Veeam backup files are not arranged in sequence.  This read activity must be performed to determine the location of the data block(s) associated with each file during granular restore sessions.  This significant level of random access is not recommended with archive tier storage devices because they are designed for optimal performance with sequential read operations.  Veeam recommends implementing EMC Data Domain Deduplication Storage Systems as a secondary target for these use cases as the more random read operations, the slower the restore will be with EMC Data Domain Deduplication Systems.
  • The backup browser may take longer than usual to open if an increment is selected and furthermore by that increments distance from the full restore point.
  • Navigating between folders within the Backup Browser may take additional time as each folder’s content must retrieved from the backup file to display it.

Backup
  • Reverse Incremental performance will be very poor due to its highly random I/O.
    Note: When the Backup Job is configured with a DDBoost repository, Veeam Backup & Replication will prevent Reversed Incremental from being selected by the user.
  • Synthetic Full creation will be very slow to a Data Domain, unless using DDBoost.
  • Synthetic Full with Transforms are not advised.

Backup Copy
  • A retention longer than 30 is not advisable as restore operations will diminish in performance.
  • The Health Check option may take a very long time as it is performing a read operation.

Replication
  • Using the Datadomain to store Replica metadata is not advisable.
    Note: Veeam Backup & Replication will prevent the user from selecting a DDBoost repository.
 
Veeam Backup & Replication Configuration

Parallel processing (global option):
This option significantly accelerates the backup process and decreases the backup window since virtual disk data is gathered simultaneously.  It also dramatically increases fragmentation in the backup files causing high random read for any restore operation. The greater volume of VMs or disks processed simultaneously will increase fragmentation and result in slow restore times. You may consider disabling Parallel Processing, this will decrease backup performance, but increase restore performance.

Storage optimization (job option):
Setting the storage optimization to Local 16TB+ has been shown to improve the effectiveness of Data Domain’s deduplication. The larger this value is, the smaller the preparation phase will be for a backup task and less memory will be used to keep storage metadata in memory.  However, the larger value may cause slowness in aforementioned restore operations due to the fact that Veeam will need to read entire blocks (ex. 4MB, the largest block size available in Veeam Backup & Replication v9) even for requests kilobytes in size. This results in redundant portions of data being read but not needed for restore.

Inline-deduplication (job option):  
Since EMC Data Domain Deduplication Systems have excellent hardware deduplication and compression capabilities, it is highly advised that Veeam built-in deduplication be disabled to decrease load on the backup proxy.

Decompress backup block before storing(repository option):
Veeam strongly recommends enabling this option so that raw data is sent to the EMC Data Domain Deduplication System, leveraging its global deduplication and compression capabilities. Leaving Veeam compression enabled may significantly impact EMC Data Domain deduplication capabilities resulting in high load and slow backup jobs.
 
Backup Mode
  • For CIFS/NFS presented repositories Forward incremental mode with periodic Active full backups is recommended to avoid the rehydration penalty during synthetic operations.
  • For DDboost enabled repositories Forward incremental mode with either Active full or Synthetic full backups is recommended. Synthetically produced full backups will generally have the best restore performance and reduce the time VM is run off of a snapshot during the backup job run. However in some environments an Active Full job may run faster.
  • Transforming previous backup chains into rollbacks is not advisable for both repository types.
  • For forever forward incremental backup and backup copy on DDboost enabled repositories, the option “Defragment and compact full backup file” should be enabled if available. In most cases a weekly schedule is appropriate. This helps to avoid excessive growth of pre-compression data size for the full backup file.
 
Repository Performance Expectations and Configuration
 
If DDBoost is not licensed on the Data Domain system it must be added as a CIFS type or Linux type repository. It is advised to use a Linux server with the volume mounted via NFS as a relay server to help improve performance. Under some circumstances, CIFS or NFS communication may perform better than DDBoost with Veeam Backup & Replication v8 because of the limitation of a single thread per backup job when using DDBoost. DDBoost has been shown to improve performance when performing Synthetic Fulls.
 
With Veeam Backup & Replication v9, support for EMC Data Domain Boost is enhanced with the introduction of the following capabilities:
  • Support EMC Data Domain Boost 3.0
  • Reduced impact of storage fragmentation during restore operations even with enabled parallel processing. This feature allows Veeam to store the VM backup in the dedicated backup chain so that fragmentation ratio will be minimum.
  • Reduce the impact of the block size so you may define any block size without impact on the restore process. Veeam will be able to read data granularly so amount of the redundant will be minimum.
 
With DDBoost
 
If the Data Domain System is licensed for DDBoost please proceed to configure it using the following steps.
 
  1. Launch the creation of a new Repository, on the Type tab select Deduplication storage appliance.
    http://helpcenter.veeam.com/backup/80/vsphere/repository_launch.html
    http://helpcenter.veeam.com/backup/80/vsphere/repository_type.html
  2. Select the deduplication storage as EMC Data Domain.
  3. On the next tab configure the information to for connecting to the Data Domain appliance.
    Very bottom on this page:
    http://helpcenter.veeam.com/backup/80/vsphere/repository_server.html
  4. On the Repository tab click Browse and select the necessary location from the list of available paths. http://helpcenter.veeam.com/backup/80/vsphere/repository_repository.html
  5. The default settings can be taken for the last steps in repository configuration.
    Unless your environment requires you to specify a different vPower NFS Server.
 
Without DDBoost

I. CIFS
  1. Launch the creation of a new Repository, on the Type tab select CIFS
    http://helpcenter.veeam.com/backup/80/vsphere/repository_launch.html
    http://helpcenter.veeam.com/backup/80/vsphere/repository_type.html
  2. On the next tab configure the path to which the Repository will write to, and set credentials to access that share.
    Second section named Shared Folder on this page:
    http://helpcenter.veeam.com/backup/80/vsphere/repository_server.html
  3. On the Repository tab within the advanced section, enable “Decompress backup data blocks before storing”
  4. The default settings can be taken for the last steps in repository configuration.
    Unless your environment requires you to specify a different vPower NFS Server.
 
II. NFS
The Data Domain will need to be configured for NFS access, and configure a Linux server to mount the volumes from the Data Domain via NFS. Please refer to the following links for further information regarding connecting Linux to the Data Domain via NFS:
http://forums.veeam.com/veeam-backup-replication-f2/veeam-datadomain-and-linux-nfs-share-t8916.html
http://tsmith.co/2014/veeam-and-datadomain/
 
  1. Launch the creation of a new Repository, on the Type tab select Linux
    http://helpcenter.veeam.com/backup/80/vsphere/repository_launch.html
    http://helpcenter.veeam.com/backup/80/vsphere/repository_type.html
  2. On the next tab select the Linux server that we be connected to. If it is not present in the list select “Add New…”
  3. On the Repository tab specify the path on the Linux server that leads to where you mounted the Data Domain via NFS. On this tab in the advanced section enabled “Decompress backup data blocks before storing.
    http://helpcenter.veeam.com/backup/80/vsphere/repository_repository.html
  4. The default settings can be taken for the last steps in repository configuration.
    Unless your environment requires you to specify a different vPower NFS Server.
 
 

Deduplication Appliance Best Practices

$
0
0

Challenge

Performance or deduplication ratio is low when backing up to a deduplication enabled storage appliance from Veeam Backup & Replication. 
Please note that following article does not pertain to ExaGrid Storage or EMC DataDomain. If you use either of these please see the corresponding articles:
If you are using ExaGrid Storage, please see https://www.veeam.com/kb2056
If you are using EMC DataDomain, please see https://www.veeam.com/kb1956

Cause

The default options within Veeam Backup & Replication are intended for non-deduplication storage.

Solution

It is advised to configure the job as following when backing up to a deduplication appliance:

  • Use “Incremental”
  • Uncheck “Enable synthetic fulls” 
  • Check “Perform active full backups periodically”
  • Uncheck “Enable inline  data deduplication”
  • Change the Compression Level to “Dedupe-friendly”
  • Change Optimize for to “Local target (16TB+ backup files)”
Veeam Backup & Replication v8

User-added image
User-added image



 

More Information

  Forever Forward Incremental, Synthetic Full, and Reverse Incremental operations will perform poorly with deduplication devices because the backup files that have been written to the storage previously must be interacted with during these operations. The act of reading existing data on deduplication device is slow because each block requested must be rehydrated and uncompressed to be read. With DataDomain deduplication appliances, however, it is possible to leverage DataDomain's own mechanism of updating file blocks' metadata internally, and consequently achieve greater performance with Synthetic Full. New full backup files can then be created without physically moving data into the file, but rather by synthesizing them from the existing data.
 
The process of rehydration and uncompressing during read operations from the deduplication storage will also impact the performance of all restore operations.
 

Contact your storage vendor to inquire about further specific details to optimize Veeam Backup & Replication to work with their storage solution.
 
 


Installation Fails with Patched Database

$
0
0

Challenge

When installing Veeam Backup & Replication, selecting an existing configuration database fails due to the following error message:
 

Unable to use database VeeamBackup, because it was created with a later version of Veeam Backup and Replication.
 
OR
 
The Configuration Database Connection Settings utility is unable to select a database due to the following error message:
 

Unable to use database VeeamBackup, because it was not created with current version of Veeam Backup and Replication.
 
In older versions of the software, or if the configuration database location is changed via the registry, no error message appears, but the Veeam Backup Service fails to start.

Cause

When Veeam Backup & Replication is updated, the configuration database is upgraded and its version number is changed. The updated database is not compatible with the Generally Available version of the software.
 
For example, Veeam Backup & Replication v9 GA (9.0.0.902) is upgraded to v9 Update 1 (9.0.0.1491), and then uninstalled. In this scenario, attempting to reinstall from the v9 ISO using the existing configuration database will fail. This is because the database version is greater than the latest version supported by the software.

Solution

Recommended Method:
 
If Veeam Backup & Replication is already installed, apply the latest update.
 
If it is not installed:

  1. Choose a new database name during installation. An empty database will be created with this name.
  2. Once installation has completed, apply the latest update.
  3. Switch from the empty database to the original configuration database using the Configuration Database Connection Settings utility.
 
To delete the empty database after installation, use SQL Server Management Studio.
  1. In Object Explorer, connect to the instance containing the empty database (such as localhost\VEEAMSQL2012).
  2. Expand Databases, right-click the empty database, and then click Delete.
  3. Confirm the correct database is selected, and then click OK.
 
 

Legacy Method:
 
The version number of the configuration database can be reverted with the following query:
 
Use VeeamBackup
UPDATE [Version] SET current_version = <see table below>
 
Build numbercurrent_version
6.5.0.109633
7.0.0.6901179
8.0.0.8171870
9.0.0.9022754
 
Once the version number is reverted, it is possible to install using the existing database. However, the software cannot be used in this configuration: it must immediately be updated to the same or later version that was previously installed. Otherwise, data corruption or unexpected behavior may occur.
 
Certain updates alter the database schema in a way that requires an additional change to the Version table before the update can be installed:
 
Updatecurrent_version
v7 Patch 4 (7.0.0.871)1200
v8 Update 3 (8.0.0.2084)2022

 
Do not attempt to revert to a previous release of the software (such as to v8 from v9).

More Information

How to apply a SQL script 
 
If there may be other problems with the configuration database, try restoring from configuration backup. It is recommended to create a configuration backup prior to updating the software.
 
ISOs containing updated versions of the software (maintenance releases) are not available. For more information on this topic, consult the Community Forum.


 

SureBackup SQL Script Verification

$
0
0

Challenge

Port 1433 is not enabled on the SQL server or there is a firewall preventing the server from establishing a connection to port 1433. SureBackup jobs will fail if they cannot establish a connection on the correct port.

Cause

Port 1433 is disabled on the SQL server or there is a firewall preventing port 1433 from being used to communicate with the SQL server.

Solution

The SQL Server Configuration Manager provides basic configuration management for SQL Server services, server protocols, client protocols and client aliases.

On the SQL Server Instance you will need to ensure the TCP/IP Protocols is listening on port 1433 for the TCP port. You may also need to list this port for the TCP Dynamic Ports.

If you have changed this port for some reason, you will need to deploy a custom script for the verification section in SureBackup. After performing the script change in production, please perform another backup job for verification.

More Information

Confirm the status of this port by performing "netstat -ban" on the masqueraded SQL server, from the command prompt.

For information on deploying a custom script please review this Veeam article: 
http://www.veeam.com/KB1312

Veeam Backup & Replication - SureBackup Step-by-Step Configuration Guide

$
0
0

Challenge

SureBackup is intended to automate and simplify the backup verification process — one of the most crucial parts of data management and protection.  SureBackup lets you validate backups of your VMs without impacting the production infrastructure. You can automatically verify every created restore point of every VM and ensure that they will function as expected in case a disaster strikes.

Solution

The following step-by-step instruction will help you to configure an Application Group, a Virtual Lab and a SureBackup job.

{iframe src="http://api.veeamuniversity.com/apps/stepbystep.php?did=1818" width="1086" height="786"} {/iframe}

More Information

For more info on Surebackup you may want to check out our helpcenter.veeam.com

Release Notes for Veeam Backup & Replication 9.0 Update 2

$
0
0

Challenge

Release Notes for Veeam Backup & Replication 9.0 Update 2

Cause

Please confirm you are running version 9.0.0.902 or 9.0.0.1491 prior to installing this update. You can check this under Help | About in Veeam Backup & Replication console. If you are using partner preview build 9.0.0.773, you must upgrade to GA build 9.0.0.902 first by installing Day 0 Update > KB2084 (Note: Please do NOT install Update0 from KB2084 over Update1. Please see the bottom of this article for the link Update2)

After upgrading, your build will be version 9.0.0.1715

Prior to installing this update please reboot the Veeam server to clear any locks on the Veeam services and when the reboot is done, please stop all the Veeam jobs and services before applying the update.

Ensure that Veeam service account has the db_owner role on the Veeam configuration database, so that the required database updates can be made during the first service start up after update. Additionally, either give Veeam service account the sysadmin role on the SQL Server hosting the configuration database, or just the right to view system table content (GRANT VIEW SERVER STATE TO "DOMAIN\ACCOUNT"). All of the above-mentioned roles and rights can be revoked after update is completed.

After installing the update, during the first start of the Veeam Backup Service, required modifications will be made to the configuration database automatically to optimize its performance. These modifications may take up to 10 minutes to complete. Please do not reboot the Veeam server, or attempt to stop the service during this operation. If there is concern regarding the time that the Veeam Backup Service takes to start after upgrade, please contact Veeam Customer Support.

Once Veeam Backup Service starts, please open the console and allow Veeam Backup & Replication to update its remote components.

If you have Enterprise Manager installed on a separate server, you need to install the update on it as well.

To obtain this update, please click here (you need to be logged in to download the update).

Solution

As a result of on-going R&D effort and in response to customer feedback, Update 2 includes over 300 enhancements and bug fixes, the most significant of which are listed below:

New platform support
  • Cisco HyperFlex HX-Series support for Direct NFS backup mode.
  • EMC Unity support for Backup from Storage Snapshots and Veeam Explorer for Storage Snapshots functionality.
  • EMC Data Domain DD OS 5.7 support for DD Boost integration.
  • ExaGrid 4.8.0.351.P28 is now the minimal ExaGrid firmware version supported.
  • NetApp Data ONTAP 8.3.2 support.
  • VMware vCloud Director 8.10 support.
  • VMware VSAN 6.2 support.

Engine
  • Backported a number of isolated Enterprise Scalability enhancements from 9.5 code branch to improve transaction log backup, tape backup and user interface performance.
  • Updated OpenSSH client to version 7.2 to enable out of the box support for modern Linux distributions.
  • Improved iSCSI target performance (iSCSI target is used to mount backup remotely in certain file-level and item-level recovery scenarios).
  • iSCSI mount operations are now retried automatically to workaround occasion “The device is not ready” errors which happen when mount operation takes too long. By default, the mount is retried 6 times every 10 seconds. To change the number of retries, create IscsiMountFsCheckRetriesCount (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the backup server.

Backup Copy
  • To reduce backup server load, Backup Copy jobs targeting shared folder or deduplicating appliance backed repositories with the gateway server setting set to Automatic selection will now start the data mover on the mount server associated with the backup repository (as opposed to the backup server). In cases when the mount server is unavailable, the data mover will be started on the backup server as before.
  • Backup Copy performance should now be more consistent due to preserving backup files cache when the job is switching to idle mode.
  • Minor reliability improvements in GFS full backup creation algorithm.

Microsoft Hyper-V
  • Find-VBRHvEntity cmdlet performance has been improved significantly when used against Hyper-V cluster.
  • Backup infrastructure resource scheduler should schedule Guest Interaction Proxy resource dramatically faster in large infrastructures (for example, 5 seconds instead of 15 minutes).

Microsoft SQL Server
  • Improved performance and reduced resource consumption of Microsoft SQL Server transaction log backups.

Microsoft Exchange
  • Added ability to force CAS server for Veeam Explorer for Microsoft Exchange (instead of automatically detecting one) via DefaultCASServer (DWORD) registry value under HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key on the backup server.
  • Added ability to change the order of Exchange autodiscovery policies for Veeam Explorer for Exchange (support only setting).

Oracle
  • Added ability to restore Oracle databases while preserving certain parameters which are critical to RMAN in scenarios such as database name change via new <Parameter Name="diagnostic_dest" StartupPath="true"/> parameter in PFileParameters.xml file of Veeam Explorer for Oracle.

Cloud Connect Replication
  • Support for Planned Failover functionality with cloud replicas. You can now perform planned failover to achieve zero data loss, for example when a natural disaster can be predicted in advance.

Veeam Cloud & Service Provider Partners
  • Update 2 introduces important changes and fixes around rental licensing, including a pilot functionality of usage reporting directly from the user interface. For additional information, as well as the list of other service provider specific enhancements and bug fixes included in this update, please refer to the issue tracking topic in the private VCSP forum. If you are VCSP but don't have access, please apply to Cloud & Service Providers group using Veeam forum's User Control Panel.

More Information

Ensure that Veeam service account has the db_owner role on the Veeam configuration database, so that the required database updates can be made during the first service start up after update. Additionally, either give Veeam service account the sysadmin role on the SQL Server hosting the configuration database, or just the right to view system table content (GRANT VIEW SERVER STATE TO "DOMAIN\ACCOUNT"). All of the above-mentioned roles and rights can be revoked after update is completed.

If you have Enterprise Manager installed on a separate server, you need to install the update on it as well.

To obtain this update, please click here (you need to be logged in to download the update).

No disks have been mounted…Windows FLR

$
0
0

Challenge

When attempting to perform a Windows FLR (Guest Files ‘Windows’) the restore fails with:

User-added image

No disks have been mounted. Either disks of this type aren’t supported or some errors occurred. See log file for details.

Solution

Confirm that the VM that the files are being restored from was using one the following filesystems.        
  • FAT
  • FAT32
  • NTFS
  • ReFS*
 ──────────────────────────────────────────────────────────
Troubleshooting Steps
──────────────────────────────────────────────────────────
 
1.  Clear all temporary files from “C:\Users\<serviceaccount>\AppData\Local\Temp\”

Note: When the Veeam Backup Service account is ‘local system’, the temporary directory used is C:\Windows\Temp.

2.  Open an Administrator Command Prompt and run the following commands:
(The installation path for Veeam if it was upgraded from 6.5 is: C:\Program Files\Veeam\Backup and Replication\)
(With Veeam v9 the operation must be performed on a Mount Server under the path: C:\Program Files\Common Files\Veeam\Backup and Replication\Mount Service)
cd “C:\Program Files\Veeam\Backup and Replication\Backup”
vdk remove
vdk install

 
3.  Confirm that the account being used for the Veeam Backup Service has the "Perform volume maintenance tasks" right assigned to it. This can be located within the group policy (gpedit.msc) under:
Computer Configuration>Windows Settings>Security Settings>Local Policies>User Rights Assignment
4.  Reboot the Veeam server

 

More Information

To support restore of files that were on an ReFS file system the Veeam server needs to be running on Server 2012/Server 2012 R2.

If the server you are attempting to restore from has volumes with native 2012 data deduplication, Veeam must be installed on the same operating system and also have the “Data Deduplication” role enabled. The “Data Deduplication” role appears under "File and ISCSi Services" in server manager.


If the Windows FLR continues to fail after attempting the steps outlined in this KB, please attempt and Other-OS FLR. For more information regarding how to do that please review the following:

http://helpcenter.veeam.com/backup/70/vsphere/multios_restore_intro.html


 
Viewing all 4476 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>